In June 2019, I conducted 5 user testing sessions for accessibility research with Fable Tech Labs, a Toronto-based start-up that’s “making it easier for digital teams to engage people with disabilities in product development.”

The goal of this initiative was to gather feedback from users with disabilities on a set of prototypes with navigation techniques for JavaScript web apps. There are multiple variations recommended in the industry for accessible, client-rendered page changes, yet very little user research on those methods. Therefore, we wanted to find out which techniques are the most ergonomic and intuitive to users with disabilities, and if any of the techniques presented barriers detracting from their browsing experience.

With this data, we can make better recommendations for accessible client-rendered websites in general and at the JavaScript framework level. By adjusting Gatsby’s implementation of client-side routing–currently leveraging @reach/router for React.js–to better support a range of people with disabilities, we can improve access for users and also potentially influence accessible UI patterns in other frameworks. Eventually (we hope), this work could encourage new browser APIs through web standards: “paving the accessible cowpaths” with solutions based in user research.

What is client-side routing?

In client-rendered JavaScript applications–also referred to as Single Page Apps–traditional HTML page reloads don’t typically occur when a user navigates through the app. Instead, client-side JavaScript handles routing through the app by controlling the browser’s history and mapping client-rendered URLs to each page or view. To ensure these experiences are accessible, developers have to recreate some of the missing browser feedback for users by manually managing focus and making announcements in assistive technology (AT).

Client-rendering in Gatsby

Gatsby builds static HTML pages with Node.js (a.k.a. “SSR”), which is great for both accessibility and performance: by default, pages will render without JavaScript and links will navigate through the site with traditional page refreshes like they should. With client-side JavaScript enabled, Gatsby scripts will download and hydrate the site into a full React.js web application that can manipulate the DOM and provide a smooth user experience. In this context, Gatsby sites must be accessible and use the best techniques possible, hence the desire for user research data to better support people with disabilities.

Common accessibility barriers in client-rendered web apps

These are some common accessibility barriers when it comes to client-side routing:

Without page refreshes, screen reader users may not be informed that the page has changed.

Without a page refresh or focus management, a user’s keyboard focus point may be kept in the same place as where they clicked, which isn’t intuitive. In layouts where the page changes partially to include a deep-linked modal dialog or other view layer, a user’s focus point could be left in an entirely wrong spot on the page.

Without visible focus outlines, sighted keyboard, switch, and voice dictation users can’t see their focus point on the screen when views change with focus management.

Some of the commonly recommended solutions include (references at the end of this post):

Dynamically set focus to an HTML wrapper element on page change, to both move focus to the new content and make an announcement in assistive technology. This pattern often uses tabindex="-1" on a DIV or other block-level element to allow focus to be placed on an otherwise non-interactive element.

Dynamically set focus to a h1-h6 heading element instead of a wrapper to move focus to new content and make a shorter screen reader announcement. This also typically requires tabindex="-1" to focus the heading with JavaScript in a cross-browser way.

Dynamically set focus to an interactive element like a button to put keyboard users on an operable button/UI control in the correct part of the app and announce it to screen reader users. The name of the button matters a lot here.

Leave focus where it is and make an ARIA Live Region announcement instead.

Reset focus to the top of the application (i.e. a parent wrapper element) to mimic a traditional browser refresh and announce new content in assistive technology.

Turn on focus outlines for keyboard and screen reader users while suppressing them for the mouse using CSS :focus-visible and polyfill or the What Input library.

Any combination of the above

In Gatsby–which uses React.js for rendering–we’re currently setting focus to an element wrapping the entire application with a custom implementation of @reach/router. But similar to some of the Reach demos, page content isn’t announced consistently in Safari and VoiceOver or NVDA and Firefox–two critical combinations for users of assistive technology. We’re actively working to improve this automatic accessibility support as we explore more component- and research-driven solutions.

By user testing a few variations of known routing techniques, we gained some valuable insight into how Gatsby and React can better support people with disabilities. We also uncovered some tips for improving usability of client-rendered apps in general, which we’ll cover below.

Working with Fable

We worked with Fable Tech Labs to conduct this research and found it to be a very productive process. I connected with Fable’s lead frontend developer Perry Trinier at the CSUN accessibility conference in early 2019. Fable’s generous offer to pair with us on this initiative has been wonderful, and we’re grateful for the opportunity to see this through.

The experience of testing with people with disabilities and the results that followed felt very important, and something product teams should seek out regularly. We’d highly recommend this process for anyone working on digital experiences; particularly if you can test prototypes to adapt to the findings early and often in design & development. User testing for accessibility can provide game-changing detail that you’ll be eternally grateful to have uncovered. Addressing accessibility issues sooner than later is both a smart business investment and can make your project more inclusive from the start!

Testing methodology

Fable’s online platform allows you to set up Meetings or Tickets for your sessions, depending on your desired outcome. In our case, we held 30-minute Meetings with Fable community members in Zoom.

Meetings are recommended for testing demos of a product with assistive technology, to co-design user experiences and conduct research, or to solve complex problems that customers have complained about. Tickets, on the other hand, are “like engaging a diverse team of QA analysts” to test usability and compatibility on important user journeys within your site. You can get video recordings of the sessions, which may come in handy for later analysis.

In our meetings, we tested with:

Sam using NVDA and Chrome on Windows 10

Ka using JAWS and Firefox on Windows

Yvone using screen magnification in Chrome on a Samsung Note 9, and ZoomText for Chrome on Mac

Vu with Dragon NaturallySpeaking and the keyboard in Chrome

Carolyn with switch access

In each of the sessions, we learned about the tester’s browsing method and in almost all cases was able to watch them browse through the prototypes. Asking a question to expand on a given detail provided lots of great insight (i.e. “tell me more about what makes that focus indicator confusing to you”). It helped to have the desired outcomes for the meetings defined and published ahead of time, so the testers knew what to keep in mind when testing and what questions they should try to answer.

Testing objectives: the techniques

We wanted to know the best approach for guiding users with disabilities through JavaScript-heavy web applications, which don’t use traditional page reloads to reset the user’s focus point and announce new pages in assistive technologies.

The prototypes for these tests covered some of the common techniques for enhancing the page loading experience for users with disabilities, each centered around the mechanics of loading new content. These techniques included:

Example 1: No focus change, with a Live Region announcement for screen readers

Example 2: Focus reset to application wrapper element, with a Live Region announcement for screen readers

Example 3: Focus changed to a heading element in the newly changed content

Example 4: Focus changed to an element wrapping the newly changed content

Example 5: Focus changed to a wrapper element at 100% width, rather than side-by-side (added during testing session #3 on zooming/magnification)

These prototypes used Doggo Ipsum for placeholder text, which made for some fun testing experiences. "Cuuuuuute long bois" was my personal favorite, captured in testing screenshots. 🙂

Testing results

Each tester had different needs and highlighted varied concerns, with some prototypes working well and others needing improvement. Some testers reviewed the links ahead of time and provided a few extra tips, which was generous and appreciated.

The prototypes were developed primarily for the desktop browsing experience, but through testing it became clear that responsive design plays a role in producing streamlined, barrier-free user interfaces, particularly for users of screen magnification. Horizontal scrolling, while unavoidable in a lot of cases, can be reduced through media queries and flexible layouts. These layout variations can also affect interaction design (i.e. focus moving down instead of across).

These tests were a good reminder that accessibility goes beyond support for screen readers, which make up a limited percentage of user needs. Support for screen magnification in web apps, for example, is very important to support a larger population of users with low vision. There are also more disabilities to consider than we were able to accommodate with this series: conducting more tests with diverse users would only improve the outcome of any recommended solutions.

Takeaways from screen reader users

A big motivation to test with screen reader users was to confirm whether the recommended focus management techniques actually worked for them. The prototypes included a few approaches specifically targeted at screen reader users, including:

Live Region announcements on view changes to inform screen reader users of changing content;

focus management sent to a wrapper element;

focus management sent to the first heading element.

Focusing on a wrapper element for new content worked okay with our test subjects: they were informed of the new content and moved into the right place. It was very subtle compared to focusing on a heading, and better than resetting the page to the top (think Gmail – it would be a pain to have to start over every time you click on a link). In general, we found that resetting focus to the top of the app would be very overwhelming, especially in large applications.

Focusing on a heading was found to be the best experience, as it would save time and make it clear what happened. In NVDA, there was some duplicate reading of content and <main> was doing some extra announcements–seemed like a screen reader quirk more than a problem with the prototype.

A back button would help to use browser history (this was a limitation of the prototypes).

Putting aria-current on links to indicate which one is active helps in applications. We made a note to try using aria-label instead of aria-labelledby on section elements to minimize duplicate announcements in NVDA and JAWS (when the section is labelledby a heading).

Add a heading to each major section of the UI, like navigation, even if visually hidden–as many screen reader users don’t navigate by landmarks but do navigate by headings.

The conclusion for these tests from one screen reader user when asked if they could recall any good examples of focus management was that “no one is doing this well.” We can do better!

Takeaways from users relying on screen magnification

Users with low vision magnify the screen and zoom into pages to see them better: screen magnification at the OS level, screen magnifying assistive technology like ZoomText, browser zoom, and more. In this user test, the participant found visible focus outlines to be helpful for orienting oneself in the application.

For sighted magnification users, our first couple of prototypes were quite useless as they relied on screen reader announcements, which were inapplicable in this scenario (the same would apply to a UI pattern like the proposed “toast” element). Despite having developed with accessibility in mind–including visible focus outlines–the prototypes pretty much fell apart when zoomed way in on a Samsung Note 9 phone with the Chrome browser.

For the focus management techniques, horizontal scrolling presented an issue without a mobile viewport: if focus was sent to a wrapper or heading spanning a width much larger than the screen, mobile Chrome would scroll to the middle of it and cut off the beginning and end of the text…making it illegible. This was exacerbated in our prototypes until we adjusted the design to fit a single column, but still wasn’t great to have a visible focus outline covering so much of the screen. Some of this seemed limited to Samsung’s Chrome browser and weren’t reproducible elsewhere, but even with slightly better scroll/focus behavior it illustrated the need for responsive pages that adapt to fit smaller screens without as much horizontal scrolling.