Accessibility Insights for Session Replay — Would You Use This Feature?

[bruno-garcia]

The Sentry team is exploring adding accessibility insights to our Session Replay product so you can have a user-friendly UI/UX for all your customers. The goal is to help your company adhere to the Web Content Accessibility Guidelines (WCAG) international standards as well as common accessibility best practices (e.g., every page has an h1 heading).

We’re interested in understanding how important and valuable accessibility insights/problem areas are for your company. Do you currently use an accessibility tool? If so, which one? The W3C has an extensive list of different tools.

If you use a tool, we’d like to understand a little more. For example, do you run this tool in CI already? Or you check your site from time to time with Lighthouse instead?

At Sentry, the approach we’ll take is to verify accessibility of pages through the Session Replay recordings. This way letting you know of of problems in real-life usage of your site, as opposed to the being limited to the pages you managed to cover with tests in CI, or manually verify. This is particularly helpful in non-static sites, where things can render very differently depending on the data available to a particular end-user.

Lastly, how would you ideally like to be made aware of these accessibility issues (e.g., via regular digest email, widget in the Sentry product, configurable alerts)?

Any feedback in this area is much appreciated!

[bruno-garcia]

If you support dark/light theme and you run tools in CI, do you re-run the whole analysis in both themes (e.g: check dark text dark background)?

[Sweater-Baron]

My team would definitely use this feature and find value in it. I'm planning to add some automated accessibility testing, but having this feature in Sentry would probably remove the need a lot of that work.

As for being made aware of the alerts, I'd want it to work pretty similarly to how Sentry tracks detected errors. Maybe even tracking them as regular Sentry issues, but as its own filterable category? Like how there's currently "info"/"warning"/"error" levels, just adding a new "accessibility" level

[patricksmms]

https://github.com/dequelabs/axe-core is the standard tool for a11y checks. People use it in unit tests. Also it's possible to build browser tests (with selenium/playwright/cypress...), injecting the script on the page and running tests with it while navigating the app.

I like the idea of having similar reports from Sentry. A widget with these reports would do fine for me. Alerts could also be useful, but only if they could be well tuned. I can imagine many false positives and/or low priority alerts driving users to disable it.

[bruno-garcia]

That's exactly what we're looking into, through playwright right now, to run the replay through the browser while collecting the a11y reports:

• https://playwright.dev/docs/accessibility-testing
• https://github.com/getsentry/session-replay-analyzer/blob/a8985a3ba254c2894a431785537080ec2e8991f3/package.json#L24C6-L24C26

Understand it might be noisy, our first take is to display the results in the replay details itself, so no alerting. This should allow us to learn and get feedback while providing value to folks. Specially in terms of noise/false positives etc.

[bruno-garcia]

We have an alpha version that lets you run accessibility checks at specific points in the Replay:

For example, here I have no errors, no Rage or Dead clicks, but some work to do to make this accessible:

Is this something you find useful? Let us know!

[jgf5013]

I know it's a year and a half later but my company uses Sentry and this would definitely be very helpful!!

[bruno-garcia]

Unfortunately we had to remove this experiment.
It was quite far away from a tool you'd be able to rely on. To get all/most a11y issues.
And the usage of it was quite low.