Automate accessibility testing with NVDA screen re

Question

I am working on implementing accessibility (for visually impaired individuals) for one of our web application. It need to be ARIA compliant. Right now we are testing our changes with screen reader manually. For example we have Tree control in our application. I open NVDA screen reader and then navigate through my Tree Nodes. NVDA screen reader speaks out

• Node XYZ expanded, (When I expand XYZ node with right arrow key)

• Node XYZ collapsed, (When I collapse XYZ node with left arrow key)

Along with the voice it also write down this text.

But all this is manual. Now we want to setup automated test cases for the same so that any regression bugs can be caught by are test cases. Do there exist any such tool which we can use to automate our test cases. Any direction will be helpful.

PS: Just for a sake of comparison. We have nunit to write test cases for c# application. After writing test cases we integrate them into our build process. Any breaking change is caught when we run the build. I am looking for something similar to test out our aria compliance and screen reader's behavior with our web application.

Answer 1

I don't know of any existing tools for testing screen readers, however, there are accessibility APIs that test websites and web applications.

axe-core from Deque Systems is widely used and well-supported.

I wrote a python package to run automated web accessibility tests that uses axe-core and selenium.

While it isn't quite what you are looking for, it does cover about 60% of accessibility guidelines, including aria roles and attributes. It should help with determining screen reader usability.

You could integrate axe into C#, similar to my python package and the Java package, also created by Deque.

I hope this helps!

Answer 2

It sounds like you're already performing some pretty good manual accessibility testing against your web application, which no automated testing tool is going to be able to replicate completely. That said, if you're looking to take care of any low-hanging fruit with an automated solution, like Kimberly suggested, there are several automated accessibility testing tools out there that you can relatively easily integrate into your existing web application's testing framework that might help you.

One such tool is Continuum, which doesn't have a C#-based library offering at the moment, but could be used in a separate testing framework to be run against your web application after it has already been built. This may be preferable depending on your use case, as code linters for accessibility aren't perfect and are highly language-dependent, whereas testing the HTML of your web application more closely matches the screen reader use case you say you're trying to test for. You could even integrate Continuum into your existing CI/CD process to make sure your application is tested during development as opposed to afterwards, to reduce your manual accessibility testing load.

Continuum has a few sample projects to get you started, depending on your technologies of choice. Free versions are available at webaccessibility.com if you're interested. Most of them are Java- or JavaScript-based at the moment.