The Importance of Manual Human Testing for Accessibility

When you need to test your website for accessibility, you might be choosing between multiple experts who will perform an audit, or be using online tools on your own. Whether you are choosing someone to do an accessibility audit, or assessing your site’s accessibility via DIY tools, it’s very important to remember that you need to include manual human testing.

We are not suggesting that you avoid automated tools – quite the contrary. They can be amazing resources for testing your site. But you need to realize that they will not find ALL the accessibility issues on any given site and that you should include manual testing. Here are some reasons why.

Lack of comprehensive coverage

Automated or AI driven accessibility tools can be an amazing part of a toolkit for assessing accessibility of a site. For instance, you may want to use the free online WAVE (website accessibility evaluation tool) from WebAIM. Or if you are using WordPress, you might want to use the Accessibility Checker plugin from Equalize Digital. But these tools are simply not able to test for all types of issues. Automated tools are only able to find about 30% of accessibility issues.

For instance, while automated tools may be able to tell you whether or not you have alt text for your images, only manual human testing will be able to tell you whether you have good alt text given how a specific image is used in the context of the website. Additionally, automated tools will never be able to tell you whether you have keyboard navigation issues – can someone use keyboard shortcuts to navigate through your website.

Automated testing tools can be great for helping you identify content related issues in particular, but these errors would not be the only types of accessibility issues a website might have present.

---

False positives and false negatives

Automated testing tools can generate false positives, which are reported as accessibility issues but are not actually problems. They can also generate false negatives, which means that they may miss real accessibility issues (as described above). This can lead to wasted time and effort in trying to fix non-existent issues, or to real issues being overlooked. Manual human testing can help you sort out false positives and false negatives.

Lack of context

Automated testing tools can’t take into account the context in which a web page is used. Accessibility issues may only arise in certain situations, for example, when using assistive technologies or when navigating a page in a specific way. Only human testing can determine whether the tab order in keyboard navigation is logical and follows and expected path for a specific type of site and its content. Only manual human testing can identify these sorts of context-specific issues.

Inability to assess subjective elements

Accessibility is not just about meeting technical standards, it also involves ensuring that content is perceivable, understandable, and usable by people with disabilities. Automated testing tools cannot assess subjective elements such as the flow of meaning within content for your specific audience or the usability of navigation, which require human judgement.

Dynamic content and functionality

Automated testing tools may not be able to detect accessibility issues related to dynamic content or functionality, such as pop-ups, menus, and forms. For instance, they cannot determine whether a contact form has accessible error notices. These issues can only be identified through manual human testing.

Human judgement is necessary

While automated testing can be a useful tool for detecting certain accessibility issues and automated testing is used by all accessibility professionals, it should not be relied on as the sole method of testing. A combination of automated and manual human testing is recommended to ensure that all accessibility issues are identified and addressed.

Manual testing does not require the tester themselves to be disabled. A trained or educated tester or accessibility professional should be able to recognize if persons with disabilities of various types might have problems using features of a site. But a thorough assessment of website accessibility will always include manual human testing.