The Ultimate Guide to Accessibility Testing

Understanding Digital Accessibility Standards

The international standards for coding and accessibility are the Web Content Accessibility Guidelines (WCAG), both versions 2.0 and 2.1. They are based on the following four foundational aspects, known as POUR, which are necessary for anyone to access and use web content:

1. Perceivable: Information and user interface (UI) components must be presented in a way that is easily processed by users. 
2. Operable: UI components and navigation must be operable by all users, such as with a keyboard rather than a mouse. 
3. Understandable: Users must be able to perceive and use a website, but they also must be able to understand it—the content and navigation cannot be beyond understanding.
4. Robust: Content must be robust enough that it can be interpreted by a variety of platforms, browsers, and devices, as well as assistive technologies. 
   
   

WCAG provides criteria for testing your content based on three levels of conformance—each with a progressively larger set of success criteria. Each higher standard is inclusive of the prior standard(s), comes with an increased effort to remediate and maintain, and can have an increased effect on the visual design.

The three levels of WCAG 2.0 conformance are:   

1. Level A: To meet this level, you have to meet 25 success criteria. For example, at this level, you can’t identify something by colors, such as in “Push the yellow arrow to proceed.” 
2. Level AA: To reach this level, you have to meet 13 additional success criteria—and those are on top of the 25 from Level A. An example of criteria at Level AA is meeting color contrast standards. 
3. Level AAA: This comes with an additional 24 success criteria (for a total of 62). Examples of achieving Level AAA success would be an enhanced level of color contrast or a removal of any element that blinks or flashes three times per second in order to decrease the likelihood of a user having a seizure. 
   
   

Additionally, WCAG 2.1 adds more success criteria as follows: 

• Level A: Five additional success criteria
• Level AA: Seven additional success criteria
• Level AAA: Five additional success criteria
  
  

Most organizations should aim to meet Level AA standards, because these are the most widely adopted standards by laws such as Section 508 of the U.S. Rehabilitation Act, the U.S. Air Carrier Access Act, and Canada’s Accessibility for Ontarians with Disabilities Act.

Step 1: Prioritize Your Digital Assets

One of the most difficult steps in planning your accessibility strategy is prioritizing your digital assets. When you consider the range of potential accessibility issues across your apps and web properties, where do you even begin? In order for your plan to be achievable, it must be manageable.
 

Start by making a list of your current digital assets and those planned for the future. These assets may include: 

• Website
• Mobile site
• Native apps
• PDFs
• Videos
  
  

Next, prioritize each property based on usage, litigation risk, how issues may impact users, your ability to remediate, and other factors. 

Step 2: Create an Accessibility Policy and Statement

Next, create an accessibility policy and statement for internal and external stakeholders that covers all of the ways that each of them may use your services. The accessibility policy should explain why accessibility is crucial to your company’s operations, and it should speak to your executives, employees, volunteers, board members, and other internal stakeholders. Executive support will be crucial for raising awareness and making accessibility a priority, as well as obtaining budget and resources. The policy should also clarify each team’s role in the process as well.

The accessibility statement, on the other hand, is for your customers, partners, vendors, and the general public. This statement provides the world with transparency into your company operations and explains your vision, mission, and priorities. 

Step 3: Create a Testing Schedule and Train Stakeholders

With your assets prioritized and a policy and statement written, it’s time to take action and create a schedule, which can vary greatly depending on where your website is in its overall development. If you’re starting a new website from scratch, make accessibility testing a priority from the start. 

If you’re integrating accessibility testing into an existing website, your teams will need to be prepared for considerable changes to the site and daily tasks. Your staff will likely need additional training to maintain and understand the value of your site’s new level of accessibility. For example: 

• Developers may need technical training on accessible web development ranging from semantic HTML to WAI-ARIA.
• QA staff may need training on how to incorporate accessibility testing into their manual and automated testing as well as site monitoring.
• The UX team may need training on inclusive design principles and guidance to update style guides.
• Marketing may need guidance on how the accessibility impacts the content they produce.
• Management may need training on how to help your company become fluent in digital accessibility regulations, guidelines, and challenges.
  
  

Making accessibility testing and awareness a core part of these roles can help ingrain the initiative into your organization so it becomes natural. 

Step 4: Communicate Your Plan Internally

Accessibility testing is a major undertaking, one that requires active participation from all departments. Buy-in from senior management is crucial for accessibility testing to be truly effective.

Start by sharing your web accessibility policy and plan with your core internal stakeholders, including staff, investors, board members, volunteers, and others who are integral to your company’s daily functions. Make the case for why web accessibility matters for your organization’s success and paint a picture of the impact it will have on the 15 percent of the global population who are estimated to live with some form of disability (including one in four U.S. adults who have a disability that impacts major life activities).

Then, walk each team through its specific role in making web accessibility a reality, so teams feel a sense of ownership and responsibility for achieving accessibility success. 

Step 5: Share Your Plan Externally

Lastly, share your plan and goals with your customers, vendors, and the general public. Explain in plain language how you plan to make your digital assets compliant with industry standards and create a dedicated webpage that outlines your accessibility statement. Regularly update this statement with your progress, and include a contact number and email address so that users can reach out with questions, concerns, or appreciation.

Even if your website isn’t yet perfect, it’s OK to communicate your accessibility statement externally. We consider this a best practice, and sharing it externally helps affirm your commitment to accessibility.

Now, it’s time to launch your web accessibility testing program with these four steps:  

Step 1: Automated Testing

Automated testing is an essential starting point for evaluating your site’s current level of accessibility. It lets you monitor key pages and conduct broad assessments of your whole site without having to look at every page individually. For a retailer with hundreds or thousands of templated product pages, automated testing is a good way to get started without feeling overwhelmed.

Here’s a checklist of considerations for choosing an effective automated testing solution:

• It tests your whole site, including login, shopping cart, user journeys, and other pages with dynamic content or functionality.
• It provides analysis of your accessibility status with real data to back it up.
• It provides results that are easily interpreted by all members of your team, including project managers, developers, QA testers, marketers, and your legal team.
• It easily integrates with your existing tech stack. 
  
  

There are many easy-to-use automated testing tools, depending on your digital assets, including the following: 

• For websites: UsableNet’s Automated Accessibility Testing Tool will scan your site and provide a report with a summary of the basic errors, breaking them down by type and location on the page.
• For mobile apps: Apple’s VoiceOver, Android’s technical guide, and Accessibility Scanner all can help spot accessibility errors in your apps.
• For PDFs: Adobe Acrobat provides user-friendly tutorials on how to create and verify documents for accessibility and how to fix issues when you encounter them.
  
  

Step 2: Manual Code Review

Although automated testing is an essential part of web accessibility, it doesn’t cover all WCAG criteria. Whether free or commercial, these tools can only take you so far—a significant portion of the success criteria requires manual review. For example, an automated tool can identify alt text, but it can’t tell you whether that alt text is relevant for accessibility.

UsableNet has broken down which WCAG criteria require manual testing and which will be covered by an automated process. If your team doesn’t have the in-house expertise to conduct a manual code review, you’ll need to consider working with an outside partner for this step.
 

Step 3: UX Review 

During user experience review, it’s important to consider accessibility. Although accessibility is often confused with usability, the two terms are quite different: 

• Accessibility generally refers to the adherence to standards, such as WCAG, when creating a website, app, or document. This can include everything from color and text to images and multimedia.
• Usability describes the actual experience of using the site, including user interface design.
  
  

The good news is that accessibility improvements can raise the level of usability for all users, regardless of disability. Another difference between the two is that usability testing can involve input from real people, not automation. This is where user testing comes in. 

Step 4: User Testing

User testing should involve both usability experts and those who use assistive technology in their everyday lives. Companies across the globe recognize that automated tools don’t always catch every WCAG issue, and they are responding to the rising demand from the disability community for user testing.

Additionally, user testing makes good business sense. Today, you wouldn’t launch a website without making sure it works on a mobile device, so why launch a site or other digital asset without sign-off from a community of people who will be using it?

To launch a user testing program: 

• Start in your own backyard. Look for local lighthouse groups or nonprofits who work with disabled communities.
• Bring in the experts. Make sure they can move at your pace and charge a price you can afford.
• Look inward. There may be people in your own company who are disabled and use assistive technology.
• Recruit your own testers using databases like Knowbility.
  
  

Step 1: Document the Results

Start by keeping detailed records of what was tested and when, as well as the standards that were used. Document where your site falls in terms of conformance (covered in Chapter 1) and the steps that you plan to take to make improvements. 

Provide instructions on how to reproduce the uncovered errors and suggest fixes. Above all, keep in mind the human beings behind the results and frame the results in terms of how they affect users.

It can be a challenge to find and address each error. There are tools available that present a side-by-side list of the errors and responsible elements in your code, which is incredibly helpful in the remediation process.

Step 2: Update Standards and Best Practices

Now that you’ve implemented and documented this process, it’s time to make it your regular operating procedure. Everything from development and design standards to templates and company policies should be updated accordingly. This builds accessibility into the very DNA of your business.

Lastly and above all else, continue to perform accessibility testing. As standards and user abilities change and evolve, it’s important to repeat these processes on an ongoing basis so you can provide the best user experience possible and shield your company from the potential of accessibility lawsuits. 

1. We don’t have full control over our website functionality because we rely on third-party tools.

Many organizations achieve functionality by using a variety of plug-and-play tools, such as for ratings and reviews, maps, shopping carts, photo galleries, and forms. Each of these components is run by its own company with its own accessibility policies.

If you don’t have the budget to invest in expanding your own digital infrastructure, you will need to find third parties whose products and values reflect your own commitment to accessibility. 

2. Our company has significant knowledge gaps.

You may have the best intentions for accessibility but lack the expertise to achieve it. You’re not alone—many companies have this challenge. If so, you may want to work with an outside accessibility expert.

Ideally, this partner will help you proactively build a process that goes beyond off-the-shelf automated testing tools and will leave you confident about maintaining accessibility as your digital footprint evolves.

3. There aren’t enough user testers.

Once you’ve made the decision to hire user testers, you need to find them. Rest assured—they are out there. But it can be difficult to quickly get a team in place and then compile and act on their feedback, particularly if you work on an agile release cycle.

Your goal should be to integrate user acceptance testing (UAT) into your verification and remediation plan. Make sure you have tools in place that allow you to quickly and easily implement the recommendations of your user testers.

4. Not everyone is on board with accessibility testing.

Although it might seem like an insurmountable challenge, it’s important for every staff member—from the office manager to the CEO—to have awareness of digital accessibility and an understanding of its importance to the organization. Designers who keep color contrast in mind will create more usable templates, content managers with knowledge of the impact of non-text content will write more accessible stories, and project managers who build accessibility into their schedules will keep everyone on track.

5. It’s too expensive.

The benefits of your organization’s new digital accessibility initiative can be broader than defending against lawsuit risks. Achieving and maintaining website accessibility can also: 

• Open your business up to the widest audience possible.
• Improve usability and SEO.
• Position your company as a thoughtful and socially progressive organization on the forefront of a more inclusive business model.
  
  

→ Use our free calculator to estimate your costs.

6. Automated testing is good enough.

Unfortunately, many organizations don’t realize that even when they invest in the most sophisticated and expensive automated tools, the tools can only find and flag a small percentage of accessibility errors. They may also display false positives, which will send you down a rabbit hole.

You will get the best value by combining an automated solution with testing performed by assistive technology users such as screen readers. People with disabilities make up a growing segment of users, so let them have a hand in making sure your digital assets are usable for them.

• Have they worked with clients in your industry? 
• Can they provide case studies or references?
• Will they let you see sample reports? 
• Do their reports provide useful, easy-to-interpret information? 
• Can you easily import their reports into your own systems?
• Once necessary fixes have been identified, what role can they play in remediation?
• After your fully accessible site and apps are up and running, will the third party help maintain them?
  
  

Although a combination of manual and automated testing is vital for achieving a fully accessible website, the testing process can feel challenging, particularly when your site has multiple releases or if you have numerous digital properties. But an accessibility platform like UsableNet’s AQA can help streamline your process with real-time dashboards and reporting, API and integrations, automated screen reader testing, and a variety of other features. 

Take Your Accessibility Testing to the Next Level with UsableNet

If you’re ready to ensure that your websites, apps, and other digital assets are fully accessible to people with disabilities, then you need a partner with unmatched expertise in technology and industry best practices. No matter what level of assistance you need, UsableNet can reduce your costs by establishing a streamlined process that includes:

• Conducting automated, manual, and user testing, and training your team members to do testing themselves
• Implementing a scalable automated testing technology to help with remediation and maintaining accessibility
• Testing for usability—not just accessibility
• Providing regular reports that speed up remediation and improve oversight
• Integrating into your current processes and technology investments
• Recognizing the complexity of the task at hand and providing a steady hand to guide you through it
  
  

Join the hundreds of brands that have achieved web accessibility by working with UsableNet. Let’s talk.