DISCOVER A POWERFUL ACCESSIBILITY TESTING TOOL.

The Ultimate Guide to Accessibility Testing

How to Ensure Your Websites and Apps Are Fully Accessible to People with Disabilities 

down

Introduction

No matter your industry, digital accessibility is imperative. Not only does a commitment to accessibility for people with disabilities open up your business to tens of millions of potentially untapped customers, but it can also help your business avoid costly lawsuits, boost SEO optimization, and so much more. 

To guarantee that your website, apps, and other digital assets and properties are accessible to as wide an audience as possible, you need regular and rigorous accessibility testing. 

In this guide, you will find the tools you need to create a comprehensive and successful accessibility testing program. This guide details key considerations for every step of the testing journey—from planning and code review to user testing and remediation.

The Ultimate Guide to Accessibility Testing is for you if: 

  • Your duties include establishing and maintaining accessibility for your business’s digital assets, such as websites, apps, PDFs, images, and videos.

  • You have a complex website that uses Java, CSS, dynamic content, and/or third-party functionality.

  • Your business is among those most frequently targeted by web accessibility lawsuits, including retail companies and the hospitality and restaurant industry

Download a PDF version of this guide by filling out this form, or keep scrolling to read.

Cover page of The Ultimate Guide to Accessibility Testin

Chapter 1

What Is Accessibility Testing?

Accessibility testing includes reviewing your website, apps, and other digital content against common standards and evaluating usability for people who use assistive technologies such as screen readers. The goal of accessibility testing is to capture this information, make improvements, and inform your digital initiatives.

Digital accessibility must become a part of every digital project you undertake. Because of this, testing for accessibility should be part of your regular processes—not a one-time project. As accessibility standards change and our shared understanding of user abilities evolves, digital experiences should also evolve to meet their needs.

Understanding Digital Accessibility Standards

The international standards for coding and accessibility are the Web Content Accessibility Guidelines (WCAG), both versions 2.0 and 2.1. They are based on the following four foundational aspects, known as POUR, which are necessary for anyone to access and use web content:

  1. Perceivable: Information and user interface (UI) components must be presented in a way that is easily processed by users. 
  2. Operable: UI components and navigation must be operable by all users, such as with a keyboard rather than a mouse. 
  3. Understandable: Users must be able to perceive and use a website, but they also must be able to understand it—the content and navigation cannot be beyond understanding.
  4. Robust: Content must be robust enough that it can be interpreted by a variety of platforms, browsers, and devices, as well as assistive technologies. 

WCAG provides criteria for testing your content based on three levels of conformance—each with a progressively larger set of success criteria. Each higher standard is inclusive of the prior standard(s), comes with an increased effort to remediate and maintain, and can have an increased effect on the visual design.

The three levels of WCAG 2.0 conformance are:   

  1. Level A: To meet this level, you have to meet 25 success criteria. For example, at this level, you can’t identify something by colors, such as in “Push the yellow arrow to proceed.” 
  2. Level AA: To reach this level, you have to meet 13 additional success criteria—and those are on top of the 25 from Level A. An example of criteria at Level AA is meeting color contrast standards. 
  3. Level AAA: This comes with an additional 24 success criteria (for a total of 62). Examples of achieving Level AAA success would be an enhanced level of color contrast or a removal of any element that blinks or flashes three times per second in order to decrease the likelihood of a user having a seizure. 

Additionally, WCAG 2.1 adds more success criteria as follows: 

  • Level A: Five additional success criteria
  • Level AA: Seven additional success criteria
  • Level AAA: Five additional success criteria

Most organizations should aim to meet Level AA standards, because these are the most widely adopted standards by laws such as Section 508 of the U.S. Rehabilitation Act, the U.S. Air Carrier Access Act, and Canada’s Accessibility for Ontarians with Disabilities Act.

"Most organizations should aim to meet Level AA standards."

2 women working at a desktop computer

Types of Accessibility Testing and Review

Conducting automated accessibility tests can help you immediately improve some aspects of website accessibility, but automated testing alone can’t help you meet WCAG or decrease the likelihood of potential lawsuits. Comprehensive accessibility testing requires a multi-layered approach that can reach every single asset in your digital ecosystem. 

To achieve accessibility for users of all abilities and maintain ADA compliance, organizations must leverage these four types of accessibility testing: 

1. Automated Testing

Start with automated testing, which quickly identifies the low-hanging fruit to remediate and improve accessibility. But that may not be enough to prevent lawsuits or help users of all abilities, automated testing lacks human interpretation and can miss vital accessibility issues.

2. Manual Code Review

UsableNet found that approximately 80 percent of WCAG 2.0 standards and 100 percent of the newly updated WCAG 2.1 success criteria require manual review. This type of testing involves real people with WCAG technical review knowledge and experience who can inspect the HTML, CSS, and Javascript to ensure it conforms to WCAG.

3. User Experience (UX) Review

UX review requires inspection of larger site design elements to test for usability and accessibility. Elements analyzed during UX review include: 

  • Visual structure
  • Logical page layout
  • Menu functionality
  • Button size

This process is particularly important for native apps and is extremely helpful for people with low vision or mobility and cognitive impairments.

4. User Testing

Whereas manual code review makes sure that your site meets WCAG, user testing lets real people with real disabilities—such as those who use screen reader technology daily—confirm that your site is usable for them.

In Chapter 3, we explain what proportion of accessibility testing falls into each of these testing buckets to help you know what to expect when testing. 

Chapter 2

Creating a Website Accessibility Testing Plan

Now that you understand the different levels of conformance and types of testing, it’s time to put digital accessibility at the top of your to-do list and create a plan. If you’re wondering what to test, who to involve, or how to communicate your intentions and goals, here are five simple steps for creating and launching your accessibility testing plan:

Step 1: Prioritize Your Digital Assets

One of the most difficult steps in planning your accessibility strategy is prioritizing your digital assets. When you consider the range of potential accessibility issues across your apps and web properties, where do you even begin? In order for your plan to be achievable, it must be manageable.
 

Start by making a list of your current digital assets and those planned for the future. These assets may include: 

  • Website
  • Mobile site
  • Native apps
  • PDFs
  • Videos

Next, prioritize each property based on usage, litigation risk, how issues may impact users, your ability to remediate, and other factors. 

Step 2: Create an Accessibility Policy and Statement

Next, create an accessibility policy and statement for internal and external stakeholders that covers all of the ways that each of them may use your services. The accessibility policy should explain why accessibility is crucial to your company’s operations, and it should speak to your executives, employees, volunteers, board members, and other internal stakeholders. Executive support will be crucial for raising awareness and making accessibility a priority, as well as obtaining budget and resources. The policy should also clarify each team’s role in the process as well.

The accessibility statement, on the other hand, is for your customers, partners, vendors, and the general public. This statement provides the world with transparency into your company operations and explains your vision, mission, and priorities. 

Step 3: Create a Testing Schedule and Train Stakeholders

With your assets prioritized and a policy and statement written, it’s time to take action and create a schedule, which can vary greatly depending on where your website is in its overall development. If you’re starting a new website from scratch, make accessibility testing a priority from the start. 

If you’re integrating accessibility testing into an existing website, your teams will need to be prepared for considerable changes to the site and daily tasks. Your staff will likely need additional training to maintain and understand the value of your site’s new level of accessibility. For example: 

  • Developers may need technical training on accessible web development ranging from semantic HTML to WAI-ARIA.
  • QA staff may need training on how to incorporate accessibility testing into their manual and automated testing as well as site monitoring.
  • The UX team may need training on inclusive design principles and guidance to update style guides.
  • Marketing may need guidance on how the accessibility impacts the content they produce.
  • Management may need training on how to help your company become fluent in digital accessibility regulations, guidelines, and challenges.

Making accessibility testing and awareness a core part of these roles can help ingrain the initiative into your organization so it becomes natural. 

Step 4: Communicate Your Plan Internally

Accessibility testing is a major undertaking, one that requires active participation from all departments. Buy-in from senior management is crucial for accessibility testing to be truly effective.

Start by sharing your web accessibility policy and plan with your core internal stakeholders, including staff, investors, board members, volunteers, and others who are integral to your company’s daily functions. Make the case for why web accessibility matters for your organization’s success and paint a picture of the impact it will have on the 15 percent of the global population who are estimated to live with some form of disability (including one in four U.S. adults who have a disability that impacts major life activities).

Then, walk each team through its specific role in making web accessibility a reality, so teams feel a sense of ownership and responsibility for achieving accessibility success. 

Step 5: Share Your Plan Externally

Lastly, share your plan and goals with your customers, vendors, and the general public. Explain in plain language how you plan to make your digital assets compliant with industry standards and create a dedicated webpage that outlines your accessibility statement. Regularly update this statement with your progress, and include a contact number and email address so that users can reach out with questions, concerns, or appreciation.

Even if your website isn’t yet perfect, it’s OK to communicate your accessibility statement externally. We consider this a best practice, and sharing it externally helps affirm your commitment to accessibility.

"Buy-in from senior management is crucial for accessibility testing to be truly effective."

Team of people in an office sitting in chairs listening to a man speak

Chapter 3

Start Website Accessibility Testing

In Chapter 1, we highlighted the four different types of accessibility testing: 

  • Automated testing
  • Manual code review
  • UX review 
  • User testing

The time you allocate to each of these will vary depending on the type of digital content you’re testing, as well as whether the asset is in development or live on the web. The chart below offers an example of what the testing and review process may look like for common digital assets, broken down by testing or review types:

 Digital Asset 

  % automated testing  

  % UX review 

 % manual code review 

 % user testing 

Existing website

20%

10%

50% 

20%

New website

10%

30%

40%

20%

Existing app

10%

60%

0%

30%

New app

10%

60%

0%

30%

PDFs

70%

0%

20%

10%

 

Now, it’s time to launch your web accessibility testing program with these four steps:  

Step 1: Automated Testing

Automated testing is an essential starting point for evaluating your site’s current level of accessibility. It lets you monitor key pages and conduct broad assessments of your whole site without having to look at every page individually. For a retailer with hundreds or thousands of templated product pages, automated testing is a good way to get started without feeling overwhelmed.

Here’s a checklist of considerations for choosing an effective automated testing solution:

  • It tests your whole site, including login, shopping cart, user journeys, and other pages with dynamic content or functionality.
  • It provides analysis of your accessibility status with real data to back it up.
  • It provides results that are easily interpreted by all members of your team, including project managers, developers, QA testers, marketers, and your legal team.
  • It easily integrates with your existing tech stack. 

There are many easy-to-use automated testing tools, depending on your digital assets, including the following: 

Step 2: Manual Code Review

Although automated testing is an essential part of web accessibility, it doesn’t cover all WCAG criteria. Whether free or commercial, these tools can only take you so far—a significant portion of the success criteria requires manual review. For example, an automated tool can identify alt text, but it can’t tell you whether that alt text is relevant for accessibility.

UsableNet has broken down which WCAG criteria require manual testing and which will be covered by an automated process. If your team doesn’t have the in-house expertise to conduct a manual code review, you’ll need to consider working with an outside partner for this step.
 

Step 3: UX Review 

During user experience review, it’s important to consider accessibility. Although accessibility is often confused with usability, the two terms are quite different

  • Accessibility generally refers to the adherence to standards, such as WCAG, when creating a website, app, or document. This can include everything from color and text to images and multimedia.
  • Usability describes the actual experience of using the site, including user interface design.

The good news is that accessibility improvements can raise the level of usability for all users, regardless of disability. Another difference between the two is that usability testing can involve input from real people, not automation. This is where user testing comes in. 

Step 4: User Testing

User testing should involve both usability experts and those who use assistive technology in their everyday lives. Companies across the globe recognize that automated tools don’t always catch every WCAG issue, and they are responding to the rising demand from the disability community for user testing.

Additionally, user testing makes good business sense. Today, you wouldn’t launch a website without making sure it works on a mobile device, so why launch a site or other digital asset without sign-off from a community of people who will be using it?

To launch a user testing program

  • Start in your own backyard. Look for local lighthouse groups or nonprofits who work with disabled communities.
  • Bring in the experts. Make sure they can move at your pace and charge a price you can afford.
  • Look inward. There may be people in your own company who are disabled and use assistive technology.
  • Recruit your own testers using databases like Knowbility.

"Companies across the globe recognize that automated tools don’t always catch every WCAG issue, and they are responding to the rising demand from the disability community for user testing."

Chapter 4

How to Interpret Accessibility Testing Results

Once you’ve completed your testing and review process, you must document the findings, get to the root of what’s causing each error, and remediate accordingly.

Step 1: Document the Results

Start by keeping detailed records of what was tested and when, as well as the standards that were used. Document where your site falls in terms of conformance (covered in Chapter 1) and the steps that you plan to take to make improvements. 

Provide instructions on how to reproduce the uncovered errors and suggest fixes. Above all, keep in mind the human beings behind the results and frame the results in terms of how they affect users.

It can be a challenge to find and address each error. There are tools available that present a side-by-side list of the errors and responsible elements in your code, which is incredibly helpful in the remediation process.

Step 2: Update Standards and Best Practices

Now that you’ve implemented and documented this process, it’s time to make it your regular operating procedure. Everything from development and design standards to templates and company policies should be updated accordingly. This builds accessibility into the very DNA of your business.

Lastly and above all else, continue to perform accessibility testing. As standards and user abilities change and evolve, it’s important to repeat these processes on an ongoing basis so you can provide the best user experience possible and shield your company from the potential of accessibility lawsuits. 

"Everything from development and design standards to templates and company policies should be updated accordingly. This builds accessibility into the very DNA of your business."

Chapter 5

6 Common Web Accessibility Testing Challenges and Myths

Any business can create a successful accessibility testing program, but there can be challenges—not to mention myths and misconceptions—that can arise around accessibility testing. Here are some of the most common concerns that come up during the accessibility testing process. 

1. We don’t have full control over our website functionality because we rely on third-party tools.

Many organizations achieve functionality by using a variety of plug-and-play tools, such as for ratings and reviews, maps, shopping carts, photo galleries, and forms. Each of these components is run by its own company with its own accessibility policies.

If you don’t have the budget to invest in expanding your own digital infrastructure, you will need to find third parties whose products and values reflect your own commitment to accessibility. 

2. Our company has significant knowledge gaps.

You may have the best intentions for accessibility but lack the expertise to achieve it. You’re not alone—many companies have this challenge. If so, you may want to work with an outside accessibility expert.

Ideally, this partner will help you proactively build a process that goes beyond off-the-shelf automated testing tools and will leave you confident about maintaining accessibility as your digital footprint evolves.

3. There aren’t enough user testers.

Once you’ve made the decision to hire user testers, you need to find them. Rest assured—they are out there. But it can be difficult to quickly get a team in place and then compile and act on their feedback, particularly if you work on an agile release cycle.

Your goal should be to integrate user acceptance testing (UAT) into your verification and remediation plan. Make sure you have tools in place that allow you to quickly and easily implement the recommendations of your user testers.

4. Not everyone is on board with accessibility testing.

Although it might seem like an insurmountable challenge, it’s important for every staff member—from the office manager to the CEO—to have awareness of digital accessibility and an understanding of its importance to the organization. Designers who keep color contrast in mind will create more usable templates, content managers with knowledge of the impact of non-text content will write more accessible stories, and project managers who build accessibility into their schedules will keep everyone on track.

5. It’s too expensive.

The benefits of your organization’s new digital accessibility initiative can be broader than defending against lawsuit risks. Achieving and maintaining website accessibility can also: 

  • Open your business up to the widest audience possible.
  • Improve usability and SEO.
  • Position your company as a thoughtful and socially progressive organization on the forefront of a more inclusive business model.

→ Use our free calculator to estimate your costs.


6. Automated testing is good enough.


Unfortunately, many organizations don’t realize that even when they invest in the most sophisticated and expensive automated tools, the tools can only find and flag a small percentage of accessibility errors. They may also display false positives, which will send you down a rabbit hole.

You will get the best value by combining an automated solution with testing performed by assistive technology users such as screen readers. People with disabilities make up a growing segment of users, so let them have a hand in making sure your digital assets are usable for them.

"[Your] partner will help you proactively build a process that goes beyond off-the-shelf automated testing tools and will leave you confident about maintaining accessibility as your digital footprint evolves."

accessibility testing

 

Chapter 6

Choosing an Expert Accessibility Testing Partner

If you’re considering working with a trusted third-party expert to facilitate your testing, here are some questions to consider: 

  • Have they worked with clients in your industry? 
  • Can they provide case studies or references?
  • Will they let you see sample reports? 
  • Do their reports provide useful, easy-to-interpret information? 
  • Can you easily import their reports into your own systems?
  • Once necessary fixes have been identified, what role can they play in remediation?
  • After your fully accessible site and apps are up and running, will the third party help maintain them?

Although a combination of manual and automated testing is vital for achieving a fully accessible website, the testing process can feel challenging, particularly when your site has multiple releases or if you have numerous digital properties. But an accessibility platform like UsableNet’s AQA can help streamline your process with real-time dashboards and reporting, API and integrations, automated screen reader testing, and a variety of other features. 


Take Your Accessibility Testing to the Next Level with UsableNet

If you’re ready to ensure that your websites, apps, and other digital assets are fully accessible to people with disabilities, then you need a partner with unmatched expertise in technology and industry best practices. No matter what level of assistance you need, UsableNet can reduce your costs by establishing a streamlined process that includes:

  • Conducting automated, manual, and user testing, and training your team members to do testing themselves
  • Implementing a scalable automated testing technology to help with remediation and maintaining accessibility
  • Testing for usability—not just accessibility
  • Providing regular reports that speed up remediation and improve oversight
  • Integrating into your current processes and technology investments
  • Recognizing the complexity of the task at hand and providing a steady hand to guide you through it

Join the hundreds of brands that have achieved web accessibility by working with UsableNet. Let’s talk. 

Chat with a Website Accessibility Expert

close chapters modal

Download a PDF version of this guide by filling out this form

Simply fill out this form to receive a PDF version of our guide.

Cover Page of The Ultimate Guide to Accessibility Testing