Testing Web Accessibility

Adrián Bolonio at UXDX Community: Eastern Europe. Video: https://youtu.be/-dBsgG7L2EQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Why web accessibility matters

[00:00:02] Hi everyone, and let's talk about testing web accessibility today. First of all, I'm going to introduce myself. My name is Adrián, I'm from Spain. I'm a team lead and a front-end developer at my company, and I'm a huge accessibility advocate. You can find me almost everywhere under my surname.

[00:00:22] Accessibility. I always get the same question about what this "a11y" is that we see all across the internet. A11y is the international acronym for accessibility, and this is because there are 11 letters between the A and the Y in the word accessibility. That said, I'm going to use this a lot in the presentation.

[00:00:46] Why is it so obvious in the real world that these four situations are completely wrong? In the first two there is a very, very steep ramp. There are stairs at the end of the ramp in the third picture, and in the fourth picture there's a ramp that is totally unusable. Why is it so obvious that this is wrong? Take a different situation: someone comes with a wheelchair, there is an accessibility button, perfect. This person presses the button, the door opens automatically, but surprise, there are stairs. It's so obvious that this is wrong. But when we move to the online world, this is not so obvious.

[00:01:30] Web accessibility is essential for developers, so as not to exclude anyone when we developers develop our applications. That means that our tools, our websites, every technology that we use is designed and developed for people with disabilities as well, so they can use them.

[00:01:54] Let's see an example. We saw some examples in the real world; let's see something in the online world. You go to an e-commerce website and you want to check your latest purchases, so you call support so they can help you. The typical answer could be: oh, you need to click on the button in the top right corner. Or maybe you want to change your email address or some settings in your profile, so they tell you you need to click on the button with the gear icon. But for blind people there's no such thing as a top right corner. There's no such thing as an icon that looks like a gear. We need to take this into consideration when we develop our applications.

[00:02:43] Let me give you some numbers. A couple of years ago we were around 7.6 billion people in the world, and more than 1 billion people live with some form of visual impairment. It could be using glasses, it could be color blindness, or it could be a hundred percent, which is totally blind. When we develop our online businesses we want to reach as many people as possible, so we cannot forget that this is our audience as well.

[00:03:11] I'm not going to talk about developing web accessibility today. I have a different talk, it's called "How does your website sound like?", and you can find it on YouTube. I'm going to talk about testing web accessibility: how can we make sure that what we develop is accessible? Automated tests are very cool, and they can free up your QA team, your testing team, from manual tests in every part of your application. But it's not magic. They cannot automatically make your site accessible. We cannot forget that automated testing can only detect 20 to 50 percent of the accessibility issues. We need to treat this automated phase as one part of a bigger testing process, as one step, where manual testing is as important as automated testing.

[00:04:03] I've created a very simple application in React with three tiny components. The first one is a button. The second one is a fake button, which is an anchor link with role button. Sneak peek: never do that. And the third one is an image. In the entry point of the application I put a bunch of errors, so let's see if we can find them, and how we can detect those errors with some automated test tools. I've divided this presentation into three different blocks. The blocks, for me, are what I use every day: I can test while I develop, I can test after I develop, and of course we're going to talk about the manual tests that I mentioned before.

Testing while you code

[00:05:05] Let's start with testing while you code. Since I'm using a React application to show you some tools, the first one I'm going to use is react-axe. The axe family is a family of tools created and developed by Deque. You can install it as a developer dependency in your application, using npm or yarn depending on the package manager that you use. You need to export this library only if you are using an environment which is not production, so you only expose these logs there. And of course you use React DOM, and you pass it through the axe engine.

[00:05:58] What is this library doing? When you execute your application in the browser, in the console log of your browser (in this case I'm using Chrome), you're going to see all the issues that your application has, with some kinds of features, let's say. One is that if you have the same issue several times, it's going to be grouped. Those groups tell you what the error is. For example, it says elements must have sufficient color contrast, because it's not WCAG AA compliant, or images must have alternate text. They also give you a severity level, from minor, moderate and serious up to critical, so this gives you more information about the priority of when you need to tackle these issues.

[00:06:55] It also gives you the information of where the error is. It gives you the HTML element in your code where the error or issue is appearing. And I think the main feature of this library is that it gives you a direct link to dequeuniversity.com, which is the official documentation from Deque, to help you learn more about the issue and also to help you fix it. It gives you all the information and all the details about how to fix this issue.

[00:07:29] You can also use linters. For those who don't know what a linter is, a linter is a config file where you put some rules, and you say: let me know when I'm doing something wrong, tell me when I'm making a mistake on this rule. You can use a plugin for accessibility. You need to configure it in your ESLint configuration file, the config file that I was mentioning before. You need to include it in your plugins in line number 2, and in line number 12 we say "extends", extending the recommended rules of the accessibility plugin. With this line alone, you would have all the rules. I prefer to extend my rules to define some parameters, which is why I have some of the config parameters in the rules section.

[00:08:30] And of course, as with every linter, it's going to show you in the code editor (in my case I use Visual Studio Code) when you have an issue. It will be red underlined, and when you hover with your mouse it's going to tell you what the issue is. It also gives you the same information in the terminal when you run the application. It's very useful to avoid shipping code that is not accessible during the development process.

[00:09:02] The third one that I want to show you, which is probably my favorite, is jest-axe. Again we are back to the axe family. And Jest, for those who don't know it, is a JavaScript library to create unit tests. In this case you can use jest-axe to create accessibility unit tests. You need to install it as a developer dependency, the same as the other ones, and you can create a simple test. The one that I have created imports the whole application, the whole app from the App.tsx file. I expect not to have any violations, obviously. What I'm doing, using react-dom/server, is to transform my whole application into a string, and then I'm throwing the string into the axe engine, and of course again I'm expecting not to have any violations.

[00:10:01] When I go to my terminal and do npm run test or yarn run test, I can see the tests running, and I can see pretty much the same information that I showed with react-axe in the console. I'm going to see what the error is and where the error is in my HTML. It gives you a bit of information, and of course again this link directly to dequeuniversity.com, which is again the official documentation. It's a very, very good library.

Testing after you code

[00:10:33] Those are the libraries, the tools that I use while I develop. But sometimes you can arrive at a company with a mature project, and they tell you they've never done any accessibility work on this one. Or maybe you think that you're done with your code, and you want to be able to test the whole DOM structure of your application. In this part I'm going to show you some tools that you can use in the terminal, so you can pass the engines or the testers over a whole HTML page or over a whole website.

[00:11:18] The first one, back to the axe family: they developed a CLI, so you can use it in the terminal. In this case you need to install it globally on your machine with the -g flag, and you can run it easily in the terminal: put axe followed by the URL that you want to test. In this case I'm testing the Stack Overflow website. It's going to start a headless Chrome instance and it's going to perform all the tests. What is it going to give you? The typical thing from axe again: what the error is, where the error is. It gives you a trace of the error and again the link to Deque University, so you can learn a bit more about it.

[00:12:03] A very similar tool is Pa11y. Again, the same as before, you need to install it globally on your machine. It really works the same as the axe one: you put pa11y followed by the URL that you want to test, and it runs the same tests. It runs a headless browser, I don't know in this case what they use, and it gives you what the error is. One more thing that they give you is the WCAG principle that you are violating. It gives you the whole trace in the HTML DOM, and it gives you the exact element that is violating this principle. In this case, of course, it's not axe, so you don't have a direct link to the documentation. It's your duty to Google it and do your homework and find out how to fix those issues. You can also run this against your localhost, so you don't need to deploy anything to a test environment, or even to production, to be able to test it.

[00:13:06] Pa11y also has a very interesting feature, because sometimes I hear people say: look, I have 25 URLs, I don't want to go one by one to test all these URLs, and I don't want to put this into my processes. You can create a JSON file, a config file with some parameters. If you look at the JSON file that I created, I have three URLs. The first one is just a plain URL for Stack Overflow. For the second one I pass a timeout of 50 seconds, which is actually quite a lot, and I'm taking a screen capture, which you can use for reporting purposes, or maybe you have a visual regression test and you can use it to compare screenshots.

[00:14:00] But I think the biggest feature of pa11y-ci is that you can perform actions. You can click on an element when the element is loaded, and you can find this element through a CSS class or an ID, in my case, and navigate through a different part of the application, and I'm taking two screenshots. In this case you just need to run pa11y-ci. It's going to look in the root of your project to find this JSON config, and it's going to perform the three tests. In this case it has found three URLs. The first one is very quick, the second one fairly quick, and the third one takes a screen capture, then navigates to the other one, creates another test and makes another screen capture. The results in the terminal are pretty similar: again, what the error is, where, and what the element is that is violating the principle. You can see in the root the screen captures that were taken.

[00:15:10] Lighthouse. Lighthouse is an application or tool created by Google, and it is included in every Google Chrome browser, but you can also use it from the terminal. Maybe you want to include this in your release pipeline, for example, or your continuous delivery or continuous integration. In this case, again, you need to install it globally on your machine, and you can pass this parameter, which is --view. With this one, after the test it presents the results to you in the browser directly. In this case, instead of a headless browser, it runs a full instance of Google Chrome, and it does a lot of tests. It does performance tests, no-JavaScript tests, no-internet tests, accessibility tests, responsive design tests, so it creates a full report.

[00:16:19] At the end you end up with an HTML report. You can click on accessibility, and the same as the other ones, it gives you what the error was, a bit of information on how to fix it, exactly which element is causing the failure, and some links that lead you to the official Google documentation about accessibility in this case. I think I've mentioned the main features, in my opinion, of every tool that I showed you.

Manual testing with browser extensions

[00:16:54] How to test it: at the beginning I said that only 20 to 50 percent of the issues can be captured via accessibility automated tests, so manual testing is as important as automated testing. I'm going to show you some of the extensions that I use in Chrome (in my case it's Chrome) to manually test the accessibility of the websites that we create. Those are some of the ones that we use, and I'm going to show you them one by one.

[00:17:30] Going back to the axe family, they developed a Chrome extension. You can find it under the developer tools in your browser. You just need to click, and it's going to perform a full analysis of the website, in this case again Stack Overflow, and it gives you the information they were giving you in the terminal: what the error is, grouped by issue. In this case it gives you the information that you can get on Deque University directly embedded in the extension. Of course you have this "learn more" link that leads you to Deque University, but you can see almost everything now from the browser.

[00:18:20] A very similar extension is the ARC Toolkit. You can also find it under the developer tools. It does something quite similar: you can perform these tests, and you can see the families of tests that it performs. In this case I'm clicking on images, and I can see which images, for example, don't have alternative text. I can go directly to the element and it's going to highlight it in the window, so I can easily see which images are the ones that don't have the alt property.

[00:18:59] Accessibility Insights: this is a really good extension. One feature that I'm going to show you now is that it gives you a map of the tab indexes on your website. Why is this so important? Because normally people who are visually impaired don't use a mouse. They normally navigate through the website using the tab key. The tab order of your website needs to be consistent with the content of your website. You cannot jump from one place to another, so you need to be sure that the order is the correct one. With this application you can build this map and see if it's consistent or not. You keep on tabbing and it creates this map of the tab order, so you can see if you jump around, or: look, this element is out of order, or we need to skip this element because it's not really relevant. I think it's very beneficial to get a full map before shipping the website to production.

[00:20:15] WAVE. WAVE is probably the most complete extension that I have found so far. It's very quick. It's not found under the developer tools; I think you can find it in the extension bar, where it has its own icon. You click, and immediately you have the test already done. You can see directly in the window all the images that don't have an alt, the ARIA controls, every role that is not a correct one. You can see the structure of the headings, which is very useful for SEO as well. You can see the contrast, or the issues with the contrast if it's failing WCAG AA or AAA. You can deactivate all the styles, so you can see if the DOM structure, the HTML, is really consistent, and you can also inspect the HTML code in more depth. I think this tool is quite complete.

[00:21:13] But sometimes we want to put ourselves in the skin of someone who's visually impaired. NoCoffee is a vision simulator, and I think it is a very valuable extension. You can simulate how people who are visually impaired will see your website. You can simulate different kinds of impairments. For example, you forgot your glasses at home, and you'll see it like this. Or maybe someone has some glare in the eye, or you can see some snow in the eye. Or maybe color blindness, or protanopia, so the colors will completely change, or there are not even colors. You cannot always rely on red is danger or bad and green is good; you need to give some information as well. You can also simulate some real disabilities, like central vision loss, side vision loss or glaucoma. I think it is a very nice application to simulate how everyone will see your website.

Resources and closing thoughts

[00:22:41] Those are the tools that I normally use while I develop, while I do my work. There are more, and I think it's your turn now to dig and research new tools. If you find some that are valid and really nice, please send me a tweet with them, because I really want to test them. The application that I created for the first part of the presentation is on my GitHub, so you can find it there, clone it and play around with it. All the slides for this presentation, and any other presentation that I did, are on Speaker Deck, slash my surname, Bolonio. You can find them there. If you don't want to watch the presentation again and you prefer to read it, I've written a three-article series on my blog, so you can read it there, see the code, click some links and play around with it.

[00:23:56] I want to take advantage of this to recommend a really cool course. It's not a technical course. It's an introduction to web accessibility for everyone, and it's official from the W3C, on edX. I think they extended the period until September. I think it's a very nice introduction to the accessibility world, so you can do it, and it's for free. I encourage you to do it.

[00:24:26] I want to leave you with a couple of sentences that I think are very important, a couple of thoughts. Trenton Merced[?] once said: it's not just about the disabled user being able to access your website, it's about everyone being able to access your website. You don't know if you forgot your glasses today, if you don't have your mouse with you. If you have kids and you have your kid in one hand, you need to use a website with one hand only, with the tab key. Some months ago I dropped my computer, so my laptop screen was completely black, and I needed to perform some actions to buy some tickets. In this case, when you're not a disabled user but you are situationally disabled, you realize how inaccessible websites are. When you develop your applications and your websites, please be aware that you're not developing only for one set of people. You should develop for the whole audience. Everyone needs to be able to access your website.

[00:25:38] Because accessibility is not a feature, and disability is not an option. We cannot treat accessibility as a feature, as a post-process, "we will do it later", because later never happens. We need to treat accessibility as part of our development process, the same as we do with UX review, the same as we do with testing, with Selenium tests, end-to-end tests, the same as we do with specification. It needs to be inside our daily process, because it doesn't take longer to include an aria-label in a button with a non-meaningful name. It takes you 15 seconds to create meaningful text in an alt property on an image. It takes you 10 seconds to describe what you see in the image. If you have an image of a person using a computer, it takes you 10 seconds to say "a person using a computer".

[00:26:38] It is not a matter of time. It is not a matter of cost; it doesn't cost anything. HTML provides everything you need. Everything is standardized; the W3C is doing an amazing job here. It's free, it's quick, and it needs to be done. It should be a must. Just keep this in mind when you're developing your applications. And thank you.