Lessons Learned From Our Accessibility-First Approach to Data Visualisation

Kent Eisenhuth at UXDX EMEA. Video: https://youtu.be/hY-5_FnS0bA

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

How might we represent data for everyone

[00:00:00] Good morning, everybody. We're going to switch gears here a little bit, and I'm going to start by asking this question: who works with data visualization, charting, dashboards? Awesome. Who loves data visualization? Fantastic.

[00:00:15] I want to share a few questions that have been on our minds at Google. How might we represent data in ways that provide value and insights to everyone, to anyone, regardless of their ability? How might we create accessible data experiences that truly meet people where they're at? These are a few questions that have been top of mind for my working group at Google, which has been striving to make data accessible through charts, graphs and visualizations.

[00:00:47] This is really important to us because if you think about it, data vis is prominently featured in a lot of our products. For example, for the folks who are using our search and news properties, we're using data visualization to help them quickly find answers to the questions they're asking of those properties. For people who use our consumer hardware devices, our mobile devices, our wearables, it's more than likely that you're using data vis to track your daily activity, perhaps your steps, or even to monitor chronic health conditions. And of course, for our enterprise customers, we use visualizations to help them make better, more well-informed business decisions.

[00:01:30] Data vis has been a popular tool for hundreds of years, and that's because visualizations were designed to really tap into the human vision system, exploiting our ability to preattentively gather, accumulate and process information presented by the environment around us. With this capability we can nearly instantaneously spot trends, patterns and outliers in all of the images that we see.

[00:02:00] I want you to take a look at this example on screen here. I'd have to imagine that when you first saw this slide you probably saw three very distinct visual patterns jumping off the screen. The one on the left is based on the size of the shape. We have another that's based on the use of fills versus borders. And we have another one out on the right that's based on the type of the shape.

[00:02:26] Let's think about those visual characteristics for a minute: shape, size, fills, borders. These are all examples of visual variables, or visual encodings, which essentially are the building blocks to all charts, graphs and visualizations. And of course, these are those building blocks that tap into that human vision system, that enable us to have a glanceable view of a complex data set.

[00:02:53] If we think about what a good visualization does, it really leverages that vision system. It enables us to understand a data set we otherwise wouldn't be able to comprehend. We can instantly see trends, patterns and outliers in that data, and hopefully it can lead us down the path of making better, more well-informed decisions.

[00:03:12] That's all well and good, and if we look at this example we can probably appreciate the value, the insights that this map is providing. But for some people this is going to be the experience, and for other folks this is going to be the experience, and for people who are blind this is going to be the experience. In all of these cases we're now missing out on that glanceable detail, that value, those insights that were provided by the map to somebody who is fully sighted and can see it.

[00:03:43] According to stackexchange.com, over 4 million people in America rely on assistive technology to consume web content. But we also know there are millions of other folks who are not using assistive technology that we need to consider. For example, over 300 million people worldwide have color deficient vision, color blindness, and that's actually a condition that on average affects one in 12 men worldwide.

[00:04:06] It's important to think about these numbers, to think about the array of conditions, and to think about how we might create accessible data experiences that provide value to everyone, so everyone has a seat at the table.

A tangled, gnarly mess of a design challenge

[00:04:19] I co-founded a working group at Google about three years ago and we've been researching this topic over time. I want to share a few lessons learned along the way as we've been making our charts, graphs and visualizations accessible. Today, when I talk about accessibility, I'm not just talking about checking the old compliance boxes. This is extending beyond that and creating an experience that's actually useful.

[00:04:47] Right out of the gate we learned this is a really difficult challenge, and this is going to be just straight up here. If you think about accessibility, at least in terms of making something that's actually useful, that's pretty difficult and requires some niche knowledge. If you think about charting and visualization in terms of making something useful, that's also a difficult problem space. And now when you take these two topical areas, data and accessibility, and combine them, you're really facing this tangled, gnarly mess of a design challenge.

[00:05:18] First I want to talk about the 300 million people worldwide with color deficient vision, the folks that might not be using assistive technology, with an array of conditions that we should consider. I want to focus on this example. This is a digital well-being visualization, and I think we can all agree that the color palette here is calm, it's cool, it's collected, and it fits within that theme of digital well-being: nice cool colors.

[00:05:44] But the fact of the matter is that for somebody who has a protanopia condition, this is what the visualization is going to look like. You'll notice the top two segments in the donut, represented by the Other category and the YouTube category, are now nearly indistinguishable from one another. Let's take it a step farther. For someone who can't see color at all, this is going to be the experience, and this visualization is just really difficult to read. Even as a data vis person, I'd say get it out of here, let's just get back to a data table, because it's still going to provide more value than that.

Color contrast, and a bit of border trickery

[00:06:18] Where do we start? Web standards are a great starting point to solve for this, and we've always used the WCAG, the Web Content Accessibility Guidelines, to do that. Let's start with color contrast. That can at least help a bit here, and if we were able to use a standard compliant palette such as the one on the screen and apply it to a visualization, a visualization might look like this. Now when we think about color standards, it's important that all colors achieve a minimum 3:1 contrast ratio with their neighboring color, and that can cause problems with charts and visualizations.

[00:06:51] Here's an example of the previous color palette applied to a stacked histogram, and I think we can agree this is a bit bolder than the previous digital well-being example. But if I were to tell you the most important metric here is represented by the red segments running along the bottoms of the stacked bars in the series, these represent events that are a bit higher in priority, maybe a bit more important to you as a user. I think we can all agree that it's difficult to see that, because now we have three colors that are almost equally competing for our attention.

[00:07:25] I'm going to do a bit of a magic trick here. It's a bit of a hack. I'm actually going to outline the top bars with a contrast ratio compliant color, and as I do that you'll notice now your attention is drawn down to those metrics that matter more. It kind of feels good. I can see those alerts again.

[00:07:47] One thing that we learned out of the gate, and why I'm showing this so early on, is that you can still remain standard compliant but draw focus to what matters in your visualization by being a bit more creative and using a combination of borders and fills. It's really important that the visualization shows everything we need to show so that it's readable using those contrast standards, but you still have some creativity to draw your attention around, and we did this through borders and fills.

[00:08:13] All right, let's go back to our digital well-being chart. We're going to fix that up real quick here. Here's where we started, and now the contrast ratio is there, we did some of the border trickery that I had mentioned earlier, and we are on our way. But there's actually still a problem with this donut chart, and the problem is that color is the only method for conveying meaning in this visualization. It's our only encoding, and it's important to use something other than color to convey meaning. In fact, it's a requirement.

Dual encoding without the chart junk

[00:08:47] I'm just going to zone in here on the problem area. For the designers in the room, the first thing that comes to mind: how do we solve for this? Let's use textures. No problem, and here's an example of a chart that did that. But now, talking about becoming useful, you have to ask yourself, is this still a glanceable chart? It feels a bit harder to read, because with the textures it's adding some cognitive load here, and I feel like Edward Tufte might have an aneurysm when looking at a chart like this, because he would say this is filled with chart junk.

[00:09:21] So one of the questions we've been pondering is how might we make charts accessible while minimizing that chart junk. I want to show a few things that we've studied and a few ideas that we've incorporated into product. Let's go back to the donut chart example. We're going to add that dual encoding, and we're going to do it with something very simple here: we're going to incorporate the legend right into the visualization. So now the text is our dual encoding, and you can see that it just points to the respective segments on the donut.

[00:09:50] Now for a lot of us who build dashboards and have to think a lot about information density, we don't always have the luxury of having the screen real estate available to do something like this. So in certain data sets where we might have a limited set of categories, we resorted to iconography to do the trick here, using basic shapes, and again thinking about that idea of using a combination of fills versus darker borders to draw focus. That's really helpful when you want to enable somebody to focus on something that's a bit more important.

[00:10:22] I think the skeptics in the room might say this is great, but how many charts actually have only two categories? There are charts with hundreds of categories, so how do we handle that? In this example you can see this tangled mess of a line chart on the left here, and we thought more about some of those core data vis principles. Let's just separate the lines and the categories out into their own mini charts: small multiples and sparklines. Even for somebody who's fully sighted, I would argue that it's actually easier to read the individual trends in the example on the right. And by the way, we no longer have to rely on color, and we have text appended to each of these examples, something that's also beneficial for accessibility.

[00:11:03] For folks who are red green color blind, we thought a lot about iconography and how we can use fills to draw focus to items that matter in a table. We also thought, in Fitbit, about how we might make more relatable charts. For example, this is showing a heart range over a 24-hour period, and in the way that we're representing the ranges in accessible form, you'll notice we have these threshold lines. If you think about what we're showing along the x-axis here, it's time, so each dot has a bit of a visual rhythm, and the higher your heart rate zone, you'll notice the closer the dots are together. It's b-boom, b-boom, b-boom.

Looking outside design for inspiration

[00:11:39] Now not every visualization is as easy to think about, and we started looking to different areas for inspiration outside of design to really think about how we might want to solve for this. For example, we've looked to nature to think about how we might create a more accessible weather visualization. We looked to halftone printing processes to think about how we might create a more accessible heat map with a dual encoding. And we even looked to architecture, taking inspiration from flow through a building, to create a more accessible donut chart that shows the full scope of the chart and the direction in which the data is moving. Is it filling up, or is it draining down to zero?

[00:12:18] What we realized is that it's much easier to create these designs when we think about accessibility first than it is to retrofit these requirements to an existing design. We're calling this our accessible first approach to visualization. At least in this particular example, as we started to do this we realized we were creating visualizations that were really starting to tap into those core preattentive processing capabilities that I mentioned at the beginning of the presentation. This is an example of how thinking about accessibility first can really net out in a better result for everyone.

Structuring a chart for keyboard navigation

[00:12:55] All right, so I've talked a lot about visual design. This was actually one of our most commonly asked questions in our Google-wide data vis office hours program in 2021 and 2022, so I wanted to start there. But let's also think about assistive technology, because that's important too. One of the things we learned right away was the importance of building a diverse team. Include people with disabilities in your design process, and think of ways in which you can invite them in to co-design solutions. This really changed our perspective in how we're creating data experiences.

[00:13:29] So a couple of lessons learned in this area. How might we structure a chart so that it's easy to navigate with a keyboard? This is really important for people who are using assistive technology, and especially important for folks with limited mobility. I'm going to show this network graph. This was a visualization of a LinkedIn network, and it was available a few years ago. If I were to ask you as designers, especially the data vis folks in the room here, how might we navigate this with a keyboard? I think one of the first ideas that pops into mind is let's just tab through all the nodes in this network, nodes representing people.

[00:14:05] Now think about a LinkedIn network. Some people might have tens of connections, others might have hundreds, but then you might have some serious influencers who could have tens of thousands of connections. And as designers we have to ask ourselves, do we really want people to have to tab through 10,000 nodes just to interact with something else on the screen? I would have to imagine the answer is a resounding no.

[00:14:27] So how do we solve for this? Well, if we look at this particular graph, you'll notice that color is used to categorize subgroups of people. These could be people that went to the same school, they could be people that belong to the same trade organization, maybe people that went to the same conference like UXDX, or work at the same company. So we could think about structuring the navigation here more hierarchically. Imagine now we only have to tab through the different groups of people, select a group, and then we could navigate to the folks in that particular subgroup.

[00:15:01] So that's one way to think about it. We could also think about using alternative keys and flattening out the navigation scheme. Imagine tabbing to this visualization and then using arrow keys to just navigate from person to person within this network. If you think about what a network graph does very well, though, it's wonderful for highlighting the most influential people in the network, so maybe we could even include keyboard shortcuts that enable you to navigate to only influencers.

[00:15:31] So I just brainstormed out loud a few ideas here. We're not going to solve this today, but what I want you to take away from this is that it's really important to think about the use case for the chart, the questions people are asking of it, and it's important to structure that chart in a way so it's easy to navigate, in a way that enables you to find answers to those questions most quickly and efficiently. So considering using alternate keys and other methods for navigation is a must here.

Using text to give a glanceable view

[00:16:00] Visualizations are great for glanceability, so how can we use text to provide a glanceable view of data for people who can't see it? It's going to be really important for people who have this experience when trying to visualize data. Now I know gen AI is all the rage these days, and I think this is a really great use case for that, but how do we do this in a way where we're not introducing unintended bias? If you think about the use cases for a lot of different charts, there's a wide array we have to consider.

[00:16:29] For example, an analyst might use a chart in a root cause analysis or when investigating something, and in this case the chart isn't going to directly answer a question. It's going to help this person unlock the right set of follow-up questions to ask in that journey. So when we're thinking about gen AI, it could be thinking about ways to use prompts to prioritize data exploration. Same if we're not using gen AI: we could think about how we might use text affordances to prioritize data exploration.

[00:17:01] Some visualizations, we're pretty confident we know the questions people are asking of the data, and I think we can all relate to the good old-fashioned stock chart here. In these cases we have enough insight to know that people are generally interested in performance trends in this data, so now we can use text to start to surface some of those insights, some of those unexpected dips and spikes in stock performance. And in the spirit of gen AI, we already know the questions people are asking of the data because we're seeing those, and we already have an idea of what insights we might want to surface.

[00:17:32] Other visualizations are used in journalism, and in this case the main takeaway of the chart is highlighted front and center. That's because this visualization is likely being used as evidence that supports the overarching narrative in the article. So in this case we could be bold and likely actually summarize the chart using text.

The screen reader experience, and other senses

[00:17:52] Okay, so moving into the screen reader experience. We actually learned a lot here, and we learned this from designing with folks who use assistive technology. Always providing context is super important, especially when navigating a complex data set in something like a flow diagram or a network graph. Always show where you came from, where you are, where you're going, and provide an overview of the data at all of those different levels.

[00:18:15] For some data sets we also found, from folks using assistive technology, that they're actually quite good at navigating the source of the data, whether it's in a tabular format or a CSV, and being able to extract their own insights from that. Now this isn't a one size fits all solution, so I'm not saying just link to the data set and we'll call it a day. No. But if you do have an accessibility testing program in place, this might be something that you want to consider.

[00:18:43] And then finally, how might we leverage other senses when representing data? Data sonification, for example, is something that has been in our lives. It's using sound to represent data, and we actually do this in some of our charts in the assistive technology experience. Now I mentioned data sonification is part of our everyday lives, and in America a lot of our elevators, if you've ever traveled in one in a tall building, you'll notice you may have heard one or two chimes. One chime typically means the elevator is traveling upwards, two chimes means that it's traveling downwards. This is a part of our everyday experience. So something that we've been starting to learn more about is how can we leverage other senses when representing data, not only with assistive technology but just as a core part of the data experience.

The business case, and starting a working group

[00:19:29] Okay, so these were all lessons learned in our actual practice of creating accessible and useful visualizations. But how can we make it happen? Because in this climate we're seeing a lot of DEI initiatives getting cut or getting cut back, and oftentimes people are getting moved around to focus on different business priorities.

[00:19:49] So what's the business case for this? Well, going all the way back to 2017, according to Annie Jean-Baptiste, there's a $1 trillion market for people with disabilities. But I think this is a timely talk here in Europe, because with the European Accessibility Act we have to think about compliance as well, and this is a way to really kickstart an accessibility initiative. But I want to challenge everyone: compliance isn't good enough. How can we take it farther? Let's use this as a boilerplate, as a starting point, to enable us to think more about this.

[00:20:18] And as I got into data vis, at least accessible data vis, I learned very quickly that charts, graphs and visualizations will block an accessibility audit. Now I mentioned the tangled, gnarly mess of a design challenge we're facing in making useful and accessible data experiences, and at a larger company like a Google I also knew that there were other teams that were interested in the same thing, asking the same questions. So I would challenge you, if you work at a large organization especially, find out who else is thinking about the same thing you are. We started a working group so we could actually learn from each other, and as we started to learn from each other we're collecting information, collecting research, so we're spending our limited time and resources focusing on the questions that were unanswered, versus reinventing the wheel.

[00:21:05] One of the lessons that we learned was that as we started to accumulate information, we had different ways of engaging with our working group. That was through guest presentations, but it wasn't just a one-way information channel. We also incorporated design reviews and brainstorming sessions, and in all of these different forums we started to identify our champions, the people that were really passionate about this topic and willing to go above and beyond to contribute something. And once we identified that group, we were actually able to start publishing our collective research internally and even externally too. So if you work at a smaller organization and maybe you don't have the resources to work with a diverse team, you can leverage some of our articles that we published, for example on the Material Design blog, that will hopefully enable you to get started.

[00:21:52] And in the spirit of UXDX, we also learned a lot about process. We spent a lot of time at whiteboards. We were coming up with ideas and prototypes that we could put in front of people who used assistive technology, and we're coming up with different techniques to invite them into our process. Through this, this inspired a book that I ended up writing called Drawing Product Ideas, which helps people build confidence in their whiteboarding skills. We spent a lot of time drawing with engineers in this case, and it just created a lot of great content for a book, because we were able to use drawing to solve a really important problem like data accessibility. So if you're interested in the book, feel free, you can find it on Amazon. It's actually offered at a discounted rate right now, and you can read more about that.

A parting thought

[00:22:36] So I'm going to offer a parting thought here. We talked a lot about lessons learned today, and I want you to think about the products you design, the devices that are going to be shipping, the capabilities that they have, the surfaces that they feature. And I want you to think about the lessons that we learned today. How can we leverage AI? How can we leverage text to provide a glanceable view of data? How can we create multi-sensory experiences so people can not only see data but maybe listen to data, or even feel data through haptic feedback?

[00:23:07] And if we start thinking about these experiences more holistically, I'm more than confident that we'll all make a lot of progress on answering that initial question I posed, of how might we represent data in ways that provide value and insight to anyone regardless of their ability, and how can we create accessible data experiences that truly meet people where they're at.

[00:23:29] So I want to thank you all here. I also want to acknowledge my colleagues. I was lucky enough to be here to represent a wonderful body of work by a lot of trusted colleagues, a lot of researchers, subject matter experts across Google that really helped put a lot of these ideas together, so I want to make sure I acknowledge that. And then if you liked this particular talk, you can follow me on social media. I'm especially active on LinkedIn. And if you want to see just the collective lessons learned in a slide, snap a picture of it. I'm going to leave you with that. Thank you.

Q&A

[00:24:03] Host: We all know accessibility is good business, but what are your favorite tools or methods to actually check the accessibility of data visualization?

[00:24:11] Kent: We actually have a trusted tester program, and a lot of the folks in that program use assistive technology as part of their day-to-day lives. So that was something that we did early on: we really engaged with them, and that's where we started inviting some of those folks even into our design process, as core members of the team. That was probably my favorite method for getting people involved, because we were able to understand an array of different perspectives that really helped us create better, more usable experiences and useful experiences.

[00:24:42] Host: Fantastic. Just quick housekeeping: if we can keep our conversations to a minimum so everyone can hear the wonderful call. All right, let's get into a question from Rebecca. Is information architecture critical to accessible data visualization?

[00:24:56] Kent: Oh, the second one. Yes, I would think so, especially when thinking about how to organize and navigate the visualization. I showed the network graph example, and if you think of a network graph, a lot of times more of a flattened hierarchy comes to mind, because the idea is let's show the most influential nodes in this network and how they're connected to one another. But if you think about how to navigate that, with the keyboard for example, we found that there was actually a lot of value in flipping around and establishing some level of hierarchy. So again, really knowing those use cases, the way in which you would think of the IA of the visualization is going to be really essential, especially for that particular experience. So yes, 100%, I think it's absolutely crucial to really take the time and think it through, and also test those assumptions too.

[00:25:46] Host: Certainly. I like this question. Another one, from Chris: any tips on how to involve designers with assistive technology on a part-time basis? Great talk, by the way.

[00:25:59] Kent: Okay, it depends. I have a couple of answers to this question. I think it depends on your organization. If you're in a larger organization, we're lucky enough to have an accessibility lab where we can actually go in and use assistive technology. There's also some plugins for browsers where you can simulate that experience, or you could try to go get JAWS and NVDA, or use TalkBack and VoiceOver on your mobile devices, to start looking at that experience. And it's really important to do so, because again, that's something that really shifted our perspectives as we started creating these experiences: really understanding how it's going to run using these different assistive technologies and tools that are available.

[00:26:39] Host: Okay, great, another great one here. Do you check with users with visual disabilities whether the visualization is appealing, or do you just check the information can be read?

[00:26:47] Kent: We do a little bit of both, and this is a really interesting one. For us, knowing that the visualizations are useful, or supporting a larger critical user journey, is really important, helping people understand a difficult problem space or maybe an abstract challenge, all those things I mentioned earlier. Yes, absolutely, first and foremost we want to make sure that the visualization is readable, so that's really important. Visual appeal is pretty difficult, and sometimes I would argue it's even subjective. I've seen, just talking to some folks out in the field and stuff, sometimes there even is a will to put accessibility requirements aside from product because maybe the visualization isn't beautiful enough. And that's something that I think is really important, and that's why I wanted to spend a lot of time talking about chart colors and creative solutions, to really think about the standards but also create focus and visual appeal, because it's hard to balance those things and you do get a lot of pushback. So we do test the general sentiment of how does this visualization feel, what is the first impression, but also is it readable as well.

[00:27:52] Host: Fantastic. I'll go to maybe our second to last one. So I used to be a professor for data visualization, so I really, really like this talk. A lot of times you get into doing and using data, and it doesn't necessarily work great with storytelling, because people who have those skills maybe don't have the other ones. Are there any tools you recommend, and how do you use data and storytelling very well together?

[00:28:15] Kent: I'm a big believer in prototyping. If you can actually work with a real data set that either is the data set or is very likely to be representative of the actual data, you can understand the shape of that data, what the outliers are looking like, and understand what that's going to be. We work very closely with a UX engineering team that helps us with that. I do not want to get into the should designers code debate. I don't think that has to be a thing. But either you align yourself to someone who does, or you just try to mess around with the data set yourself, and you'll start seeing the shape of that data, and that can also unlock ways to improve some of the storytelling. And then again, I can't stress enough, just going back to those core use cases: why are we using the visualization to begin with, what decisions is it supporting? Really being vigilant about answering those questions first can really aid in the way you would tell a story that would help someone using your product.

[00:29:09] Host: Perfect, thank you so much, Kent. Thank you. A round of applause for everyone.