Lessons Learned From Our Accessibility-First Approach to Data Visualization

Kent Eisenhuth at UXDX USA. Video: https://youtu.be/tvlw_Fvb-AY

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Why data visualization needs to work for everyone

[00:00:00] Host: I would like to welcome Kent Eisenhuth, a staff UX designer, who's going to give a wonderful talk about data visualization and accessibility. So please give him a round of applause.

[00:00:13] Kent: Good morning. Good morning. How might we represent data in ways that provide value and insights to anyone, regardless of their ability? How might we create accessible data experiences that meet people where they are? These are a few questions that have been top of mind for my working group, which is striving to make data accessible through charts, graphs and visualizations.

[00:00:43] This is an important challenge for my group, because if you think about the products I work on, dataviz is often prominently featured in them. For example, people who are using our news properties and Search are using visualizations to help them find answers to questions they're asking of the product. For people who use our consumer hardware devices, like mobile devices or wearables, it's very likely that dataviz is a part of their everyday lives. In fact, they're typically using it to monitor their physical activity or even track chronic health conditions. And for our enterprise customers, we're using visualizations to help them make better, more well-informed business decisions.

[00:01:29] The reason why we've used dataviz is because it taps into the human visual system. If we think about what the human visual system does, it enables us to subconsciously gather, accumulate and process information presented by the environment around us. This very capability enabled us to survive as a modern species for hundreds of thousands of years. With that capability, we can almost instantly and subconsciously spot trends, patterns and outliers in all of the images we see.

[00:02:10] If we look at this example, you might notice there are three visual patterns just jumping off this slide. One is defined by the size of the shape, another is defined by the use of borders versus fills, and another is defined by shape type. If we start to think about those visual characteristics, size, fills, borders, shape type, these are all examples of visual variables, or encodings, which are essentially the building blocks of all visualizations.

[00:02:46] Visualizations, of course, exploit that human visual system, and they enable us to glance at a visualization, a picture of data, and understand a very complex data set, one we otherwise might not be able to comprehend. Good visualizations will also lead us down the path of making better, more well-informed decisions. They can do other things too, like helping our teams gain a shared understanding of a very difficult and abstract problem space.

[00:03:15] Now, this is all well and good, and a lot of us can experience a visualization in its entirety. For some of us, this is going to be the experience. For other people, this might be the experience. And for some folks, this is going to be the experience. In all of these cases, we're now missing out on those glanceable details, the value, those insights provided by the visualization to those people who can see it in its entirety.

[00:03:47] According to StackExchange.com, we know that over 4 million Americans rely on assistive technology to consume web content. We also know, according to the CDC, that over 41 million people in America have some sort of disability. But then we also need to think about folks who are not using assistive technology. For example, there are about 300 million people worldwide with color-deficient vision; it actually affects one in 12 men. These are all conditions we need to consider when we're creating data experiences, to ensure that everybody has a seat at the table.

[00:04:23] I have a working group that's been working on this topic for about three years, and today I want to share a few lessons we learned along the way. Right out of the gate, I'm just going to say that this is a difficult challenge. Dataviz, at least doing it right and making something useful, is difficult. Creating a useful and accessible experience is also difficult. And when you start combining these two topics, we're facing a real tangled, gnarly mess of a design challenge.

Accessible chart color palettes

[00:04:51] I want to talk a bit more about useful visualizations, and let's first dive into thinking about people who are not using assistive technology but might have vision disabilities. Accessible chart color palettes are actually one of the most widely discussed topics in our Google-wide dataviz office hours program. Believe it or not, chart colors is a loaded topic. I want to look at this example of a digital well-being chart. I think we can all agree it fits the theme of digital well-being: the palette is cool, it's calm, it's collected. But the fact of the matter is somebody with a protanopia condition is going to see the chart this way, and you'll notice that the upper two segments in this donut are now pretty much impossible to distinguish from one another.

[00:05:46] Let's take that a step further. Somebody who can't see any colors is going to experience the chart this way. And now I would say: get the dataviz out of here, let's get back to a table, because this is pretty much a waste of screen real estate.

[00:06:00] So where do we go from here? How do we fix this? Well, we can start with web standards. For us, that means we follow WCAG standards, the Web Content Accessibility Guidelines, and those standards call for a minimum 3:1 contrast ratio between a color and its neighboring color. This is a palette that does just that, and I think we can all agree it's really bold. When we apply it to a visualization, it might look like this.

[00:06:29] This is all well and good, but if I were to tell you that the most important metric on this visualization is represented by the red segments at the bottom of each of the stacked bars in the series, I think you might want to guess again, because it's probably difficult to actually see that. We can no longer glance at this visualization and focus on the metrics that matter most.

[00:06:53] One thing we learned early on is we can do a little bit of a magic trick here. Imagine we place some outlines, or borders, around the upper segments and use lighter fills. That will enable us to remain standards compliant, but now you'll notice your attention is drawn right down to the metrics that matter most. So something we learned early on was that we can use a combination of borders and fills to draw focus, yet still create an accessible visualization that meets the visual design standards from WCAG.

Using more than color, and minimizing chart junk

[00:07:27] We're going to go back to the donut chart now and do a quick update on that, and you'll notice it looks a bit better now. But there's still a problem with this donut chart, and here it is: we're still really only relying on color to convey meaning, and someone actually has to look at each segment in this donut chart and visually map it back to its corresponding legend item. So it's important to use something other than color to convey meaning. That could be a texture, an icon, another visual encoding or an overlay of text, but it's important to do this so that we minimize our reliance on color.

[00:08:09] Sounds pretty easy, right? But a lot of times, if we just throw patterns on visualizations, sure, this might meet accessibility standards, but you have to ask yourself: is this still glanceable? Is it still readable? So how do we make these experiences accessible and useful? How do we minimize chart junk? For those of you Edward Tufte fans out there, it's a classic term.

[00:08:35] One way we might be able to do it is to get rid of the legend. We can think about how to overlay text and other imagery on top of the chart to explain the metrics. But we don't always have the luxury of being able to just place text around the visualization, like this one. Other ways we might consider are adding symbols or icons, and I would say do this sparingly, only in a chart that has a limited number of categories. Here we actually used a combination of fills and borders to draw focus to the metrics that matter. And the cool thing about this is, if I want to highlight something in this chart, you'll notice I can do that by bumping up the shade of color, so we're still drawing focus.

[00:09:19] But some charts are a tangled, knotty mess, and I'm sure we've all seen an overloaded line chart before, like the one on the left here. This is where the dataviz expertise comes in, to start thinking about other ways to tell the same story and visualize the same information. In this case, we can break up this complex chart into sparklines, or mini charts, where arguably we can all better read the individual trends. You'll notice we removed our reliance on color here, and we have text appended to each of them, all things that are going to be great for accessibility.

[00:09:54] You'll also think about status. For those of you who design dashboards, it's a pretty important metric, and we've been thinking about ways to combine outlines and filled icons, where the filled icons draw your attention to the items that matter most, those items that need your attention. Someone who's red-green colorblind can now scan this table and look at the icons that are filled, and they'll hopefully jump out at you.

Accessibility first leads to better charts

[00:10:19] But not all these challenges are straightforward, and we found ourselves taking inspiration from unexpected places. For example, we looked to nature to think about how we might visualize a weather system. We looked to halftone printing processes to think about how we might reimagine a heat map with dual encodings. And we also took inspiration from architecture, and the way people flow through a building, to think about how we might reimagine a progress indicator, in this case using an inner circle that meets the color contrast criteria, yet is animated and helps us understand that the chart is filling up rather than draining down to zero.

[00:10:59] As we started thinking about this, this is where we got the idea that thinking about accessibility first generally leads to a better outcome. It's easier to think about accessibility at the start of the design process than to retrofit these requirements and ideas to existing charts. As we started to do that, we realized we were creating experiences that, even for people who are fully sighted, get back to really exploiting those pre-attentive processing capabilities of the human visual system, all the things I mentioned earlier in my talk.

Keyboard navigation for complex charts

[00:11:32] Okay, let's move over to the assistive technology experience, and I'd love to share a few lessons learned on that end as well. Right out of the gate, lesson learned: it's important to build a diverse team of people to help you with this work. We partnered with people who use assistive technology in their daily lives. We brought in people with disabilities to help us in our design process, and this really changed our perspective and the way we approached these design challenges.

[00:12:03] Let's think about the assistive technology experience a bit more. Navigation is a difficult design challenge, especially with dataviz. How might we structure our chart so it's easy to navigate with a keyboard? This is going to be really useful for people with limited mobility or using assistive technology. Here's a visualization that was popular a few years ago: a visualization of a LinkedIn network. I want to do a bit of a design exercise here and ask you: how might we navigate this experience with the keyboard?

[00:12:36] The first thought that might come to mind is, well, you can tab through all the nodes, or people, in this network. But some people are influencers and have hundreds of thousands of connections. So we want to ask ourselves: is it responsible of us as designers to require people to tab through thousands of people, especially if they're only trying to interact with something that appears after this visualization on the web page? I think the answer is no.

[00:13:02] So how might we reorganize this? You'll notice that color is used to indicate different pockets of people in this network. These could be people who went to the same university, people who work at the same company, or people who belong to the same social circle. So perhaps we could think of a hierarchical navigation. Perhaps you tab to the visualization, tab through the different groups, select a group, and then tab through the individuals in that particular group. A lot less tabbing.

[00:13:31] Another thing we might think about is how we might leverage alternative keys. Maybe you could arrow through the visualization. We could flatten the experience: you could tab to it, select it, and then arrow through it as you wish. Now, if we think about a lot of network graphs and their core use cases, what they do well is help us spot the most influential nodes on the network, or in this case the people in your network who might be influencers or have the most connections. So perhaps we could use keyboard shortcuts that enable you to navigate only to the people who are very well connected, or more influential, in your network.

[00:14:09] These are three different ideas I just threw out there; the possibilities are really endless. And the way we would solve for this is right here: it's important to think about what questions people are asking of the visualization, and craft a navigation experience that helps them quickly find answers to those questions. Of course, at Google, in a lot of our products, we've been thinking about this a lot, and here you can see an example of keyboard navigation where we have synchronized brushing through a very complex dashboard, and we're using the keyboard to find out more about different categorical metrics.

Using text, screen readers and sound

[00:14:47] Okay, I've talked a lot about navigation. Let's move on to text, and this is actually one of my favorite topics. How might we leverage text to guide people through the experience? This is easier said than done, and it's going to be very helpful for people who can't experience the visualization or see it. How can we leverage text to provide a glanceable view of data, and how can we do it without introducing... Now, this is going to vary depending on the use case of the chart.

[00:15:16] We have visualizations that are more exploratory in nature. Typically these are used by a data analyst. These visualizations aren't going to directly answer questions, but they're going to unlock the next set of follow-up questions this person is going to ask in an investigation. This could be a root cause analysis of something. In this case, we might use text to prioritize affordances for data exploration.

[00:15:40] Then there are other charts where, as designers, we're pretty confident we know the questions people are asking of the data. In this case, we might be able to use text to uncover some insights, maybe highlight some interesting trends, unexpected dips and spikes in the stock's performance in this case, and of course give an indicator of how well this stock is performing at that point in time.

[00:16:04] Other visualizations are used in journalism, and in a lot of these cases the main takeaways are highlighted front and center for us. In these instances, it's important to do that, because the visualization is typically a supporting element in that article; it's supporting the overarching narrative. So in this case, we might be able to provide a bit of a summary of what's happening in this particular visualization. I'm really excited to think more about the potential of large language models and AI, and how they might be leveraged to help us understand what's happening in visualizations we cannot see.

[00:16:43] While we're on the topic of text, let's talk about the screen reader experience. As I mentioned before, we worked with people who use assistive technology in their daily lives. One thing we learned along the way is that when you're navigating through the data experience, it's always important that the screen reader is constantly orienting people within the data set, explaining where we came from, where we are now and where you can go, and at every step of the way highlighting what's happening in the data at that level of the experience.

[00:17:16] We also learned that it's really important to give access to the underlying data set, especially for people who are interested in data exploration. This is not only good for data integrity and transparency, but we also learned that a lot of people who use screen readers are actually very proficient at navigating a basic data table and pulling out some basic insights from it. I would say use this sparingly. This is not the silver bullet that's going to solve the problem, but it's something you could add to your toolkit and test.

[00:17:46] I also want to talk a little bit about how we might leverage other senses, in this case sound, when representing data. Data sonification is a really great tool: how can you play audio tones that help people listen to a data set and experience it in a really compelling way? If you're not familiar with it, data sonification is actually part of our everyday lives today. If you've ever called an elevator, when the doors open you probably heard a chime. Did you know that one chime means the elevator is traveling upwards, and two chimes mean it's traveling downwards? You may not have noticed, but that is a great example of data sonification, and it's part of the core experience. So what we're thinking about is how we can bring data sonification into that core experience, so it's not available only to folks who are using assistive technology, but available to us all.

Making the business case and building a community

[00:18:42] I've talked about a lot of lessons learned here in our actual work over the last few years, but I want to talk a bit more about making it happen. In the current economic climate, we're seeing a lot of DEI initiatives getting defunded, or even cut in some cases. So if this is something you believe in and you want to bring it to your organization, how do you do it?

[00:19:06] You can make a business case for this. Of course, we all want to be good citizens of the web and create accessible experiences, and for us at Google that's always been part of our core mission. But sometimes it needs more than the case for good citizenship to get the attention of business leaders. So, a few business cases. According to Annie Jean-Baptiste's book, there's a one trillion dollar market for people with disabilities in the consumer market, as of 2017. But there's also a liability that comes with this. If you've ever tried to pass a visualization through an accessibility audit, you'll know it will crash the audit; it will block it and it will be flagged. Giving access to the data isn't good enough, but thinking about some of the other techniques I shared is going to become important, and this can get the attention of business leaders in your organization.

[00:20:00] We've always been meeting compliance, but we've been thinking a lot about how we do that while still creating a thoughtful and useful experience. That takes a lot of capacity, a lot of brain power, a lot of extra effort; I hope you could see that through some of the challenges I highlighted earlier in my talk. I worked on a team in Cloud, and we were solving these problems there, and we started to realize there were other teams in our organization, working on other products, asking the same questions and trying to solve the same problems.

[00:20:33] So we huddled up with them and created our data accessibility working group, which we so lovingly refer to as DAWG [?]. We felt we could learn from each other. Let's not spend our extra time away from our day job reinventing the wheel. Let's leverage each other's research, learn from each other, and use what's already out there and proven within our organization. In doing that, we started to collect a repository of knowledge we could eventually publish to other people in our company, and even people in the outside community. The idea was that we really wanted to help them get started.

[00:21:09] In building a community organically like this, there were a few lessons learned, and this is applicable to an accessibility challenge or any challenge you can get behind as an organization. We felt it was important to learn from each other, but presentations only give you one-way feedback. We wanted to really empower people in our working group. We wanted them to engage, so we opened up space for brainstorming and for community contribution. Doing this over the past three years, as we started to talk about this topic more and see who our champions were within the group, we could identify who was actually going to help us publish something.

[00:21:52] We ended up publishing a whole plethora of resources internally: demos, assistive technology demos, design patterns, guidelines, resources that could help anyone get started. That could be the person casually drawing a chart in Slides or PowerPoint, through to the person actually deploying a production-ready visualization in products. And for folks here today in the external community, we actually wrote about this topic, packaged some of our top tips for getting started and published them on the Material Design blog, so I would encourage you all to go out there and check that out.

Design sprints, drawing and a parting thought

[00:22:29] A quick note about process, thinking about the spirit of this conference. Design sprints were very important in helping us problem solve here. We did a lot of rapid prototyping, so we could get some of our work in front of people who use assistive technology, learn about their experience, make a few adjustments, and then get it back and retest. So design sprints were really important, and rapid prototyping was really important. We did a lot of drawing, and that time at the whiteboard with my engineering partners inspired me to actually write a book on UX drawing. It's for anyone who goes into design sprints and might not be comfortable with drawing. It's a deep dive into the visual language of UX that can enable you to be engaged in the problem-solving process and solve a detailed problem like data accessibility, like we did. This week, for conference-goers, we're offering the book at a discounted rate with Wiley, the publisher, so you can head out to their site and use the promo code. You can also get it at a discounted rate at Amazon too.

[00:23:31] I want to leave you with a parting thought. As we think about the devices we design for and the surfaces we share data on, I want you to think about the core capabilities those devices ship with, and about the lessons learned today. Think about how we can leverage high-resolution displays to create a beautiful visualization that's not only useful to someone with a vision disability, but also useful to someone who is fully sighted. Think about data sonification and how we can leverage a device's audio playback capability. Think about how we talk about data and use text, and what the narrated response of a data experience could be. Think about haptic feedback. All these things will enable us to not only see data, but listen to data and even feel data. If we start thinking about these experiences more holistically, together we will all make a lot of progress towards answering the initial question I posed: how might we create accessible data experiences that truly meet people where they are?

[00:24:38] Thank you very much. I also want to give a shout-out to all of my colleagues, engineers, product managers, designers, researchers, and people who use assistive technology, who helped us through our design process. I'm really lucky to be here representing the body of work of a large number of my colleagues, so I want to make sure I acknowledge that as well. If you liked the talk, feel free to follow me on social media and we can keep the discussion going there. And finally, I'm going to leave you with all the lessons learned I covered today, so if you'd like to take a picture of this for your own records, you can. Thank you very much, everybody.

Q&A

[00:25:21] Host: Thank you so much, Kent, for an amazing talk. A lot of times when we think about accessibility we think about digital, but it's about thinking about all these other experiences where we might encounter them. I want to share a personal story. When I first started in a leadership role, we would look at some of our roadmaps and dashboards, and I'm red-green colorblind. So you can imagine what it's like for me when I'm looking at that dashboard, in a meeting with a bunch of stakeholders, and I have no idea if we're crushing it or if we're in trouble. So the empathy and care in your talk, thinking about how people interpret the data and the stories the data visualization tells, thank you so much. Amazing talk. I brought my friend out here to see if there are any questions. Okay, I'm going to throw it in the back. Nice. That was actually perfect.

[00:26:19] Audience: Thank you so much for a wonderful presentation and all of the insights you shared. Super awesome, I really appreciate it. I'm curious how your team is structured, and what knowledge the people on your team have around accessibility.

[00:26:41] Kent: Yeah, that's a great question, because this problem at least requires people with expertise from many different backgrounds. The way our organization is structured, we have a central accessibility team who's always advocated for accessibility on a lot of our products. That team consists of accessibility analysts, subject matter experts, and yes, people who have disabilities and use assistive technology in their day-to-day lives. So we worked with them a lot. On core product teams we had the typical triad of engineering, UX design and research, and product management. Together we worked with those accessibility experts and also brought data visualization expertise into the picture, to be able to solve some of those problems.

[00:27:29] The working dynamic was very fluid, as I mentioned before. First we started with standards, to understand what we needed to do to get to compliance. As we started working with those standards, we realized it was really easy to make an experience that wasn't useful, so we started focusing more of our energy on that. The cool thing about working at a place like Google, or a very large organization, is that there's probably somebody else simultaneously trying to solve the same problem. So for us it was a matter of looking to people within the organization who had a view across different products and could help us connect those dots and start to engage with those other teams.

[00:28:09] Of course, other product teams are going to be structured very similarly. But as we started gathering research, even test data from accessibility test sessions and usability studies, we could start to look at what we could leverage. Because three or four years ago there really wasn't a lot of information out there on this topic, we looked from within to start putting something out there, and that's what we were trying to do with our working group: help other people get started. But it really took some collaboration with other teams, and of course those accessibility experts I mentioned before.

[00:28:42] Audience: Thank you.

[00:28:43] Host: Yep. We just want to make sure we answer a question online too. We've got an online question here from Lindsey Defore [?]. She says: when it comes to accessibility for things like using specific colors or fill styles for data visualization, do you find it best practice to make that the standard or an optional configuration?

[00:29:05] Kent: Ah, so yes, we always strive to make that the standard if we can. If we start out by saying we're just going to resort to an accessibility mode, I feel like that becomes a junk drawer for features, if you will. So let's be a bit more thoughtful about how we apply these visual design techniques to the core visualization. I actually wrote an article, published in Smashing Magazine last year with a colleague, about this particular topic and thought process. I'm sure I could share the link through the conference organizers for anybody who's interested. It's called "Our Accessibility-First Approach to Chart Visual Design," and you can learn more about the thought process there, and why it is better to provide this for everyone rather than resorting to an accessibility mode.

[00:29:57] Audience: Thank you. Oh, hi, I'm Jason, and I was very curious about the data sonification you mentioned. At Google, are there sound engineers who create these sounds, and is there prototyping and testing for these sounds?

[00:30:14] Kent: Yes, we actually have a team that specializes in data sonification. It's wired up to line charts today, in a very few specific products, and they've done a lot of work with audio engineers to think about techniques for actually quantifying data through sound. When we first heard this, we thought we were going to hear these really nice, sparkly musical tones, but it's not that easy, because the scales don't actually match up. I can't speak to the testing environment quite as much, since I come from a design background, but I could circle back with the team who has worked a lot on that to provide an answer about the testing environment.

[00:30:54] Audience: Oh yeah, thank you. I'm definitely interested in that.

[00:30:59] Host: Awesome. And I think we're right at time, so, Kent, thank you so much.

[00:31:03] Kent: Thank you, thank you, everybody. It's a pleasure being here.