How to Design for AI Driven User Experiences

Shari Benko at UXDX Community: Accessibility, AI UX, and Feature Sunsetting. Video: https://youtu.be/np2RVv-LQng

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Poor usability, not AI, is the threat

[00:00:00] Shari: I'm here to present the AI UI tenets and traps, which are a tool for UX designers to help humanity avoid the downfall of human civilization as we know it today. Strong words, I know. But first, a little disclaimer, and I'm sure you all appreciate the little dark design pattern I have here: the font is a little too small for you to read the disclaimer, but since I'm a good UXer, I'll make it bigger. I am not an AI expert, and I would never claim to be one. I am a UX researcher, designer, leader and teacher. I have 15 years of design experience. I have a master's degree in interaction design, and the last eight years I've been able to do user experience research and design.

[00:00:39] Two of those years were specifically in the realm of AI. I was working with teams designing and building platforms to enable data scientists to build the algorithms. I shouldn't say build the algorithms; they were in charge of training models to do artificial intelligence. So I got to know data scientists really specifically, and from there I started to think about how UX can apply to the development and design of user interfaces that employ AI.

[00:01:19] I differ from most people in what I think will lead to the downfall of human civilization as we know it. I actually don't believe the statement that AI will be the downfall. There are many reasons why I don't think AI is to be as feared as most people believe, but that could be a topic for a whole other time. I could spend hours talking to you about this. But what I do believe is going to be the downfall of human civilization as we know and love it today is poor usability.

[00:01:45] I'll let that sink in a little bit. This is a stronger message than I normally provide when teaching about the value of including UX on your team, but frankly, my patience for poor usability is running a little thin these days. Lately I've felt that the products I'm using for work or at home are harder to use than they were, say, 5 or 10 years ago. And I was starting to wonder if it was just me getting older and crankier. I do go to bed at 8 o'clock at night, which is an indication that I might be getting old. But maybe it's just what I like to call the curse of UX: once you see it, you can't stop seeing it everywhere. Once you learn what bad design is, nothing seems like a good experience.

Feature cramming and levels of abstraction

[00:02:27] So I thought maybe it was just me, but it turns out I'm not really alone. The article referenced here details an interview with Jakob Nielsen and his thoughts on the subject. He thinks the same thing, but he attributes the cause of this to something called feature cramming. Because we have a psychological proclivity to want things that have lots of features, companies know that selling a product that has seemingly everything actually works. We will buy a product with a ton of things in it just because it has a ton of things. So they cram their designs full of features. But many research articles point to the fact that we as users typically only use about 20% of the features provided in a product. That leads to a product being crammed with things we don't need, bogging down the usability. So it turns out I'm not crazy. I mean, I might be still, but it's not the curse. This is actually happening.

[00:03:22] So I started to think, okay, feature cramming is one thing. What else could be contributing to this? I'm going to go down a bit of a rabbit hole here. This is my own hypothesis, not tested by any means, but I believe that as the level of abstraction in programming languages increases, the usability of products produced using those languages decreases. Now, I could give you a lesson in what we mean by abstraction levels, but that's not what this talk is here about today.

[00:03:50] This is just my hypothesis. As higher levels of abstraction in programming languages happen, that means there's more availability of pre-built, prefabricated, pre-designed components. To me, that leads to the increased ease of someone without UX knowledge being able to throw an experience together. And we all know that throwing an experience together gets it to the market faster, but it doesn't necessarily mean good usability, and I think we're seeing that today.

[00:04:23] What does this all mean in terms of AI? Well, if you think about AI as just one more level of abstraction, that's what worries me. That's what keeps me up at night. This is what I have nightmares about. I wake up thinking the world's going to end, and I need a specific tool to save me or my family, and I can't use that tool. That's what I fear.

Combining the Microsoft guidelines with UI tenets and traps

[00:04:44] So as I started to think about that, I thought, okay, what can I do? Well, what I'm about to present is a combination of the Microsoft guidelines for human-AI interaction and the UI tenets and traps cards. And this is something I wanted to share with everyone here, because if we're going to stop the downfall of humanity, we need a tool to help us do that.

[00:05:06] So why not just use the Microsoft guidelines? Well, back in 2019, when I first started looking for a set of guidelines specifically for human-AI interactions, I landed on this paper, and it was fantastic, because it combined research from 20 years into a single unified set of guidelines. At the time, this was really the only place where everything was consolidated into one location. But as I started to try and apply these guidelines, they were a little difficult for me to remember and recall easily on the fly. That doesn't mean they won't work for you, so feel free to use them. In fact, since then Microsoft has expanded these into their own toolkit, called the HAX Toolkit.

[00:05:43] But I also found that some of these guidelines were worded using a little bit more complex language than I preferred to use, which meant I wound up rewording them in order to communicate them to stakeholders. And I'm a bit lazy, so I wanted to find a way to repackage them that made it easier for me to use. I had also had a manager that gave me these tenets and traps cards. UI tenets and traps boils usability issues into single-word tenets and traps of two words or less to avoid. The tenets describe general attributes of good interface design. Traps describe common design problems that degrade interface goodness.

[00:06:20] I think my brain functions in a way that means I tend to remember what not to do better than I remember what you should do, and that's why I really liked UI tenets and traps. An example of one of these cards is what you're seeing on the left. The tenet is understandable, the trap is poor grouping, and it describes: a critical relationship between two or more otherwise noticeable cues, labels, icons, affordances or prompts is not obvious. So it's a pretty easy concept to grasp.

[00:06:51] I was able to use five of the eight UI tenets and carry them over to match the Microsoft guidelines, so this is a really good combination of the two tools. I created three specific tenets that were unique to the AI realm. What you're seeing here is the format that I'm going to use. I will go through all 16 of these. I did provide a number in the upper right, so if you get bored, you'll know where in this presentation I am, and you can come back when I'm done if you really don't want to listen to any more of these. On the left is the tenet; it'll be sitting on top of the pillar. The little icon there is there to indicate when I have reused an existing tenet. And then I've identified which of the Microsoft guidelines the trap is actually referencing. I will try to provide examples of each one of these, but since we have very limited time, I may skip over a couple.

Discreet, forgiving and habituating

[00:07:47] The first one is an existing tenet called discreet. The trap is poor AI timing: the user's current task and environment are disrupted by an AI recommendation or decision. An example of this is my favorite, little Clippy. He's a perfect example to use, even though he's a little bit old. He was actually a really brilliant application from an implementation standpoint. He was the precursor to assistant-style interfaces and actually used a bit of machine learning that was quite amazing at the time. But the problem was he was implemented really poorly, and people got really annoyed with him because he was disrupting their current task and environment. So the lesson here is just don't be Clippy.

[00:08:33] The next one is forgiving. The trap is missing recovery. This is when the AI system doesn't provide a way for the user to refine or recover when the decision or prediction is wrong. This one I don't have an example of, but if you think about this from the perspective of a domain where safety is an issue for the decisions that this thing is making, health care for one, I think you can imagine how important this trap is to avoid in those situations.

[00:09:01] The next one is habituating: disruptive AI change. Disruptive changes happen when updating and adapting the AI system's behavior. This trap speaks specifically to the fact that as humans we tend to dislike change, so ensuring that you don't change too much when you release a new version of your tool is important. But I also feel like this should serve as a warning that if you don't design the first version well, you might be stuck with some usability issues that you perpetuate because of the risk of causing disruptive change.

[00:09:34] And I'm going to get on my soapbox a little bit, but one example could be ChatGPT. I am questioning whether or not a chat interface is the best implementation for large language models, considering we chat as humans in one or two lines, but prompts are turning out to be much longer inputs. So is a small box at the bottom of the screen the best input for a prompt? I don't know. Just something to think about as we go forward.

Responsive and smart

[00:10:01] The next one is a tenet called responsive. This is an existing tenet. The trap is difficult AI dismissal: undesired AI system services are difficult to ignore or dismiss. And here's Clippy again, poor guy. It was rumored that he was so embedded that folks turned to editing the code just to be able to turn him off. So the trap here is: don't trap people into using him, or it. I'm not sure why I was calling him "him," although it's Clippy. Anyways.

[00:10:33] The next one is responsive: forced AI control. The AI system makes decisions for the end user without first allowing the user to approve the decision. Interestingly, this is mapped to Microsoft guideline G10, which recommends scoping services when in doubt. I actually reworded this to be a little bit more forceful and to make sure humans are in the loop, and that's simply because of the domain that I work in. At this point in time, allowing an AI system to make decisions for you seems like a really big safety issue, and despite my optimism in the technology, I do feel like we should proceed with caution to minimize the safety risks.

[00:11:11] Yep, I just realized that was a reveal that I didn't hit on time. Moving on to the next one. The next one is another responsive tenet. The trap is rigid AI: the AI system does not allow the user to globally customize what the AI system monitors and how it behaves. The example that Microsoft gives for this one is the case of a photo organizer. Remember, they structured their examples as good examples rather than bad ones. So they say the product allows users to turn on location history so the AI can group photos by where you have been.

[00:11:48] Again, some of these start to feel a little bit redundant. Remember, I created these in 2019, and I really haven't dusted them off until just recently. So I think some of my future efforts on this are going to be to whittle these down, to reduce some of the redundancies that I'm beginning to feel as I'm presenting this to y'all.

[00:12:06] The next one is the UI tenet smart. Dumb AI: no retraining plan is in place for the model to learn from users' behavior or feedback over time. This is a risk with any tool: it becomes outdated over time. In the case of AI, this means it was trained on a set of data that is old. An example of this is actually ChatGPT 3.5. I know if you all have had the chance to use ChatGPT, it's all the rage. But basically 3.5 was trained on stale data a bit, so some of the answers that it provided were a little bit outdated. And the sad part is now you have to pay for the updated data, version four.

[00:12:53] But I'm hoping that this trap is going to change over time, as things get cheaper to implement, retraining the models at the edge, inline basically, so that as data becomes stale it can replace it without the users even knowing it. But this was five years ago when I created this, and I haven't really seen a big change that says this shouldn't be a tenet and trap to keep in mind, so I'm still including it here.

Trustworthy: hidden abilities, reliability and explanation

[00:13:19] The next set of tenets are customized to AI specifically, and this is the concept of trustworthy. I gave a talk back in 2018 on how to design for distrust, because I was seeing a general lack of trust in the technology, which was contrary to how I envisioned the technology changing the world for the better. And the EU in fact has started to head down the path of regulation, which touches on a lot of these tenets that are in the trustworthy category.

[00:13:52] This trap here is called hidden AI abilities: AI decisions or recommendations that could impact the user's current task are hidden from view. I don't have a visual representation of an example here, but a current colleague of mine and I were just chatting about how he rented a car, and he wasn't expecting the car to actually make decisions for him, like braking, because the indicator for the assist technology wasn't really clear. It wasn't apparent. So he said he was going down the road at 70 miles an hour, and the car identified a guardrail [?] as an obstacle and started braking his vehicle, and it gave him a start. So this is what we mean by hidden AI abilities.

[00:14:37] The next one is another trustworthy tenet. The trap is invisible AI reliability metrics. This is all part of the explainability of what the AI tool is doing, and the Microsoft guidelines are a little softer in their wording than mine. Hopefully you can read this, but theirs says: make clear how well the system can do what it can do; help the user understand how often the AI system may make mistakes. The example on the left that you're about to see here is a little bit closer to what I was picturing. The wording that Microsoft uses in the example on the right-hand side is a little bit of hedging language: "We think you'll like..." Now, if you imagine that in the context of, say, healthcare, and it's trying to detect cancer on an image: "We think it might be cancer." I don't know how well that might go over.

[00:15:35] So from my perspective, and in the domain that I work in, I prefer a little bit more concrete, a little bit more succinct application of this. The example on the left is a location prediction, and the number on the right is the confidence that the model had in its recommendation that it was that particular country. So just something a little bit more apparent than "we think," or just a wording change, might provide a little bit more indication of the reliability or confidence measure of the output.

[00:16:10] Trustworthy: this trap is ambiguous behavior. The AI system provides no explanation of why the system behaved as it did. Again, this is another part of the explainability of AI, which actually is easier said than done sometimes. But I liked the implementation that Copilot took in their search tool with large language models. They actually put footnotes that referenced: this is where we got this information from. I think that explanation of where it got the answer is part of avoiding this trap.

Trustworthy: feedback and change

[00:16:49] The next one is missing user feedback. The AI system does not allow the user to provide feedback indicating their preferences during regular interaction with the system. I recently saw a post on LinkedIn where someone was commenting about their experience with the new ChatGPT 4, and they detailed how annoying the responses were and how long-winded the responses were. He went so far as to call it botsplaining, which I love and am probably going to borrow and use regularly. But he actually prompted the AI to send back this emoji for every response, because he got so annoyed. It's funny, but it also describes his level of annoyance.

[00:17:29] And I have since read a post that says you can adjust these settings, but it was difficult for me to find. So this is what we mean by allowing the user to adjust their preferences for how the system works. They do have tools out there on ChatGPT that allow you to tailor how it answers back, or the tone of voice, which is nice. So that's a good example of that.

[00:17:56] Trustworthy: hidden feedback impact. The AI system does not update or convey how user actions will impact future behaviors of the system. The example that the Microsoft guidelines suggest for this is: the product communicates that hiding an ad will adjust the relevance of future ads. So it's almost like the user takes an action and it's actually going to tailor the outputs of the system. I tend to think of this one in a slightly different light than, say, the commercial products that are just pushing advertisements. In a similar manner to the importance of having a human in the loop, we need to make sure we aren't just adjusting the outputs without letting the user know and decide if they want their current decision to impact it. So that's what we mean by hidden feedback impact. And again, I don't have a ton of examples of these things, because I feel like user interfaces for AI are still new and evolving.

[00:18:53] This one is trustworthy: ambiguous change. The AI system does not inform the user when the AI system adds or updates its capabilities. Very similar. And again, I feel like some of these are a little bit redundant, but still useful to make sure that you're fully thinking through what it is you're designing.

Unbiased and understandable

[00:19:10] The next one's pretty obvious: unbiased, the AI system's language. And I say it's pretty obvious; it's probably not, and that's why I have it in here, so that we all make sure we're stopping and understanding that you could potentially reinforce undesirable and unfair stereotypes and biases just by training the algorithm on data that might be biased. I put this one in here because I don't know that the designer of the user interface necessarily can do a ton about this. I feel like if you're part of the team that's building these tools, bring it up, and bring it up every time.

[00:19:50] Another AI UI tenet is understandable: memory challenge. The system requires the user to remember information that is easy to forget. This is tied to G12 of Microsoft, but I actually just straight pulled a UI trap for this one. Some of the Microsoft guidelines are actually tenets and traps that you would keep in mind as you were designing any system. So again, my next steps are to whittle these down to reduce redundancies.

[00:20:23] The next one is task mismatch. The AI decisions or recommendations shown are not relevant to the user's current task and environment. Again, this one matches an existing tenet and trap called forced syntax, just slightly differently worded. And I did accidentally land on an example of this while I was posting on Meta. I wanted to try their AI tool to rewrite my post to be funnier, so I clicked it. It returned an absolutely terrible joke, so I thought it'd be even better to post the original text than post the output of the AI. However, it turns out you can't select an individual sentence and rewrite it. It will only rewrite the entire post for you. So that did not match my expectations.

[00:21:06] And funnily enough, the next trap we have here is expectation mismatch. So again, I'm feeling like some of these are very redundant, and that's another reason why I feel like we can refine the Microsoft guidelines and combine them with UI tenets, and that way we're not trying to divide ourselves as UXers into two categories. I feel like we need to bring all these together, merge them, and reduce some of those repetitive ones. Again, this one matches forced syntax, just slightly different wording. And the wording in the Microsoft guidelines did have an important part, where it was touching on the social and cultural impacts. So that is important, and I think there might be some good benefits to combining the two.

[00:21:59] The next one is invisible element. Again, UI tenets and traps already has this: the AI system services are difficult to find or invoke when needed. Again, very general, the same kind of concepts as a lot of UI tenets and traps. Basically, don't make things hard to find when you need them. Don't cram your product full of unnecessary features, and maybe it might be easier to accomplish in the long run.

[00:22:28] So this is how I've been able to remember what not to do when designing for AI, and I hope these tools work for you as well. All I really want to do is avoid the downfall of human civilization as we know it today. So use these, or try one of these other tools. Let's save human civilization together. And just so you know, if you go to my website, the presentation on there is not exactly the same presentation you're seeing here. I meant to update it, but unfortunately we had a bit of a natural disaster in my area. I hope you enjoyed the presentation. I'm a little bit early, Rory, sorry about that. That's what I've got for you. Questions?

Q&A

[00:23:09] Rory: Excellent. I won't complain about you being too early. You're off the hook.

[00:23:13] Shari: Awesome.

[00:23:15] Rory: So if anybody does have any questions out there, please feel free to write them in on whichever platform you're watching, and I'll put those questions through to Shari. But when you were going through those, I have a couple of questions, if that pleases you.

[00:23:30] Shari: Yes.

[00:23:31] Rory: In particular, one or two of them referenced, I think it was, not remembering, or something like that. Now, that to me is actually a problem with the technology today, in that it's very hard to get it to remember how people are reacting and things like that. So how much do you think some of these things are a problem of it's just very early and we haven't figured it out, versus people are rushing and not doing it well enough?

[00:24:06] Shari: Yeah, I think that is definitely something to keep in mind: it is very early. One of the challenges I had when I was looking for guidelines in general was that while AI has been around for 20 years, implementations of it haven't, mostly because of the computational limitations. It is very computationally costly to actually implement these things in the real world, and it's only now that it's becoming an inflection point, where our chip technology is allowing these things to be trained and deployed in the real world. So it is a little bit of that: we just don't know what we don't know. And yeah, for sure this technology is new.

[00:24:55] Shari: But I do think that's really why I'm doing this. So many of these guidelines, when I went through them to try and merge them with UI tenets and traps, felt very redundant to the stuff that we are already doing. So I think, while it's brand new, it's really important to get ahead of the usability of these things while it's being fleshed out. Did that answer the question?

[00:25:28] Rory: Yeah, I think so. But it brings me to Clippy, who got picked on a little bit, and we'll call him a he. What I'm seeing, my personal opinion, is that every company wants to do AI now. It doesn't matter whether you've got a business case or there's a real need; you have to tack AI onto your product. I'm just seeing those popups everywhere. So many tools I open up, and it's like, "Oh, have you tried our new AI feature?" So I guess the challenge is, I'm sure a lot of those teams realize they don't have a great AI case, but they have to do it because then their share price goes up. So how do you balance it? If you're on that team, what can you do as a designer to try and push back against that "just bolt on AI wherever you can"?

[00:26:21] Shari: Well, I do think there are use cases for it, if you dig. So even if you're told, "Well, you've got to include AI here," I think if you start interviewing users and you start identifying where in their process they think they could use some improvements, that's essentially what most AI tools are there to do today: to improve how we do things, not necessarily replace what we're doing. It's just going to change what we're doing and try to make it better.

[00:26:55] Shari: When I talked through the fact that I feel like the products we are using have become less usable, I mean that I now have 10 tools out there to do one thing. I have one task to do, and I have to go to this tool and this tool and this tool, and maybe AI can be the bridge that mends that gap. So I think the use cases are there. Even if your company is just saying you have to use it, well, then find the right place for it, and make sure that you're applying it in the right way.

[00:27:33] Rory: Makes sense. And one thing I think is going to happen over the next while is all the legacy companies are bolting it on as an add-on. I've noticed everyone charges for it. It's very rare it comes free. But I think that leaves room for disruptive technology companies who just build with AI from the ground up and really embed it, instead of thinking of it as a bolt-on.

[00:27:53] Shari: Absolutely, I agree with you too. And I do think the bolt-ons are probably not going to be as usable as we think they are, so they might just come and go as a fanfare kind of thing today.

[00:28:09] Rory: The hype cycle.

[00:28:10] Shari: Yeah. And like I said, I've been doing this for a long time, and I've seen the ups and downs of "companies need AI, put some AI in there." That happened when I was working in it six years ago, and then it dropped off, and here we are again, with whatever the next tool is that comes along and makes it popular again. On the one hand, as I mentioned earlier, I'm a little disappointed that it is a chat tool. I think large language models can have so much use beyond just a chat tool, but I'm afraid we're pigeonholing ourselves into the interaction that ChatGPT put together. So those bolt-on applications, I don't know that they're actually going to make our lives easier. And you're right, I do think if we're going to do it, it might need to be from the ground up.

[00:29:02] Rory: Excellent. Well, that brings us pretty much to time. I do have more questions, but I'll have to hold them for next time. So thank you very much, Shari, for sharing. And as somebody said, titanium sin [?] said, "No Terminators for us."

[00:29:21] Shari: Thanks. You could play on Terminators.

[00:29:24] [laughter]

[00:29:26] Rory: Yep. So thank you very much.

[00:29:29] Shari: Thank you.