Keeping up with AI in the race to get ahead

Prianka Murthy at UXDX Community: Using Comedy and AI Insights for Team Success. Video: https://youtu.be/M1PujtLiwg8

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Keeping up with AI while staying true to UX

[00:00:00] I will be talking a little bit about keeping up with AI. I think this is a hot topic at the moment. I think we all have seen that there's been an AI boom in the last couple years. There is mounting pressure to be at the forefront of innovation and to be putting out all of these great AI experiences. But what does that really mean in UX, and how do we make sure that we are keeping ahead while still staying true to our practice, staying true to UX methodologies, and keeping users at the forefront of our work? This is a little bit of what I'll be talking about today.

[00:00:41] A little bit about what I'll be talking about: I'll introduce myself a little bit, so you know why I'm even talking about this. I'll give an introduction, go over some of the objectives of my talk, and then we'll get right into the meat of it.

[00:01:01] A little bit about me. I am a former academic turned UX researcher. It seems like a really popular career pivot; I think four or five people I know in my life are people who made that career pivot. I have the privilege of having experience in qualitative and quantitative research in UX. I have also had the privilege of working across a lot of different industries. I've worked in industrial design, mental health, and more recently in the finance industry. I think having experience working across a variety of industries has helped me be a little bit more flexible and adaptable as a UX researcher, and has given me a lot more insight into the way that different industries work, and has allowed me to harness that knowledge in my current role.

[00:01:51] Why am I talking about AI? As a researcher I've mainly worked in spaces where I am either the only researcher or I have a couple of supports, but it's mainly me scaling and strategizing about frameworks and research. I have spent a lot of time thinking about this and aggregating data on how we do research on AI and what that looks like in UX, and I have also spoken to a lot of folks who do a lot of AI work. I had all this data and I figured, why not share it with the world, as I think there isn't a lot of precedent about this right now.

[00:02:34] Part of putting this presentation together, I had a lot of fun finding interesting AI pictures that are weird to look at, so you'll see a lot of those pictures throughout this presentation, because they gave me a lot of pleasure.

[00:02:49] As we know, AI is still an emerging industry. There are some conversations happening about UX for AI, but there isn't a lot of precedent on how we should be formally attacking this issue and how to integrate AI into your research practice. I think this is important whether you're currently working to scale UX research up at a smaller company or you're at a company with a more established practice. Whether you're a designer, a researcher, a product manager or owner, figuring out the best ways to integrate and leverage AI can be tricky. It's something that I have tried to distill in this talk, and I've tried to talk a little bit about the main segments that I think are important. There are going to be a lot of things I missed, but I hope that I can touch on the main things that I think are of interest.

Objectives and the basics

[00:03:45] Objectives. With AI innovation speeding up, there's so much competition between industries and trying to keep up, and I think there's a lot of pressure on product teams to be putting out these AI experiences without making sure we're going through the right vetting processes. AI should not be exempt from good research. It shouldn't be exempt from UX design, and it should be seen as an even higher priority, because I think AI is a high-risk, high-reward system. We should be making sure we're putting enough time into some of this front-end work.

[00:04:25] I'll go over the basics: how do we make sure we're incorporating AI into our practice, and what does that really look like? How can we make sure we're keeping up while not driving ourselves insane trying to stay on top of every single AI feature that's been put out? How do we develop a strategy? How do we create effective strategies with AI in mind that still incorporate the larger business goals and objectives that you're working within, and also how you are looking to contain it within your UX practice? And also managing stakeholders, which I think is the overarching part of this, the key piece to making sure that we can get what we want from these AI experiences. I pulled these topics together from my own experiences of what I wish I had known before I started doing research with AI, so I hope it can be helpful in that capacity.

[00:05:24] I'll go over some of the basics first. As a researcher, it's no surprise that I think research comes first. But I think it's easy to think of AI as a catch-all solution. For any of you who work especially in more tech-focused companies, AI is starting to be that thing where we're like, why don't we solve this with AI, why don't we use AI to do this, why don't we just build a chatbot? Not every problem requires an AI solution. Make sure that you are critically thinking about the problem space, your users, the pain points and your goals before you even get to AI and what that should look like. And if you do go with AI solutions, validate heavily with users. I'll go into this in a little bit more detail.

[00:06:17] Creating effective AI roadmaps. It's lucky we're doing this early on in the year, but as roadmaps are built out for the year or for the quarter, consider what products or features might benefit from AI. Part of that comes from understanding what AI use cases exist out there and what features are being put out. This is probably more relevant if you're a product manager, but even as a UXer, being able to think about that helps you scope out projects and think about what features could map onto your product needs, based on research that you've already done.

[00:07:00] Something that I have done in the past is working with your team to do AI-centered sprints, to get them started thinking about AI solutions. Something that I found is it's not really intuitive to how I think. I'm not always thinking, well, AI could solve this problem. But if you're immersing yourself in it a little bit more and working with your team to get the creative juices flowing on how AI can be utilized, I think that's where you find the maximum impact.

[00:07:28] Also make sure to keep up to date on how your competitors are using AI. I wish there was some competitive analysis newsletter distributed about this, but keep up to date as much as you can on AI trends for various features within your industry, but also across other industries. We focus a lot on what tech is doing or what we're doing within our own industry, but there are a lot of really interesting features being built out by other industries that we could learn from. Keeping yourself siloed into just what other people in healthcare are doing or other people in finance are doing isn't necessarily serving your needs, so make sure that you are taking an agnostic approach in that sense.

Ethics in AI

[00:08:15] I'd be remiss if I didn't at least mention ethics in AI. Obviously I could give an entire talk about ethics in AI; maybe not me, someone else should give an entire talk about ethics in AI, and I think there's an entire panel that talks about this. But it's important to make sure you're embedding ethical principles into the work that you're doing. I think there are three main principles to make sure that you're considering, so that the ethical implications of your AI aren't putting anyone at risk, aren't making people uncomfortable, and aren't setting yourselves up for lawsuits, things like that.

[00:08:50] Transparency is number one. Making sure that your users know what you're using AI for and what the AI does is really important. You don't want to be jump-scaring your users with, surprise, this was an AI feature and you had no idea. That's not a great way to build trust with your users, so make sure that they have all of that knowledge. Also privacy. Data and how data is being used is another hot topic, but make sure you're telling your users how you're going to be using their data, what's going to happen with the data and how it's being stored. Things like that are incredibly important and invaluable.

[00:09:31] Lastly, neutrality. Hopefully none of us are setting out to create experiences that would be discriminatory or biased in any way, but make sure you're striving towards creating experiences that aren't biased or discriminatory, that you're considering that when testing things out, and that you're considering it in the product roadmap.

Developing a strategy

[00:09:55] Now I'll move on to talk a little bit about developing a strategy. Where do we start? I think this is a loaded question. I've tried to once again distill it into some easy key points, and I've used a lot of resources to put this together. GitLab has a great resource on this that I pulled a lot of data from.

[00:10:19] Where to start: understanding the problem space. What do you know about the problem space? I think this is good for some self-reflection, when a request comes in if you're a UXer, or if you're a PM thinking about what features make sense, for your understanding of the users and the problem space. This will determine what research you pursue, and I'll talk a little bit more about what kind of research and how to choose that based on your understanding.

[00:10:49] Then you go into user research. Make sure you take the time to collect data from users, whether it's more generative research such as user interviews, or you are looking for more feedback on solutions. Making sure that you do get some touch point with your users will be invaluable.

[00:11:05] Validation and iteration is a great part of the UX process. Obviously in an ideal world we would do this with every product, but that's not always the case. Make sure that throughout the design process there's a continuous feedback loop with your users. Every time you're creating new concepts, you have new pages, you want to be going back to your users saying, hey, does this make sense? Does this look like what you want, what you envisioned?

[00:11:33] Then continuing research. I think it's important in the long run to make sure that you're setting up with your stakeholders that this is going to be long-term research. As soon as you're finished with the product per se and you've handed it off to development, the research doesn't stop there. After the product goes live you still want to be collecting some metrics on user satisfaction, seeing how people are responding to the product, and then continue to do research throughout, as things change and as you want to add new features and functionality. Make sure that you're setting your stakeholders up with that expectation.

Choosing the kind of research

[00:12:13] Deciding on what kind of research to pursue. When you don't really know the problem space, you want to do user interviews. This is the case where someone says, hey, we want to come in and create an AI recommendation feature for this group, and it's for a particular application within a suite of applications you've never worked with. This is a good time to take the time to interview your users, to understand their pain points and their goals and understand who they are. You want to take the opportunity to create personas, representations of your users that you can refer back to throughout. Since you want to be doing continuous research, having a living artifact that represents your user pain points, needs and goals will be helpful to refer back to throughout the research process. Then work with your users to map out their current journeys and also their ideal solution states. In an ideal world, what do you want this to look like? That might not necessarily be functionally possible, but it gives you something to strive for, something that you want to get as close to as you can.

[00:13:31] When you do know the problem space, and this is probably more likely, as a researcher or a designer or a PM at your company chances are you've worked with these users a lot. You have specific subject matter expertise in that area. You know the current pain points and goals. You might not know them for this specific use case, and that's important to evaluate, but you want to start with the problem. Look at the problem space and evaluate your own understanding. Do some self-reflection: how much do you really know? Make sure you go through the research that you already have. Assuming you've done research on these users, and you've spoken with them and had touch points with them, review all that data to see what the gaps are, what else you want to know and what you need to validate. Then start thinking about solutions, which you can then go back and validate with your users.

[00:14:24] An important point to also mention is to make sure that you understand what kind of mistakes your feature can have, and the consequences. A lot of us are seeing these stories come out about an AI feature gone wrong. One of my co-workers was telling us this: I think it was Air Canada. They had a chatbot, and one of the people who was supposed to go on a flight canceled their flight through advice from the chatbot, because the chatbot told them they could get a full refund. They were like, great, so they canceled their flight. Then Air Canada said, hey, this is not our policy, and tried to argue that the chatbot is a separate entity from them and therefore they're not responsible for the responses from their chatbot. Of course the court ruled that that didn't make any sense, because they created the chatbot and they should have thought about this and thought about their own policies.

[00:15:25] Make sure to consider examples we've seen in the media of AI gone wrong and the potential impact. Make sure you have a good understanding of that, and you do a lot of critical thinking and research and testing to make sure that that doesn't happen. At the end of the day, whatever AI feature you create is an extension of your company. Whether or not you want to argue it's not, it is.

Managing stakeholders

[00:15:51] Finally, to talk a little bit about managing stakeholders, my favorite topic. When a stakeholder approaches you, you might have two situations: one where they want to build an AI experience, or one where they want to build an AI solution. What's the difference here? An example I saw recently of an AI experience was Netflix's recommendations feature, versus an AI solution, where you might have an AI chatbot or something.

[00:16:31] When you get approached with an AI request, start by asking the right questions. What do they want to build? Why do they want to build it? Where did they get their data that's informing them that this is a good solution or a good experience? What are they looking to accomplish? I think this is important ahead of any project, but especially with AI projects, where a lot of the priorities can get a little muddied. A lot of the reasons we get approached for this stuff is because a competing company put this out, so therefore we have to put it out now. So make sure that you're doing some critical evaluation of that space.

[00:17:10] Managing requests. When you're talking with stakeholders, make sure that you're setting up reasonable expectations. We talked a little bit about setting them up for a continuous research process. Create a research plan that's in line with your understanding of the space, and make sure that you're advocating for the kind of research that you need to be able to accomplish the goal that you have and that will deliver the best results. Emphasize the need to get it right, versus putting out something that's partially ready just to get it out in the world. This is a tension and a line that you'll often walk in a lot of spaces, but with AI you really want to make sure that you're doing your due diligence. You're doing your research, you're working with your users to make sure you're putting out something great, rather than putting out something mediocre that is going to fall flat. Then you lose trust from your customers, you lose satisfaction and things like that, and you'll have problems with retention. So make sure that you're keeping those priorities in mind.

[00:18:18] Lastly, why do we take a user-centered approach? It feels like I should have started with this, but I'm going to end on it, because knowing how to pitch this to stakeholders is a big priority and will get you the results that you need. Why do we take a user-centered approach for AI? I think for the same reasons we do for other products. Better innovation: having access to users and speaking with them can help you uncover new opportunities and new gaps in the space to create a unique experience. As much as I'd love to say you can just copy what the other company did, you want to make sure that you're in touch with your users and you can create something new and interesting, rather than redoing the same thing over and over.

[00:19:05] Neutrality: testing your experience with users to make sure that you understand what the biases are, any discriminatory issues, any large issues in missing use cases or something. You'll know, and you can quickly resolve them. Finally, user satisfaction. As we know, UX research leads to higher satisfaction and higher retention, and AI shouldn't be any different. The higher the stakes, the higher the reward, but high visibility can also lead to high downsides. So make sure that user experience is a top priority going into your AI experiences.

[00:19:46] Thank you so much. That was my talk. I'll hand it back to you, Katherine.

Q&A

[00:19:54] Katherine: Fantastic, thank you so much, that was really great. Lots of questions, if you're up for it. First question: what do you use AI for the most? In conducting research, I guess; I'm going to assume that's what they're talking about. I would assume probably in your role, if you can talk about it. If there's anything you can't talk about, don't worry about that, but just in general in your role. Another question that's quite similar is around AI for iteration: how much context does AI get for iteration? So maybe it's at what stage do you use it most. That's the way that I would interpret it.

[00:20:35] Prianka: Got it. As a researcher I actually don't use AI very much. I have limitations at my company on how much I can use it; I'd love to use it. But if I could, I know there are a lot of really great resources. From a qualitative perspective, I know there are a lot of summarization tools that are coming out, and data annotation AI tools that are coming out. I would always prefer to do that stuff on my own. But from a quantitative perspective, having AI run those numbers for you without having to do it manually is a really big win. That would be my take on that.

[00:21:18] Katherine: Love it. Going on to finance, banking, government, all these companies. I had a chat recently with a government organization who cannot use AI, and they're building their own for privacy reasons, and the same will go with people, as you talked about data. What do you recommend, where to start, or what to do in those scenarios?

[00:21:47] Prianka: Where the banks or government are building their own?

[00:21:51] Katherine: Yeah, I guess more from the perspective of, you're not able to use it in your role. How do you actually go about it? Do you avoid it totally? How can you pitch it to management?

[00:22:06] Prianka: I would imagine that management, and maybe I'm wrong about this, is coming to you and asking you to build AI experiences. As much as you might not be allowed to use those features, you can use them as a reference for competitive analysis. Something I've enjoyed doing is, in my downtime, testing out a lot of these AI experiences so I understand how they work, especially ones that I can't necessarily use in my own role. If management isn't coming to you, I think talking to whoever you report to about maybe embedding some of this. It depends on the scale of company you work at, obviously. If you work in a big government agency, those decisions might not be up to you. But I would be shocked if management wasn't thinking about how they can start to create these experiences, because this feels like the big topic of how do we excite the shareholders and how do we get to the front of the times, things like that.

[00:23:13] Katherine: Makes a lot of sense. How do you conduct research when you're trying to build for AI? How do you actually get feedback when you're building something brand new?

[00:23:28] Prianka: If you're starting from scratch, it's a little bit of what I was getting at with what problems you are trying to solve. Just saying you want to build an AI experience is great, but going in just feeling like you want to create an AI experience, versus having problems that you think could be solved by AI, are two different situations. For example, if you want to build a chatbot experience, understand what problems it would be solving and what solutions it is really providing, and make sure you're validating with your users: is this something you actually want? Is it something you're actually going to use? Versus having certain problems that you want solved and seeing how AI can fit in as a solution. These are two different roadmaps, but I think both are equally valid. Make sure that you're in touch with your users and you have a good idea of how they want it to look, how they want it to function and what they're actually looking for, because AI chat is such a broad topic. Seeing what niche this is filling and what need it is serving are the critical questions you need to start asking yourself before even going into the research process.

[00:24:38] Katherine: That's a really nice follow-through from your talk: what is the need and the purpose, and how do you prioritize, potentially. How do you actually find what the need is? And it might be a nice segue simply to Dan Way's [?] opportunity solution trees, just mapping it out. That was really fantastic, thank you. A standard question that I ask: what blogs and resources do you use to stay on top of things?

[00:25:05] Prianka: When you asked about Dan Way [?], I knew I had some, but then I went back to see what I use. I just see them, and I don't actually log what I'm reading. But I do follow the Medium UX design thread, and they send out really great newsletters. I do also get the emails from UXDX; Rory's in my inbox all the time. On LinkedIn, honestly, I really like following certain influencers in the space. For research I really like Aona Yang [?]. She does a lot of YouTube videos on UX research, how she got into it, methodologies, things like that. In general, if I see someone who's putting out really good content on LinkedIn I will follow them, and usually they're posting interesting thought pieces about how to conduct research and different things that I hadn't thought about, from interesting new lenses. That's something that I really like doing.

[00:26:07] Katherine: Love it, that's really nice, thank you for that. And what tools do you use to help you in your work?

[00:26:17] Prianka: I use Figma on my own. I've worked a lot in contract roles, and in a contract capacity I use Miro. I really like Miro. I like Figma, FigJam. I think that's the majority. Figma is a big one, always Figma and FigJam. There are just so many options, but I do really like Miro.

[00:26:46] Katherine: Miro is great; they're all really great options. Thank you so much, Prianka. I really value that, and I know other people will. It was a really nice logical run-through, so thank you so much, really great. We will see you next time, and keep in touch.

[00:27:02] Prianka: All right, thanks so much.