How to Use AI Ethically in Your UX Research Workflow

Shreya Thakkar at UXDX Community: Design for Everyone: Ethical AI and Inclusive UX. Video: https://youtu.be/7GwUoN8hB9U

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

A story of curiosity and a fake citation

[00:00:07] Good afternoon or good morning, everyone. I know you all would be joining from different time zones. My name is Shreya. I am a senior UX researcher at Electrolux Group, and today I want to share my story of figuring out how to use AI ethically in UX research, as a story. It began as a story of curiosity. We made mistakes. We built something that worked, and I'm sure we will again need to rebuild, as these tools are evolving so quickly.

[00:00:37] A little bit about my background. I work for a home appliance manufacturing company, which means we talk to people about their vacuum cleaners, washing machines, how they clean their homes, the home ecosystem, going from applications that they use to how they cook and the appliances and how they interact, what their routines are like. It sounds a little mundane, but it's actually very deep personal research. We are watching them in their homes, understanding their behaviors, their frustrations, their value system. A lot of their personal data comes to us.

[00:01:15] Before this I worked across multiple different industries, going from healthcare experience design, mobility, transportation and also consumer tech and assistive technology. Today I'll be sharing how AI tools are part of my research garden. This story is about learning which tools help us grow and which ones destroy the garden if we are not careful. As a disclaimer, everything I share today is my personal perspective and not my company's.

[00:01:55] We know that AI is everywhere: news alerts, LinkedIn posts, product launches. The sparkle icon has been popping up everywhere, and companies use this magic sparkle icon as a metaphor for AI, and it also shifts perception from threatening automation to accessible magic. Sparkles reframe AI from being threatening automation to accessible magic.

[00:02:21] In early 2023, when ChatGPT was new and we all were on it, I asked ChatGPT for peer-reviewed sources on aging and assistive technology for a literature review. It gave me a citation: author name, journal, volume, page range. It looked completely real. I searched for it. Thirty minutes later, I confirmed the paper didn't exist. The author didn't exist. When I told ChatGPT, it apologized and gave me another fake one, and then another made-up one. I think most of us have had some version of these moments since the time we have been using these AI tools, and I'm sure we've all been through this.

[00:03:12] What I learned from that experience came down to two things. First, AI doesn't know what it doesn't know. Hallucinations aren't bugs. They are features of how these models work: they generate the most plausible next token. Second, verification isn't optional. If I hadn't checked that day whatever the AI tools were giving me, and I had shipped it, we would be building on sand. That was one of those moments that didn't push me away from AI, but it pushed me into having a deeper understanding of how these tools work and how we can use them more responsibly.

[00:03:52] As UX researchers we are uniquely positioned, in a way that we don't just use AI. We also study how humans interact with it. We are the bridge between the builders and the people who live with it. For most of today's talk we'll be looking into AI for researchers, which covers the tool side and not the subject side. I think as UX researchers it's an equally important role, and we have this dual role, which is our superpower.

Learning how AI works and cataloging the tools

[00:04:30] To start with, back in the day I did something that probably sounds obvious in retrospect: I decided to actually learn what AI is and how it works, what its limitations are. I started off where most of us do, which was googling traditional programming, how you write rules, what those rules are, what neural networks are, narrow AI, general AI, going through all these terms to what generative AI is in the first place. The more I understood the mechanics, the less mysterious the failures became. The fake citation I talked about made sense once I understood how the language models actually work and what those neural networks are, because at the end of the day it was a black box, and it gets data and could make up citations in certain places.

[00:05:23] After that I went full into researcher mode. I built a Miro board and started cataloging everything: transcription tools, synthesis tools, coding assistants, image generation, and also organized by the research phases that you see here. First was research intake, secondary research, how we do participant recruitment, analysis. Every week there were new ones. I would add one on Monday and by Friday there would be three more claiming to do the same thing, and the board got messier.

[00:06:01] But that mess taught me something. The problem was never finding tools. I was in this race of "What are these new tools? What can they do?" The marketing of these tools was quite good, so we feel, oh, here's another tool that can help us out. But I later realized it was about knowing which ones deserved a place in our workflow, and why.

AI slop and cognitive offloading

[00:06:24] After that, all the content that these tools were generating, whether it was conversation-based or part of the software where we saw those sparkle icons coming up, was flooding everywhere, which is now called AI slop. Over 50% of long-form LinkedIn posts were now AI generated. Merriam-Webster made "slop" the word of the year, because a lot of the web content that we are seeing right now is generated by AI. It's not just an internet quality problem. It's affecting how we think as well.

[00:07:11] What's happening to our brains when we're reading so much slop everywhere, whether it's talking to AI or understanding our research context? Researchers call it cognitive offloading: we let AI do the thinking, and our critical thinking muscles are not used enough. We found that human cognition was impacted a lot. We stopped thinking in the first place. We stopped questioning the sources. We accept the first plausible answer that comes up, which is exactly what I did when I trusted the fake citation. As researchers we are supposed to be the last line of defense against bad information and bad data, and to use our own judgment in weeding out the bad data that comes to us.

[00:08:09] Where are we today? I was talking to a lot of researchers on LinkedIn, at conferences and on my own team, and what I found wasn't binary. It was a spectrum. There were skeptics on one end, and a lot of full believers using a lot of different kinds of agents on the other, and most of us sit in a messy middle. Here's the thing: everyone's moving. Where you sit shapes how you use AI and how you talk about it, whether you bother to verify what it gives you, or whether you don't bother on quick research that we do. We just check the resources and we keep moving forward.

[00:08:54] I bet some of you are secretly using AI and not telling your teams. And some of us are refusing to use AI and feeling like we're falling behind. I've been on both sides. Talking about shame, the AI shame: my hallucination moment was a shame moment for me. I felt like I should have known better. I should have understood how we use these tools, and also, what if I had not checked all the information it was giving me that day? Abby Binder [?] writes about this very beautifully, about the shame dynamics that are shaping up in our relationship with AI as a craft, and we do need to name it before we move past it, that it is something we all are going through.

Prompt crafting and context engineering

[00:09:44] The first hallucination was one of the turning points, but instead of walking away I tried to learn and make it a learning journey, and since then I have been on this journey of learning about these tools and the new platforms that come across. Everything from here forward is what came out of it, built from mistakes, experiments, what we did as a team together, what was right and what did not work for us.

[00:10:15] When it comes to working with AI, there were two approaches that initially came up, and they serve different purposes. One was prompt crafting and the second one was context engineering. Prompt crafting is what most of us do: choosing the right words, providing a little bit of context, specifying the format of what we want to come out of it. It's a personal skill in how individuals get better outputs, but it only scales as far as the person's expertise.

[00:10:51] The next one is context engineering, and it is about how you set up the system. It's not about the prompt. It's about the scaffolding, the entire world of knowledge the model needs: instructions, memory, retrieved documents, tools, structured output formats, why we are doing this in the first place, and understanding the dynamics in our teams. You've probably noticed the same shift in 2026, which is also called the year of context: we need to understand the full context before just getting the data out. It is also known as the year of processes, playbooks and workflows. The individual prompt era could be over very soon, and the systems era will be here.

[00:11:43] I learned that AI is genuinely fast. Whether it's data analysis, summarization, pattern recognition or transcription, it handles these things well, but it trips on accuracy, cultural nuances, emotional depth, when we hesitate, when we are thinking about something. It only takes the data that we say out loud, and it doesn't take the nonverbal communication. That's a fundamental characteristic of how these tools work. So the question becomes: how do you work responsibly with a tool that's so fast but not as reliable as we thought?

The ethical questions researchers face every day

[00:12:23] As researchers we come across these questions every day. The moment you copy-paste an interview transcript into an AI tool, you're making ethical decisions whether you realize it or not. Consent: did our participants agree to AI processing of their words? Privacy: where does this data go, who can access it, for how long? Bias: AI doesn't eliminate it, it amplifies it at scale. Transparency: do our stakeholders know which part of the analysis was AI assisted and which was not? And accuracy: how do you catch it when it goes wrong, before it reaches a product decision? These weren't abstract problems. These are everyday decisions that all UX researchers are making.

[00:13:13] At this point, I had some knowledge about how these tools work and understood the technology, the ethics, the risk. But everyone on my team was doing their own thing. Some people were using AI for everything, others for nothing. Quality depended on who was running the analysis. We needed to move from individual experimentation to a shared practice.

[00:13:41] Here was the core problem. Give two researchers the same data set and the same AI tool, and you would see widely different quality depending on who ran it. The answers and the insights would be similar, but the quality of responses would differ a lot based on how they had used it, and there was also the pressure of moving faster. There was no framework to protect rigor under the pressure and the speed that we wanted to make decisions at.

[00:14:11] So we audited how each team member was actually using AI, designed a phase-by-phase integration framework, and established principles and tool-specific playbooks, and not a top-down policy. We already had established ethical practices around data consent, privacy and anonymization, which weren't new. We were standing on the shoulders of brilliant researchers, and I did not have to start from scratch. Fellow researchers had already created valuable frameworks, whether it was crafty trust master [?] on UX research, and how we understand the different frameworks out there.

Three principles and a phase-by-phase workflow

[00:15:02] We distilled everything into three principles. First, human in the loop at every decision point: AI suggests, but the human is always the one making the decision. Second, verify before you ship: every AI-generated output gets checked against the source material before it reaches stakeholders, every time. The third one is transparent methods disclosure: every deliverable says exactly what AI-assisted tool we used, what was human-led and what was AI assisted.

[00:15:41] After that we started mapping the AI-assisted research workflow: what AI does, what humans do, and what ethics are required, from planning, data collection and analysis to reporting. There are the things we need humans to lead on: defining objectives, ethical decisions, asking follow-ups, reading body language, building trust, validating themes, interpreting context, telling the story. I think that is one of the bigger things. Then there's the ethics layer: how we document usage in the plan, choose compliant tools in the company, and anonymize before we upload anything. One of the major points was how we strip the PII, set human review checkpoints, write AI disclosures and review biases in synthesis.

[00:16:37] Also, not everything needed AI. That was one of the first things. Just because you can run something through an AI tool doesn't mean we should. So we also built a decision tree: low risk, let AI help; the other, which always needs human verification, is a little higher risk. In the green zone, transcription, initial coding and tagging suggestions were low risk, and AI could help. The yellow zone was synthesis, which was theme generation. AI can assist, but nothing moves forward without human verification. The third was the red zone, which was cultural interpretation, ethical decisions and final stakeholder recommendations. They always stay human-led.

Prompt templates, disclosure and data preparation

[00:17:28] We also built a prompt template library. Building these prompts and working around them really helped us. Standardizing those prompts, rather than one-sentence conversations, which would make it a little difficult for every researcher to go through. We built prompt template libraries with standardized reusable prompts built on frameworks from other researchers that we took, tried, experimented with and built our own. Each template has three parts: the prompt itself, the expected output format and the guardrail, meaning what we check before trusting the output. Thematic analysis, discussion guides and cross-study synthesis were all covered, and they are all living documents. Every time the tool upgrades and we need to make changes to our templates, we go in and make those changes.

[00:18:30] One of the things that we also brought in, and went really deep into, was AI disclosure and data preparation. Every deliverable from our team would include a transparency card: which tools we are using, which version, how much human oversight was involved, and how we validate the outputs. The second one was data prepping, which was very important. If you feed messy data to AI tools, you get messy outputs. Another important thing was to remove all the PII, any personally identifiable information, before anything goes near an AI model. That was non-negotiable. We would also tag, clean up the formats, make sure it's clean and ready for proper research context, and name things consistently for our research, which was some of the non-glamorous work, but the difference in the AI outputs was something you could see immediately.

AI raises the floor, and principles over products

[00:19:39] One of the other concepts that came up from Harvard Business School and Wharton research on AI, which I found very interesting, was that AI raises the floor. It brings everyone's baseline up, whether it was transcription, initial coding or pattern spotting, and tasks that used to take days now take a couple of hours. But the ceiling is still ours. I don't think the ceiling has moved enough. It depends on our human intelligence: how we interpret culture and emotional nuances, how we read body language, how we understand nonverbal communication. AI hasn't raised the ceiling at the moment.

[00:20:26] The AI tool landscape also churns constantly. Tools get adopted, hyped, abandoned, replaced, bought out. A lot of things happen, and if we build our entire framework on a specific tool, it dies with that tool. So the three principles don't mention any tools, and we did not want them to. The decision tree asks about the task type and the risk level, not which software to open, and we don't make our decisions based on the software. When the tool dies, we don't want our practice to die as well. We want our practice to survive. So we build around principles, not products.

[00:21:17] One of the last things to keep on our radar is that the next wave of AI might not be a single AI assistant. It's teams of specialized AI agents, and we'll be the orchestrator, coordinating multiple agents each handling different tasks, from planning to coding to analysis, all running simultaneously. But if we don't have an ethical practice with one tool, how do we prepare for orchestrating many tools? That's exactly why frameworks matter more than tools, and the principles you build today will be the foundation of tomorrow's complexity.

The human parts become more essential

[00:22:00] I also want to bring up something that might feel counterintuitive: the more AI I have integrated into my practice, the more essential the human parts have become. Reading body language and cultural context, asking follow-up questions, the things that we saw and felt when we were interacting with users, telling the story of how we move stakeholders to take action. No model does that.

[00:22:28] One quick example: I do a lot of global research, where we have a particular UI to test in different parts of the world and we want to see how each country responds to it. A lot of times I've seen that, in cultural context, it would give stereotypes of what a country's culture is and build a story around it. My data did not have that information. My data did not have any of it. But the moment it read the name of the country and the geography, it stereotyped it into certain cultural norms that were on the internet about the country. AI does make us faster. Your judgment is what makes it useful. That's not just a slogan; it really works.

[00:23:27] I started this talk with the garden metaphor, so just to end it there. I started with curiosity, made mistakes, learned which AI tools flourish and which turn into invasive weeds. I also built a framework, not alone, standing on the shoulders of brilliant researchers who shared openly, and I think I'm also sharing my journey now. These tools will keep changing and models will keep improving, but the principles are the seeds that grow with us, and the garden is yours. Thank you.

Q&A

[00:24:04] Host: Thank you very much. That was a really detailed view of how you've analyzed AI throughout your flow across all the different users, so thank you for sharing that. Just a reminder to everybody out there: if you have any questions, please write them in on whichever platform you're watching on, and we'll put those questions through to Shreya. First off, I want to ask a question I was thinking about. This sounds like a lot of detailed work. How did you get approval or sign-off from your bosses to invest all of this time in the research that you did across the organization?

[00:24:41] Shreya: To be very honest, I initially started out of my own curiosity, and I would work on it on weekends just to improve my practice. Once I had a clear idea, "We can involve everyone, have a workshop, get one or two hours of everyone's time and bring them in," that's when it was something we could scale. Then I could also sell it to the managers and everyone else: it's important, we've done this, and this is how adopting AI in a systematic way helps the team. Rather than creating another project on the side where I'm resourced to do it, I just did it on the weekends and then presented it to the team. That's how it worked initially.

[00:25:29] Host: Makes sense. I guess it's one of those challenges that sometimes you don't get the authority.

[00:25:36] Shreya: That is true. Currently I have three projects running, and if I worked on any other project my manager would say, "We have three projects we need to make decisions on." So utilizing my weekends for this really helped.

[00:25:52] Host: This is more of a navel-gazing question, but there's been a big shift I've noticed, moving away from using AI as a chatbot to more agents, OpenClaw [?] and things like that. Do you think the research you've done will enable that, because you have a much better understanding of each layer of what's happening? Could that be utilized to take the next step, from chatbots towards agents?

[00:26:20] Shreya: It's something I am also learning at the moment. I have tried Claude Cowork, giving it a whole file: "This is the research work, this is what I'm trying to do," and asking it to act like an expert and do the research and thinking work. I've been going back and forth on agents, because I do want to keep the research work human-led in some areas, and there's a lot of work, like cleaning up the data, which is not as glamorous. Agents are something I am learning while we speak. That's why I said I've built this, and I know we'll be rebuilding it again when the agents come and we are using them. But currently a lot of companies are also a little apprehensive about which tools they bring into the company for us to use and which not, so it's a back and forth. I am sure that when these agents are there, we will be reading a lot. I did see myself reading through a lot of slop that it was producing, even when I had very strict prompts telling it a particular format.

[00:27:40] Host: Yeah, it's definitely a learning curve. Hopefully they'll improve. Brilliant, that takes us to time. We want to thank you once again, Shreya. I hope everybody out there enjoyed your detailed flow of how you went from "Where is this going?" right through to helping everybody in the organization with their adoption. Thank you very much.