When AI and Teams Blur the Lines: Who Owns the Research?

Tanya Adlam at UXDX EMEA. Video: https://youtu.be/AF4OpI0-HNQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

A question without a clear answer yet

[00:00:10] Yeah, it is bright up here. Excellent, thank you. When I was talking about doing this in December, I knew the way that we were working was going to be changing. I knew that AI would accelerate the democratization of research in our teams. But what I didn't realize was how much everything would have changed, or at least give the illusion of the acceleration of democratization.

[00:00:37] It's a big question for us, and I don't think there's a clear answer yet. I'm not up here to tell you that we've solved it. But I think there are ways that we are working through it, and I just want to share some of those things with you. The way we are working is very different than it was five months ago, and I expect that to be the same this time next year. What I say now probably will be very different.

[00:00:57] I'm Tanya Adlam and I'm the research director for monday.com. monday.com is an AI work platform where we're integrating AI into everything that we do, both internally and externally. We just launched our custom agent builder, so workforces can build their own agent workforces to do the work for them. We are constantly evolving the way that we work, and as researchers we're doing that as well.

[00:01:29] I think we've heard this stat a number of times, but engineers, especially in the big OpenAI and Anthropic world, are saying that 100% of their code is written by AI, and this is the direction that we're all experimenting with and we're all moving towards. Only a year ago this felt really aspirational, but the intent is real. Engineering is being restructured around this belief, and whole product organizations are being restructured around it.

[00:01:58] The question for me is really, what is it for the people whose job it is to understand the humans that the product is serving? What does that mean when agents can build the spec and build the code, and everything closes in about two days? Where does the research fit? Does it fit anymore? As a function, we need to answer that question.

Role boundaries are dissolving

[00:02:24] I just want to go a bit back into UXR. It's earned its place over the last 30 years, but every disruption that we've had, remote research, unmoderated testing, synthesis tools, all of those have changed how we have worked. Today, this is all about who does the work. That's a really different inflection point that we're at, which leaves us with really big and different challenges.

[00:02:52] We talked about this, but role boundaries are dissolving across the industry, and not just by accident. We're deliberately starting to see structural changes. AI collapses the cost of execution for every discipline. Engineers can design, designers can code, PMs can push code and do their own discovery. And it isn't just an industry trend. It's integral to the way that we're working at Monday. And yes, the Monday builders are doing their own research, or they're doing something that looks like research.

[00:03:26] I just want to talk a bit about how, in the research community, we can often frame democratization as the enemy. That's not how we work at Monday. Way before AI came along, we would genuinely see the upsides to democratization. Monday builders have always been asked to talk to their customers. We built a culture that deliberately, over the years, means that they can talk to customers. UXR have created research fluency at scale.

[00:03:55] Research isn't gatekept. It means that the product instinct is there and embedded across the org. UXR have built the culture that it's normal to talk to your customers, and this is genuinely good. The upside of that with AI is that it can happen faster. There's more volume. It's a lower barrier to get good enough quality for discovery work, and that's the real value we've seen.

[00:04:22] However, it's complicated, because any UXR team that has managed democratization has had to manage that risk. Without quality, you don't get better research. You get false confidence. You get siloed versions of who the user is that sit within teams. Recency wins regardless of whether it's right or not, and the loudest, most recent signals dominate regardless of how that was put together.

[00:04:49] And with AI accelerating all that, it just gets harder to spot those risks. Bad research with confidence is harder to spot. AI synthesis can look rigorous, but not necessarily be. It shapes an insight, but without necessarily the substance. You lose the nuances that research can pick out. And the part that keeps me up at night is that that mediocre synthesis then ends up in the product loop.

What AI is doing to research teams

[00:05:22] I am not a designer, and I got AI to do this. I wanted to also pick out a little bit around what is happening with the teams. AI can be genuinely hard on a team's motivation. We are both celebrating and fearing it, all in one hour. They can see the benefits of what it can bring, but in the next moment we're doomscrolling about what it means for the team. What does it mean for the job that they know and love as it is now? Are they learning quick enough? Can I keep up with the pace that everyone else seems to be running at and actually do my job?

[00:05:56] And it isn't just a UXR thing. I think every discipline is feeling it. But particularly for UXR, we're sitting there with this verification burden on one end. You're looking at the exhaustion of reviewing work that you didn't do, that was built by a system that you didn't create and that you can't inspect, that sounds like it's self-regulating, but probably isn't. And how do you know what's right if you haven't been in the weeds of the research yourself?

[00:06:26] For example, we fed a whole load of interviews into our chosen AI tool. It then talked passionately about what Sarah had been doing and how she felt about our feature. We never interviewed Sarah. She wasn't one of the transcripts we fed into the loop, and we only knew that because we had conducted the interviews.

[00:06:47] And on the other side of that, there's also just this slow disengagement if you're there supervising work that used to be yours. These are the trends that are coming up. I'm not really here to talk about it much more, but I think it's a really important thing, how we manage our teams through this AI change.

Hold the craft, let go of gatekeeping

[00:07:10] There we go. I'm going to look at democratization and that acceleration in a bit more detail. It isn't the enemy. I don't feel that it is the enemy. I've talked about the risks, and they are very real, but they existed before AI, and they are accelerated by it. There is a real impulse to protect research by making it harder to access, and that to me is the wrong response. UXR has spent so long really convincing organizations that the insight matters, and to respond by holding onto who generates that research is contradicting ourselves.

[00:07:50] The risks are that teams are building on compelling assumptions about who the user is, and the quality is collapsing. But the answer isn't to gatekeep it further. It's not to hold it closer. It's to own the craft that we have: to be the people who hold the quality, not the enforcers of quality, but to build the infrastructure that makes quality the default rather than the exception. We're holding the craft. We're letting go of gatekeeping. That's our principle, and how we've worked through the way that we're working with the teams. However, that is obviously easier to say than it is to operationalize.

[00:08:32] Let me be specific about what we've seen AI can and cannot do very well, because I think that nuance here is really important. AI synthesis flattens the signal. It confirms existing data patterns. It lacks intuition. It can't tell you when you're finding an artifact of a method rather than the insight. And it definitely can't hold two contradicting points of view and make a decision on which one is most likely to be true. And it doesn't know what it doesn't know.

[00:09:05] That's not me being defensive. That comes from a lot of tried and tested real-life plus AI experience, and a lot of evidence that I've read elsewhere. If you look carefully at AI synthesis, you will find, and I've read this a lot of times, that it's often slop. Not always, but often it can just be very flat. And the problem is it doesn't look like that. It's written, it's polished, it's confident, and it's cited.

Building tools that work over and over

[00:09:39] Good news: my team is still doing research, and we're doing it with more leverage. We're doing it at higher volume, and we're doing it with the quality bar that AI alone can't reach. And our product teams still want UXR. However much they are empowered to do the research, they still want a researcher in their team.

[00:09:59] There's something that we learned the hard way. The gap between an AI research tool that works and one that consistently produces high quality output is enormous, and it's really invisible until you've built both. The way the data gets ingested really matters. Whether the transcripts come from some automation through n8n [?], or the agent that you've built especially to ingest it, or are just directly uploaded, the same prompt will provide three different versions of quality as its output. It's the kind of thing that just doesn't show up in a demo.

[00:10:40] And while we were all learning this in our teams, the PMs and designers were creating their own synthesis tools and shipping them all over the place, and writing posts about how great their synthesis tool was. And that's right, they were moving fast. They weren't waiting for UXR to catch up. But when we looked at the tools they were creating and the prompts that they were putting into them, they were weak. You get all of the AI synthesis that's flat and non-nuanced, and you're just creating very mediocre insights. That's because it's difficult to write prompts that are reliable, that consistently work over and over again, for different profiles and different question structures.

[00:11:25] And this created real tension within the team. Were we working? We weren't moving fast enough. Are we behind like everyone else? Are we going to be obsolete within the company? But actually, we held our own, and we're delivering tools with UXR approval that are really working and being used across the PMs and the designers, who don't want to maintain and hold their own UXR tools.

The amplification loop

[00:11:48] If democratization isn't the risk, this I think is the real risk: the amplification loop. You start with the mediocre synthesis that AI has done, or the poor synthesis; it doesn't really matter which one, it all works the same. That synthesis then gets translated into product decisions. Those decisions get passed on to an agent that builds the spec. The agent builds the code, and builds it faithfully, because that's what agents do. They execute with precision on the instructions that you have given them. And so you get a wrong or non-differentiating product, not because you were careless, but because that's the information that was input.

[00:12:31] And this can happen anyway with democratized research. You often find non-researchers will validate their own biases and products. But with AI this happens at speed, and it happens before you know it, and you can't catch it. The fix isn't to slow down that loop. It's really to build the infrastructure where UXR judgment is built into the cycle: let research deal with the biggest questions, and then the infrastructure deals with the rest.

[00:13:08] I also want to push back on something that I've heard in conversations, that everything has to change about research, that we have to reinvent the way that we do it from scratch. I don't think that's true. We had embedded models: a researcher in each team, or over a couple of teams, that did everything that came up in the queue for research, from usability through to deep foundational research. That served a much slower world, and doesn't work at the pace that we're moving at the moment. The knowledge rots really quickly, and the user is changing constantly in these fast-moving products. A finding from six months ago will be completely misleading now.

[00:13:53] But research as a craft, I don't feel, is threatened. Rigor, judgment and sustained ownership are all exactly what we need, and they are what makes research irreplaceable. We pick up on the nuance and the differentiation, and that is, I think, where increasingly the product advantage actually is. Our model needed to change, but the craft, I don't think, has to.

A UXR intelligence model in three streams

[00:14:21] Which brings me on to how we at Monday are dealing and working with the teams. We're building what we're calling a UXR intelligence model, so less about being built into the product teams. We've got a core team that will deal with living and breathing foundational knowledge: the personas, the jobs to be done, AI trust frameworks, our sentiment surveys, all the stuff that the whole organization navigates on.

[00:14:48] It's not a repository. It's not a quarterly refresh of data. It's not a Slack channel or a Monday board that people come and visit once in a while and then forget about. These are going to be real programs of work where the organization holds its organizational truth. We're using AI to help build those models and those tools to feed into that, to help create the work and deliver the work in a much more impactful way. And it lives where our builders are actually working. We have it in Cursor and in Claude, wherever those specs are being written. It's a centralized approach, but it can extract more nuance than what we're doing at the moment, and also utilize the work that the non-researchers are doing.

[00:15:35] The second stream we have is a little bit more like the traditional embedded. It's high impact product research. It's fast cycles of research, finished in a very small amount of time, anywhere from two days to two weeks for one of the largest ones, scoped on a single high impact question. Close collaboration with PMs and designers, and AI-moderated interviews to gain the volume without sacrificing quality. We're dealing with some of our biggest pain points, such as recruitment, by using AI to help us create pools in a way that we couldn't do before.

[00:16:11] And we won't be touching usability or simple feature discovery anymore. That will be handled by the teams with the tools that we create for them. In many cases we're just working on really big high-level questions on the product and then handing them over to the teams to work on as they iterate, wrapped up in the AI programs that we've created for them. So it's a centralized embedded model. These are two of the streams that we have.

[00:16:36] And then the final stream, which is probably the most different from how we've ever worked before, is this stream called Enable. It's our job to build the AI tools and infrastructure that power the other two areas and the non-researchers. It's to build self-serve tools that raise the quality for everyone doing their own research, as well as more focused UXR tools that we will use just ourselves, which will live at a higher quality. And we're using AI to shift our methodologies. Just seeing the way that we can work with AI means that we've changed the methodologies and use different things. It's like research ops, but it isn't. It's also product ownership of the research tooling across the organization.

[00:17:26] And all of this is encased in the research guild. We are still training our non-researchers in the fundamentals of research, but also how to use AI within that, because if you don't have that fluency and support network, the research will not be strong or have the quality. It's really around how we approach the AI world, where somebody could just write, "Synthesize this," and see what comes out. We're really scaffolding our non-researchers.

What we've built in five months

[00:17:58] This is where I want to talk about some of the products that we've built over the last three to five months. We've split it into two things. One is a more operational process side, and one is the tools and skills. On our ops side, quite often our non-researchers struggle to understand where they should be doing research and what that research should be. So we've vibe coded a whole interactive end-to-end research process map, which links off to loads of other things.

[00:18:30] It doesn't sound that exciting, but we haven't been able to do it in the past, because we'd have to either pay someone to do it, it takes time, and you have to maintain it. We could do this in a couple of days, and it's instrumental for the non-researchers to just work so much faster. All our incentives are now automated. We don't need to talk to someone to pay someone. Everything just goes out there.

[00:18:57] We have a planner and recruitment agent. Rather than the non-researchers just coming up with a few questions and talking to customers, they put those into our recruitment agent. They get a whole plan of research and a discussion guide, and it's at a level and a quality that we're really pleased with. And we've also created for them an interviewer coach that takes their transcripts and gives them advice on where they could do better.

[00:19:22] And then for ourselves and for the non-researchers, we've created a number of different things. First of all, it's right down there: the prompt library was a reaction. Just seeing all the prompts out there, we created a big vibe-coded prompt library that they could use and pick from. We have the UXR brain, which is our chat. It's just a chat. You can put in, "What are the biggest pain points for our CRM tool?" and it will go through all the research that we have, synthesize it, and give you results.

[00:19:53] We've built UXR skills into Claude Code and Cursor. We're creating a jobs to be done skill. A lot of people talk about jobs to be done, a lot of PMs and designers, and it's loosely based on jobs to be done. We're building that skill to help produce really concrete output. We do a big sentiment analysis every six months. We've created an analyzer that we can give to the team so they can cut the data in their own way without having to come to anyone. It's just helping us move really faster through everything.

Four challenges

[00:20:30] There are four challenges, I think, that we've had. I'm not going to pretend any of this is easy, or that we've come to the conclusion. There are still things that we're working through. One of the big things is that we have more responsibility for quality, not less. And we're still running our own research, and we're pushing the direction of the business. In a democratized world, we're not doing less, and AI is not helping us do less. We're doing more, but quicker and at more volume. It's definitely more demanding.

[00:21:00] Repositioning is really hard. You need the leadership to back you. Because Monday is moving at such pace, we've been lucky enough that the way that we're moving and changing has been really supported by our leadership. But if your leadership doesn't buy into building some sort of intelligence model, you'll end up with the same service model that you had before while trying to do an intelligence model, and it's just not going to work.

[00:21:27] Thirdly, visible value at pace is really hard. The old metrics that we had for research, studies completed, stakeholder satisfaction, readouts attended, don't capture what matters anymore. The value is that the decisions are much better, that the specs have started in the right frame, that the studies that didn't need to run didn't happen because we've actually accessed our knowledge. That prevention is invisible, and the maintenance is invisible. That's all going to need framing in a slightly different way when we come to performance management.

[00:22:06] And then the other thing that I've touched on is just the quality. The quality needed to build these AI tools takes longer than anybody expects, and in an organization that moves at such pace, it's hard to make sure that that quality is there in your product. When our builders spin up a synthesis tool in half an hour and we say we need a month to produce something, they question why we're doing that, why it takes us so long to build the same thing.

[00:22:35] But I say to them, we're not building the same thing. We're building something that works not once, not twice, but over and over again. We're building something that they can use and that we will maintain, and it will work across various different inputs and will provide really reliable outputs. That distinction can be really hard to communicate at times, and easy to dismiss. It's something that we've really had to defend.

Who owns the research

[00:23:03] Just lastly, I think in terms of UXR and research, it's really in our hands. We have spent the last 30 years learning how to translate human behaviors into product decisions, and that in itself is genuinely hard. But the knowledge is real. We're now building the infrastructure that means that knowledge doesn't get lost. We're designing the quality into the tools that we're providing to the teams before anyone starts the research. We're repositioning the way we approach the organization's accumulated information and understanding of its users, and we're organizing and operationalizing that in a different way than we've done before.

[00:23:43] I think researchers that work with this and not against it won't be outrun. I think we'll be making sure that the organization isn't building the wrong thing. And then, finally, the question: who owns the research? I don't think it's as simple as that. I think everyone touches the research now. But research owns the truth of it and the quality of it, and the infrastructure that makes that reliable. That's me. Thank you.

Q&A

[00:24:21] Host: Thank you very much. Excellent. It's research moving fast. I love the flattening and the averaging, because it's all built on averages, so it's generating averages which are weak. Let's see. It seems like it's the number one question whenever you share that you've built something cool: is that something that you can sanitize and share with people, your skills and prompt library?

[00:24:50] Tanya: I'll have a look. I think it's one of those things we've spent a long time working on, but there are definitely prompts that I think are very universal. But yeah, I'll have a look. I can't promise anything.

[00:25:04] Host: So we'll be in touch, and hopefully we'll have something to share from that. Okay. You talked about the issues of low quality AI synthesis, and also about how you've made your own. How have you made sure that your own tools are high quality, and you're not falling into, as you said, the flattening and the averaging?

[00:25:24] Tanya: I have a team that's really, really picky. They're very, very high on quality. Quite often I'm the one that's going, "It's good enough. It's good enough." We run real world tests. We've also tested with a lot of external tools. We've tested with synthetic users. We're currently doing a lot more with synthetic interviewers. So we're constantly checking against the real world. In terms of the skills that we've created, quite often the prompt has to change slightly, and that's where the researcher comes in. We change it slightly, and then they run with it afterwards. But we're there to make sure that what they're putting in is going to get the right outputs, and we know that because we've done a lot of testing.

[00:26:10] Host: And every time there's a new model, do you have to run it again to see how the underlying model has changed things?

[00:26:17] Tanya: A little bit, yes. That's what I mean. It's not less work, it's more work, continuously.

[00:26:25] Host: But it's democratizing.

[00:26:26] Tanya: It's democratized.

[00:26:29] Host: What methods do we have as researchers and designers to ensure other team members and leadership understand the value of specialist collaboration with AI?

[00:26:39] Tanya: Because it gives a very confident answer, but it's not necessarily the best one until you have, as you said, done that specialist work. I think quite a lot of that has come from the work that we've done up to this point, showing those two different things. We've given guidelines around when to use synthetic users and when not to use them. We give guidelines around the different types of skills that we put in. And in terms of how we push that out, it's just that constant conversation.

[00:27:10] But I think what's really important is we've just done a big reorg in the way that we're working, and every PM and designer has come up to me and gone, "Who's my UXR?" And I've gone, "It's not going to work that way anymore." So there's still an appetite, and I think it's because, as you can see from my slides, I'm not a designer. I wouldn't profess to do that, and they don't want to either. They want to be building on the evidence. That's really important, and that's the way Monday has been built. They've spoken to customers.

[00:27:43] Host: So there's already a built-in culture.

[00:27:45] Tanya: Yeah.

[00:27:46] Host: So just change your company culture to have an inbuilt fire.

[00:27:50] Tanya: If you could.

[00:27:52] Host: Okay. Do you see any value in doing qualitative research with AI agents pretending to be people? You mentioned synthetic.

[00:28:01] Tanya: Yes. I'm keen, but from the tests that we've done, what we've seen in terms of results is that it works well with simple usability. We've done some big market pieces, and I think we've seen it does a really good baseline. But I sometimes question whether I could have just typed into Claude, "What do the marketing people think?" and maybe got the same results. I think the value will come. I'm not sure it's quite there yet. I wouldn't be comfortable doing all our research on that just yet.

[00:28:36] Host: But you mentioned when to use synthetic users and when not to use them. So you are using them for some use cases. What would those be?

[00:28:44] Tanya: We use them occasionally, and it tends to be when it's a broader piece that we want. Also, a lot of the work that we do is with a Monday user that's used X and done that, and by that time it's very specialist.

[00:29:01] Host: It's able to pull in the data to make up that persona.

[00:29:04] Tanya: Although we are looking at how to input our own data into those synthetic users and see whether it works, but it's still very much in that experiment stage.

[00:29:13] Host: Okay. "Would you be able to talk through a real life example as to how this works? It's hard to visualize. Thanks."

[00:29:23] Tanya: In terms of the structure of the team?

[00:29:28] Host: Because you've created all of these things, how do you get them adopted by the teams? It's all in aid of democratizing, and this is the assumption, please correct me if I'm wrong. But what if people are ignoring it? How do you make sure that all of your hard work gets utilized?

[00:29:46] Tanya: As I said, a lot of PMs and designers were creating their own tools and throwing them out there. When we first started, there was just that FOMO that I was talking about earlier. Everyone was creating tools because they felt they had to, and they knew they weren't the best tools, or weren't necessarily producing the most accurate output. What we do, once we've created these tools, is go into the big product meetings and the design meetings, talk through the different tools, and then take teams through them. It is that communication of what we're doing. And then those team members don't want to be creating their own tools, so they're utilizing ours, and it's got a stamp of UXR approval.

[00:30:30] Host: And how do you communicate it, though? I guess one of the risks is it's being built in one corner of a big organization.

[00:30:37] Tanya: We have a marketplace in Claude, so our skills are there and they download them into the system. We're also building a layer of what builders in Monday should know, and our skills will be there. They can access them really easily. And then we have our own Slack group where we do a lot of promotion. We've rebranded ourselves recently.

[00:30:55] Host: Okay, brilliant. So it's the same old: you have to publicize it and get it out there. Can you tell us more about using AI for usability testing?

[00:31:07] Tanya: Yes. We haven't done a huge amount of it yet, just because we've done a lot of testing of different platforms. I think what we've probably done is test quite a few platforms, and then we're also looking at whether we want to build something ourselves. One of the things around this model is that the Enable piece has to really work out what we want to buy in and what we want to create ourselves.

[00:31:37] Usability again does give really good results that sound very confident, but we have noticed that sometimes it will say something about a feature, and you spot that it hasn't actually been there and hasn't looked at it. So what we've been using it for is a very simple first pass. The designer uses it, makes the simple changes, and then we would definitely recommend going and talking to real people.

[00:32:05] Host: Brilliant. So it's a great tool, but you still have the real people, and you can improve on the base models.

[00:32:12] Tanya: No, because it can't take the time. We watch people struggle. It just goes through the loop really quickly. It doesn't mimic the real person going through the process. So, yeah.

[00:32:25] Host: Perfect. Well, that brings us to time. Thank you very much.

[00:32:28] Tanya: Thanks.