Democratizing Research

Jessa Parette, Erin Howard, Kendall Avery, Rima Campbell at UXDX USA. Video: https://www.youtube.com/watch?v=iALahafYEnM

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

What democratization of research means

[00:00:01] Rima: Today's topic is democratization of research. What does it really mean? Democratization of research really means that you give everyone in the company access to customers and to data. You give them and empower them, give them the ability to run their own research and collect data, and build a culture of a data-driven organization.

[00:00:23] Rima: As we all know, the demand for user researchers who are talented is rapidly growing these days. Finding those talents in the industry and hiring them is becoming a little bit challenging for us, because right now, according to Nielsen Norman, we have 1 million UX professionals in the workforce, and by 2050 we're going to need around 100 million more user researchers to actually cope with the demand. That's a good problem to have.

[00:01:02] Rima: So with that, democratization of research was born. We have leaders from different sizes of organization who are actually leading democratization programs in their organizations, whether they're centralized or decentralized, so it will be a great opportunity to hear from our distinguished leaders how they support democratization in their organization today. Maybe we'll start with Jessa.

[00:01:31] Jessa: Hi, thank you. It's so great to be here. My name is Jessa. I currently work at Capital One. I lead research and design for all of Capital One's auto finance, with our research, our design strategy and our design system. For democratization, just to lay the groundwork, you have to define what it means and what it doesn't mean. That is the way that you figure out how to scale, and that's the way we've set up the different structures in our team.

[00:02:00] Jessa: One size does not fit all. You will have to adjust as your company grows. But it starts with really understanding and deciding the battles that you fight in terms of what does democratization mean, what does it not mean, and when does it happen, because that will tell you things like budget, head count and team size. I'll pass it over to my other panelists.

[00:02:23] Erin: Sure. I'm Erin, I lead research and design at Charles River, and we're in a different phase of our organizational journey through democratization right now. I actually don't have a user research function. My product designers do all generative research, all the research that we do. We do outsource to some strategic partners for some of the very skills-based survey design, etc. But we're in the phase of hiring generalists so that we can create potential career paths around research in the future, bring research as part of our organization through our whole discovery life cycle, embed it alongside product creation, and adopt a continuous discovery model as the way we work moving forward.

[00:03:14] Kendall: And I'm Kendall, I lead the Rider and Maps research team at Uber. Our democratization journey has gone a little bit from research did everything. The research team owned all research processes. Part of it was because we had a really large team, and then as our team started to scale, the types of problems and projects we were getting pulled onto, we realized that at a smaller team size we didn't have the ability to staff and support all the different projects and areas that we really needed to.

[00:03:41] Kendall: And so we're in this mode of actually introducing more democratization, starting with defining it, really understanding what is it that we are democratizing, what is it that we are not, what are the guard rails, what's the governance that we have to incorporate there. And then really making sure that we're setting those expectations with our partners to say, this is what research means, this is the brand of research, and how do we make sure that we maintain that brand quality of research even if researchers themselves are not the ones leading it.

Running the program: self-service, structure and guard rails

[00:04:13] Rima: That's terrific, thank you, ladies. I hear from our customers at UserZoom often, and I had my own experience back at Citi, where democratization of research is really a new node. Back like six, seven years ago, only specialized researchers should be doing research, whether it's generative research or just purely validating design across the design life cycle. So I guess the question here would be, as you establish democratization and have that practice, how do you run that program? Some customers ask us to run boot camps to teach research 101 to product designers, or designers, purely non-researchers. Others do a train-the-trainer program. So maybe talk a little bit about that. As you established that program, how do you do it?

[00:05:15] Jessa: I'll speak to my most current experience. Capital One, being a larger organization, we have the luxury of being a larger organization, but also some of the challenges: the demand for what people want and need will always surpass however many researchers or designers that you will ever have.

[00:05:33] Jessa: So the way we are structured, I've actually split research operations and researchers. There's two different functions. Research operations is centralized. The reason we do that is at the size and scale of our company there are certain legal ramifications, tools, process, access, licensing that you really do need to have a central area of control in order to allow product teams to self-serve, because not all product teams will have researchers.

[00:06:04] Jessa: Now, researchers still roll up to me, but researchers are embedded in the lines of business. They work with the product owners, they work with the product teams to prioritize. So we have a combination of both: a self-service model where product teams can do their own research, and our operations team supports the legal side of it, making sure it's compliant, because being a bank, I promise you we have more regulations than you could ever think possible. It's really fun.

[00:06:32] Jessa: And then there is researchers in those product lines. That's how we're set up, and there is training that comes in both of those aspects. Managing it will always be a conversation of demand and budget versus size and scale, and it really comes down to: you're saying you want to do this, it will take X number of people. Do I get X number of people? No. Then you don't get what you want. Which is not always easy to have, but that's how we're set up.

[00:07:06] Jessa: We balance it carefully, because we had an average of 76 project demands within our first quarter, a 59% increase in the demand for researchers, and we had zero additional head count added. So those are some of the challenges you have to be able to manage at that scale.

[00:07:24] Rima: Excellent.

[00:07:26] Erin: For us, our growth is pretty rapid. We've gone from one designer to 10 in the last 15 months, and with that we're actually using each other as our network for standards, our creation of processes. We're building the plane while flying, if you will, in many ways. But we do a lot where we co-work with each other around creating our frameworks for the conversations that we're having, or the outcome results guidance documents that we're creating, or how we're doing the analysis on moderated testing. We use each other as guides and trials there.

[00:08:04] Erin: We're not yet empowering that activity directly in our product teams or engineering teams. We're not at that training point, we're still owning that ourselves, but we're working on how we figure that out as part of our agile model and our continuous discovery model, and how we do that at scale and speed. So I see our lives changing and looking a little bit more structured in the future, but we're just not quite there yet. We're all forming as we're going, and normalizing processes where we can along the way.

[00:08:39] Kendall: I think from our standpoint we're looking at how do we add the right process around what should researchers be taking on, what should be democratized. Part of that is we very recently started looking at what's the research engagement model. On a matrix of business risk, high business risk to low risk, and then level of uncertainty with designs or questions that we have, we've broken it out into four different buckets. We're calling it a Bop It, because there's take it, manage it, enable it, pass it. It kind of felt like a toy.

[00:09:14] Kendall: And so based on where the business risk and uncertainty fell, that's where we're able to make, from a researcher standpoint: is this something that needs to be owned by researchers in-house, we're the ones with the expertise, we have to be the ones executing it? Is it something where we don't have the capacity to support it in-house, but it does need dedicated, qualified research support to be owning it and getting us the insights? Maybe that's where we pull in a vendor, a contractor, we look at other ways we can get this data.

[00:09:43] Kendall: The enable it bucket is really where we're starting to experiment with democratizing. So even within this larger guard rail of what's our engagement model of researchers, then within enable it: okay, how do we enable it, what are the guard rails within that? We're starting to build out templates, we're starting to build out playbooks, things that are very specific and not allowing for a ton of wiggle room within those. If you're going to enable it, you're going to follow these standards and practices that have been agreed upon across the entire research org. So we're trying to centralize some of the standards and the way that we as researchers work.

Setting the standards and knowing whether they're applied

[00:10:22] Rima: Kendall, I love the idea that you brought up, the standard, the research question, the task question. How do you establish those, how do you share them, and also I wonder, how do you know they're applied correctly?

[00:10:40] Kendall: Great question. The way that it ended up actually working was a little serendipitously. One of the designers on our team had experience at a previous company leading some research. She wanted to unblock herself and just be able to lead research, and so she was empowered by her manager to take on what does it look like from a designer's point of view to run self-service research. We found out that there was a researcher in another org that was trying to do the exact same thing, and so we were actually able to bring the two of them together and say, from a design and research point of view, what does it take for a designer to feel confident in the research?

[00:11:19] Kendall: Because I think one of the things that sometimes happens is when researchers are the ones democratizing and leading the process, we forget how much we know, and so we forget what's important to teach. So bringing in that designer to also say, hey, if I'm the one executing, what are the questions I actually have? And then having that researcher there to make sure, hey, you should add a SUS or a UMUX, there's something at the end of it to make sure that it's a standardized questionnaire, you're asking the same thing of everybody, this is how you make sure it's not a leading question. So there was some guidance over it, but I think what's really making it successful is that it's not just prescriptive from a researcher's point of view, it's really collaborative from research and design together.

[00:12:01] Rima: Love it, love it, that's pretty cool. Now I wonder if Erin or Jessa can add to the quality of the insights coming out of those types of research that you decide to empower a designer or non-researcher to do.

[00:12:16] Jessa: You should answer first.

[00:12:19] Erin: I think quality is a difficult thing, because unless you can — for us, we don't have a baseline to measure against, we're establishing it. So we're starting to learn and pressure test our own assumptions and quality, and work against our own previous experience. Okay, how did I do this last time, did it work, did it not work, did it give me the answer I needed? Okay, what do I need to change? And continuously improving from that perspective for ourselves. But at the end of the day, did it help you move your product forward, did it help me make a decision, did it connect to the business need, do you feel informed about what you went to research? I think some of that is where the quality comes in.

[00:12:58] Erin: It also, to your point, the level of severity of getting it wrong impacts whether or not you feel like you can measure that quality. And when you start to get into those more severe problems, that's where you tighten those guard rails. Sometimes we have more than one designer on those more tightly guard-railed calls or sessions, or we engage with a preferred service provider for a survey product, because we're not survey writers. Because we know we can't achieve that, we know we have to outsource that and bring a vendor in to support that piece. So there's a little bit of that balance.

[00:13:36] Jessa: And to add to what you're saying, one of the things that I found is helpful in defining with a research team: take a two-in-a-box model. Two squares, two squares, four all together. And understand what is quality defined, what is quality undefined, and then in your next plane, what is quality that is allowed, what is quality that is not allowed. You can start to bucket things. For example, quality needs to be defined in the number of users that you use for a usability test. You can't just talk to three people and then make a product decision that's two million dollars because you called someone up. Okay, we're going to define quality in terms of the usability testing.

[00:14:18] Jessa: Quality could be a little bit undefined with the product team when it comes to things like color, template, format. I don't know, we're just using that as an example. When does quality really, really matter, and when is it okay to be a little bit loose? Because in democratization you're going to have to choose what battles you fight, what you are allowed to do, what you can give control up on. Maybe you want a standardized template for everything, that'd be really, really great, but the bigger battle you're now fighting is the fact that you have teams that are going to the field and they are talking to customers and they have not had any sort of consent form signed. Oh my goodness, that's a bigger risk.

[00:14:58] Jessa: So what can help is define those in those buckets, and you can start to say, okay, here are the things that I'm going to let product teams do in this undefined bucket. Here's the things like, yeah, go nuts, go for it, not a problem. Here are the things that we absolutely need to have a product roadmap for, or be working towards, to really clearly define. And oh, by the way, I'm going to need a person, full-time head count, to manage that at some point.

[00:15:23] Jessa: I have found that has helped as well, because when you start to grow out of that, hey, now we're starting to define, you're going to get to a critical point where the ramifications are not just preference. The ramifications will hit your product, the ramifications will hit your platforms and your services, and heaven forbid the ramifications will hit the media outlet because of something that happened. That's when you find really, really quickly what a small decision should have been made before that was not made. So just to add to that, it's a helpful, quick back of the napkin that you can do with your research team or with your designers, and come to an agreement, and then gather agreement with your product team if you're in a pinch.

Where insights live and who maintains them

[00:16:07] Rima: Lovely, loving the risk aspect of it, balancing all that based on expectation in different organizations. Now, where do you store all this data? How do you make it accessible? What kind of tools do you use? What kind of reporting do you make accessible, is it snippets, is it highlights? Tell me a little bit about that.

[00:16:35] Kendall: Building off of the conversation, I don't know if folks joined the research and design ops conversation earlier, but they're an invaluable resource. So if you're able to grow a research ops team, I think it's been incredibly helpful for us. They help us manage — we have report templates that for the most part we use for all of our research reports. And one thing that I've learned there is it's not just the quality of the insights, it's the quality of the storytelling and how you're actually sharing those insights and whether or not it's going to land. So bring in design partners to help create those templates, make sure that you're able to tell a beautiful story that is accurate and data-driven and something that's going to move the business forward. That's one thing that we try and use, templatizing things wherever possible.

[00:17:19] Kendall: We also have an in-house research repository that is really useful, and we've been really encouraging the team and finding efficiencies wherever possible. We found that some of the teams were already doing their own research repository in a spreadsheet, and instead of asking them to pick up how they were working and change to a different model, we just added a Google script that could pull everything in. Basically, how can we make sure that it's the most efficient way possible, that all of the insights can be in one centralized location without adding additional workload onto the researchers themselves, but finding different ways that you can make those efficiencies up front so that they just become part of the process, it just becomes standard?

[00:17:59] Kendall: I think that's part of democratization. The scale piece of it is really where it comes into play, and it's not just scaling the skills of research across your team, but giving them the tools so that they can scale efficiently and effectively and not feel like they have to learn how to create a research report and how to do the research and how to do all the legal stuff at the same time. The more we can just give it to them up front and it's a little bit more plug and play with research oversight and guidance, the more efficient your teams will be.

[00:18:30] Rima: Excellent. Erin, any perspective?

[00:18:33] Erin: Only that I think if my teammates are watching this they're going to go, yes, Kendall, all of that. Because we're on that precipice, we're at that moment, we're at that moment of scale, and so now is the time that we're beginning to look at that research repository tool, that ability to cross-share. Right now, sharing our findings and our discoveries and our direction and our connection to the business happens through our demos and our agile process. It's part of the demo structure, it's how it gets out to the organization.

[00:19:00] Erin: But we're finding the opportunity now that we really can help cross-pollinate each other, and that research repository is part of our next step. They'll probably want me to go talk to Dovetail really quickly, but there's a lot of tools out there and so we just haven't embraced exactly what that looks like for us. But we know it's on the horizon, it's part of our next growth, and I truly believe it'll be an unlocker for us in how we partner across our teams and our sprint and our scrum teams, and decouple those independent silos that that's creating.

[00:19:36] Jessa: We've gone through, with the different teams I've worked with, different approaches in terms of managing insights, and you'll probably find that you'll start something and then stop it, and it doesn't get off the ground, or it starts and then someone else discovers something. That's a really, really good reason why you need research operations, because guess what, when people don't have something they might assume that it doesn't exist, and they're going to go build their own, and then you have someone who's doing duplicate work.

[00:20:06] Jessa: But we've used everything, starting with some homegrown things. What is accessible to everyone in your enterprise? That's a really big question, because you might have an instance where if you have a centralized licensing platform where only certain people get a license to a tool, and you use that to house your insights, well, suddenly your product owners may not get access to it. That's a problem. So what is central to your organization that you can kludge together or use that everybody has access to, and then how do you manage it?

[00:20:38] Jessa: We've used things from Airtable to, in smaller teams, Excel spreadsheets, to Google Docs, to Google Drives. We've even used Jira at some point to understand what insights were requested. And then of course there are more standardized tools which are great, like Handrail, Dovetail, some other things. There are systems that house insights for those research tools. The challenge is, how do you manage all of that?

[00:21:06] Jessa: The skill set that comes with it is not just a researcher, it's actually program managers, process managers. There are specific information architecture skill types that go with how do you standardize tagging a Google presentation so that it is actually searchable across the enterprise, so that if you have it in a centralized location everybody can look for it. Some of the things that we've done is also, you really need to understand how people search for insights. Product owners might go, do people really like eating in the cafeteria? Well, you need to understand semantics and how people search for things, because if your documents aren't tagged correctly and managed correctly, no one will ever be able to find them.

[00:21:51] Jessa: So it takes trial, it takes practice, but then it takes the discipline of someone maintaining and sustaining that system, or I promise you, I absolutely promise you with every fiber of my being, it will become wild, it will become unruly, and then it will become unusable. And then you'll have someone going, we don't have anything for our insights, we'll make something, and you're like, oh my gosh, it's like Groundhog Day, we're just doing the work all over again. So that's where it's like, experiment, yes, but then also really clearly understand when that trigger moment is of, okay, for hell or high water, we've got to get someone to manage this way of doing insights.

The shelf life of research data

[00:22:33] Rima: I love it. This rising of research operations as a role becoming really critical as the team matures, making sure the data is accessible, how you tag it, all that, to make sure, because you're a consumer of insight. But what about the shelf life of that data? We know from what we've been through in the past couple of years, a lot of data is really not as valuable anymore, because consumer behavior has changed, everything is shifting to digital. So how do you manage the life cycle of that data?

[00:23:16] Jessa: So I look at it in three ways, and this is a rule of thumb, it's not a do-it-every-single-time. Insights and data, you need to bucket them into defined types, or else you're going to come up with the generalization. Qualitative insights tend to — where maybe they had a longer shelf life of like 12 months prior to the pandemic, I've been talking to some researchers in the field, some of my friends, and what we're finding is that those insights now actually have a shelf life of maybe six to eight months. Qualitative insights, like empathy studies, user interviews: we are changing faster and the demographics are changing faster due to some very turbulent times. That's a very mild way to put it. So your qualitative probably has a shorter shelf life than before.

[00:24:04] Jessa: Quantitative information then falls into two buckets. There's the automated one, where it's part of your system, you are measuring and you're automating and the insight is coming from a byproduct of your system, like bounce rates or number of users in a system, number of click-through rates. Those things tend to have a little bit of a longer shelf life. Dust it off every 12 months, make sure that your systems are integrated, your formulas are still correct.

[00:24:28] Jessa: But then moment-in-time quantitative becomes outdated the minute you deploy your next iteration. You can't use, oh hey, this was the benchmark and this is how much it improved, and then six months down the road pull that study and go, we improved it this much and this is what users said. No, no, you've had two deployments since then. Your fundamental landscape has changed. You can't just go, this was moment in time, and we're going to use this moment in time to determine something 12 months later. That becomes outdated the minute you deploy something new. But I'll defer to the others.

[00:25:07] Erin: I have a tendency to just agree. But I think some of the things around sentiment and empathy and some of that stuff, it a little bit depends on your industry too. We're B2B versus B2C, and I think our B2B landscape tends to change a little bit slower than B2C, or B2B2C, and so we probably have a little bit more length of time with that, especially if we're using it more directionally and then we re-dive in based on that direction, versus choosing it as law. But not much longer, maybe a year. It does have, I think, a slight tweak depending on your industry.

[00:25:51] Jessa: And I think there's also a caveat, just to add to that: there's a difference between a trend and an insight, and there's a difference between an insight and a fact. And those can get pretty convoluted sometimes. Trends tend to stay a little bit longer than insights, which tend to be a little bit more moment in time, or maybe have a little bit of a half-life of understanding a product. There's definitely a difference. So I would totally agree that it depends on your industry, and it depends on, are you calling a trend an insight when it actually isn't?

[00:26:26] Kendall: I think from our standpoint we look at what past research do we have when we're in the prioritization process. So if there's a new research question that comes up, it's, is this actually truly a new research question, or do we have research from the past? Sometimes we have studies from like 2018 and we look at it and it's like, does this still feel right, does it feel like there's something fundamentally different? Because I think when you have a smaller team you have to be ruthless in your prioritization.

[00:26:54] Kendall: So what we've tried to do with our partners is say, if they're requesting basically the 2018 study redone for 2022, is there a new question we can add to the mix, can this add a little bit of extra? Really relying on lit reviews, taking a look at all of the research and saying, yes, let's double check that these things either are still true, maybe they're not true, but making sure that you're still moving forward with everything and always trying to identify that there's this net new question yet to be asked, to just make sure that you're constantly bringing additional value to the research.

[00:27:32] Rima: Agreed, great point, great point.

When the model stops scaling

[00:27:36] Rima: Let's talk about scalability a little bit. As the team grows, how do you, and at what moment in time do you say, this model doesn't work, it's not scalable? Can you talk about that a little bit? Let's start over there.

[00:27:54] Erin: For me, I think that that moment — scale is perceptive to the size of your organization. Scale for you could be from one to two versus 10 to 20. So that moment is really, I think, sometimes going out to prioritization: how much are you able to get done, how effective are you, how are you able to connect with the problems, how do you understand the business strategy? And if that's starting to get stretched, then you're at a scale moment. And that could be a small scale moment or a big scale moment, or a big adjustment to the way your organization is designed, or how you support, or whether you are a service offering or an embedded service. Some of that starts to come into play.

[00:28:39] Erin: So I don't see scale as needing to be this massive growth moment. It's just small triggers. It's just recognizing that something's changing the way you're working, and now do you need to readjust and recalibrate and maybe ask for more resources, or re-prioritize, or move away from a project, or look for a tool? But scale's sort of relative.

[00:29:02] Kendall: Couldn't agree more. I think it's really identifying the reflection moment. It's looking back and saying, okay, take the last six months, how many projects have we been able to do, what types of projects? Okay, what are the things that we couldn't support, why couldn't we support them, was it the right decision not to support them? One thing that we've been looking at with why democratization is something worth investing in for our team is sometimes there's a lot of smaller questions that are low business risk but high uncertainty that just don't fall into our camp in terms of all the other things we have to prioritize.

[00:29:39] Kendall: But one of the things that that leads to is then the designers maybe don't have the confidence that they need to feel confident in their work and move that forward. And so we have to introduce democratization, because that's a scale gap for us. We aren't at a position where we can support designers and their confidence in making sure that their designs are getting tested, and whatever it might be for them, because we have so many other generative and foundational efforts that we're taking on.

[00:30:08] Kendall: And so recognizing not only just what are the types of research that you're doing, but take a look at your team and see, are there areas that my team is not being supported in, and what are the things that I need to do as a researcher to help support them, and really embed yourself as that research partner and not just a resource who owns the research and then kind of disappears? It's really, how do I make myself a partner to all these different teams and figure out how I need to scale our practice to support all these individual functions?

[00:30:37] Jessa: There's the scaling of practice, and then there's the scaling of the things that help the practice, and it is very tempting for companies not to recognize the latter. Because oftentimes we will ask people to take on more and more and more, and then you sit back and you realize 50% of a researcher's time is actually creating Jira stories, or figuring out the tool that they need to use, or talking to legal about the compliance, and you're like, hang on. All right, that might be a moment where you need to talk about what needs to scale.

[00:31:12] Jessa: So indications can come when you look at job expectations versus what people are actually doing, and that's harder than you think. It means actually having some very interesting conversations with your people and making sure that you're actually looking at workload and burnout.

[00:31:30] Jessa: So for example, going back to define what you need to democratize and what you want to democratize: are you going to democratize the governance of insights? Is that a decision you're going to make? Can anybody just write an insight, or is that something that you are going to say, no, no, that needs to fall under the highly controlled quality, highly managed? Well, when your company starts to scale and you start to add in more and more products, more and more developers, there's going to be a point where researchers are not going to have the capacity to have that same level of quality. That's an indicator. Okay, we decided that the governance of insights was something that we're going to democratize, and we're now at a scalable moment where we need to bring in operations to actually manage that full time. So that's an indicator.

[00:32:15] Jessa: And it's also an indicator of scale: what do you need to scale? Is it doing the research, is it understanding the research, is it managing the tools, is it managing the platforms? What is it that needs to scale? Because while you will always feel the pressure, and I'm sure you always feel the pressure, everybody wants everything all at the same time, what would relieve the pressure enough that actually you have a leeway of about six to eight months, or maybe longer, to continue as is?

[00:32:47] Jessa: And it might be, hey, we actually just need another researcher, we don't need to scale in our operations, we actually just need another person. Or it might be the flip side: okay, the skill sets that a researcher has, quantitative, generative, evaluative, those are no longer fitting the type of work that we have to do 50% of the time, we have to actually add a program manager for research. So it's not an either/or, but it does take careful management and understanding the strategy of your team and identifying what I call the triggers: when this happened, it's my trigger that I need to have this discussion. Don't just let it happen to you, because otherwise you will be controlled by the tides of the product, versus helping the products mature in terms of being able to self-service.

Q&A

[00:33:34] Rima: You actually answered my next question.

[00:33:36] Jessa: Oh, thank you. Sorry.

[00:33:39] Rima: I was just literally going to ask, what's the future of research, what are the biggest challenges, and I think you all covered all that. Thank you so much, this is great. I wanted to turn to our audience and see if there is any question that I may have failed to ask and you would like to ask one of our panelists. Anyone? I don't know if I can see.

[00:34:12] Audience: Hi there. You made a really interesting comment about defining insights versus other things that are maybe conflated with insights, like facts or trends or recommendations. How do you define an insight?

[00:34:30] Jessa: An insight is an amalgamation of three different data points that configure to a unique understanding of a situation. So it is not a trend line of, oh, we're seeing more and more people buy milk, that means X. Hang on: we're seeing more and more people buy milk, and more and more people are doing X, and there's a decrease of this. The insight is that new thing. So it's two to three actual different points of data coming together that generates a new thought. That is an insight. An insight is not a trend line. That's how I would define it, but feel free to absolutely disagree with me.

[00:35:10] Kendall: I think I've heard this a lot, like what's the difference between an insight and a finding? A finding exists, there's proof of existence. If a person — let's say I talked to six people and five of them roughly say the same thing and one person doesn't. That's the finding. They said that thing, they exist. I don't know if I'd call it an insight because I only heard it from that one person.

[00:35:34] Kendall: And so one thing that I'm thoughtful of, and I'm actually thinking maybe we would incorporate into our democratization, like the synthesis piece, is making sure that you're including multiple data points, multiple voices. One thing I think is super strong is when I'm crafting my insight, if I have three different quotes from three different people roughly saying the same thing, that's really hard to argue with. What are the chances that three people who've never met each other, live all across the world, are saying roughly the same thing? There's something there. And then to strengthen it further, is there quantitative data that I can bring in, how do I triangulate this to really strengthen it?

[00:36:13] Kendall: But it's not to say that that finding does not hold any weight, it just needs to be further investigated. I think that's one of the risks in democratizing, is making sure what's the governance around findings versus insights, and how do you make sure that you can maintain that quality brand of research, that you can trust the level of work that's coming out, but also letting people discover those insights, because that's the light bulb moment for a lot of folks.

[00:36:41] Erin: It's interesting, I'm the only non-researcher on the stage, I think, because I don't actually have a formal training background in it. But what I would say is the insight is what connects you to the business strategy. It's where you have the opportunity to be influential. A finding, an observation, a theory, that's all great, that's all really valuable, that sparks valuable conversation and valuable ideation. But an insight is evidence that can inform business strategy, and evidence requires different aspects of facts, different pieces of the equation, whether that be what the customer said, what the user said, what the business needs. All of that comes together to a precipice, and those are the pieces that I feel like are how research and the activity can be super influential.

[00:37:35] Jessa: And I also think of it this way. Talking about an insights library, where you're putting your research findings: what's more helpful, having just a list of here's statistics that you have to then interpret yourself, or having a very thoughtfully, well-crafted, these are three different things happening, here's an interpretation of those three things, and here's possible outcomes? That's an insight, that's different.

[00:38:02] Jessa: So a thought experiment I'll do is this. Okay, there's a rise in automated self-driving vehicles. With the rise in self-driving vehicles, traditional vehicle servicing organizations, like Walmart servicing organizations, their workforce is mostly maintained of skilled mechanics. But with the rise of self-driving vehicles, and with the skilled mechanic field, are you going to have to have a moment in time where you're going to have to start hiring engineers to staff your Walmart tire and automobile sections? And so we're missing a third piece, we're missing the last moment, I still don't have that research, but that kind of gives you an example of two different facts coming together giving you an insight of understanding. Do we need to drive the business strategy in an auto tire, does O'Reilly's Auto need to start hiring engineers to be their service people instead of just mechanics? That's the difference between an insight and just a kind of fact finding.

[00:39:05] Rima: That's excellent, excellent. Do we have one more? Time for one more question. Let's try. Anyone, go ahead. Yep, just throw it over, like a beach ball.

[00:39:24] Audience: Organizing the findings itself: do you think that's a job for researchers, or do you think that's something for the UX ops[?] job? It's kind of a trend of people creating their own systems.

[00:39:47] Kendall: I think if you have a research operations team, if you have operations people who can help develop that framework, absolutely lean into it. If that's not the case, I do think it falls on the researcher, as the work isn't done just because the report is done. So how do you make sure that you are maintaining, I call it stickiness, what are the different ways that you're making sure that your research is sticking with the people that it needs to?

[00:40:15] Kendall: If that's creating a repository on your own, I think this kind of goes back to the insights piece, is making sure that you have a point of view that goes along with the research insights, and so not just saying this is what we found, but that next level of, this is what it means to the business. And then one thing that I'm starting to implement with my team is we're calling them point of view docs, and so it's a combination of a lit review of here's all the past research we've done on this topic, so I have it all consolidated in one place, but then what's my point of view based on all these findings, what do I think the business should do moving forward?

[00:40:52] Kendall: And from a lot of product folks I've spoken with, that's what they're looking for: okay, we learned this, what's next? And so a repository can be a place to get that, but I think the more valuable piece is that commentary and narrative that comes with the insights, and I think that typically falls on the researcher, and then also creating a good alliance of partners who can also advocate for the impact of research and how it can move forward.

[00:41:18] Rima: That's an excellent point, how you make that insight actionable in that moment in time, contextual. Terrific, thank you so very much. I hope everyone enjoyed the conversation and learned from it, that they can go apply it. I appreciate it.

[00:41:33] Jessa: Great discussion, thank you all so much.