Building an Organisational User Research Culture

Caylie Panuccio at UXDX EMEA. Video: https://www.youtube.com/watch?v=DU6aROEcPMw

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Why democratize research

[00:00:00] Hi, everyone. My name is Caylie, and I'm a UX researcher at Atlassian, speaking to you today from the lands of the Wurundjeri people of the Kulin Nation. Now, Atlassian is a pretty cool place to work as a researcher. Our tools and processes are pretty well established, and we have decent researcher coverage across our products. However, that's not always the case in other organizations. As researchers, we're often in a small team or a team of one, and to keep up with demand we need to upskill our cross-functional team members, in particular UX designers and product managers. We definitely need to ensure that the research conducted by our product teams is of high quality and answers the right questions.

[00:00:39] In each organization I've worked in, researchers just can't keep up with the demand for those juicy insights to inform decision making, and this creates a need for democratizing research. I have a fair bit of experience in democratizing research while maintaining research quality. In this talk I'll share with you how to go about democratization in your organization while maintaining research rigor, as well as your sanity.

A little bit about me

[00:01:03] But first, a little bit about me. I've never really known what I wanted to do. I suggested to my careers counselor in school that I go on to study anthropology, because I'd always had an interest in humans, and he replied, "Well, what will you do for a real job?" We got along well, so this is actually quite funny. So instead I chose to study Japanese, Mandarin and linguistics, way back when, because I apparently like a challenge. At the time I wanted to be an interpreter, and then I changed my mind.

[00:01:33] I then studied a master of banking and finance, thinking that the combination of money stuff and Asian language stuff would be useful to someone somewhere. Luckily for me this hunch proved pretty correct, and I landed a graduate position at National Australia Bank, which is one of Australia's largest banks. I was then lucky enough to rotate through the experience design team, and I was hooked. It made just so much sense to me to conduct research with the people you are actually building products for.

[00:02:00] I remained at NAB for a few more years as a UX researcher, and then I jumped across to SEEK, one of Australia's largest job platforms. I spent the past year establishing the design research practice at Coles, one of Australia's largest supermarkets, and I've recently landed at Atlassian, a tech giant and Australia's most mature research practice.

[00:02:21] Cool. With that out of the way, what's the recipe for building an organizational research culture? Our method for the recipe has four steps. First, you need to learn the signs that it's time to democratize. Too early in your organization's journey and you won't get any takers. Too late and it will be happening without you, potentially in risky ways. Next, you need to set up your UX research team for success. When you democratize, their roles and ways of working will be changing a little, and you need to prepare for that.

[00:02:53] The next step is ensuring you've got the right tools, processes and education in place, so people are empowered to conduct research well. Finally, once you're getting all those juicy insights, you need to make sure they're being heard in the right places and at the right decision points. I will say now that I've written this talk assuming that there is at least one UX research person in your organization. If you don't have a UX researcher, that's okay. I'll point out where in this talk you can take parts and adapt them to your context.

Recognizing the need to democratize

[00:03:30] Cool, so let's dive into the first step: recognizing the need to democratize. Democratization of research is so hot right now. As many of us may be feeling, for whatever reason, the demand for research grows, and UX researchers in the job market just can't keep up with that demand. That demand could come about as a result of a few things. Maybe it's industry trends like continuous discovery or customer centricity. Maybe it's the organization realizing that research is important and wanting to do more of it. Or maybe it's just UX and design and product maturing like other specializations.

[00:04:06] To keep up with this demand, many research teams turn to democratizing their research processes. As with just about anything in UX, there are lots of definitions out there regarding what democratizing research actually is. For the purposes of this talk I'm going to define it like this: democratizing UX research means enabling our cross-functional team members to take part in or carry out UX research activities.

[00:04:33] Let's look at how a research team might reach this point, so you can recognize it in your own organization. What I'm going to go through is based on my experiences at NAB, SEEK and Coles. What I'm about to talk through aren't by any means hard and fast rules. This is a generalization of what I've seen and experienced working in a few different places.

[00:04:54] I'd like you to imagine a small research team in an organization where UX research is relatively new. This team might be a team of one or a team of a few people, and it's centralized. They all hang out together. When a research team is relatively new to an organization, people with the job title UX researcher typically conduct all types of research across the product development life cycle, or Double Diamond. They might be supported by the cross-functional team, mostly design or product, for things like note-taking and synthesis, but the researcher is ultimately responsible for the whole lot.

[00:05:29] As the research team gets some runs on the board, the demand for research intensifies. This demand could also coincide with industry trends, as I mentioned earlier. This is your tipping point. This is the point where you need to start thinking about how to democratize, so research runs smoothly for everyone and you and the organization are getting quality results. Your tipping point is telling you two things. A, you need to decide how to best utilize your dedicated researchers, and B, you need to train the designers and product folks to do some research and do it well. Now you know when to democratize, let's look at how we make that successful. The other parts of the diagram I'll touch on a little bit later.

Structuring your team for success

[00:06:14] We're up to the second step now: structuring your team for success. Being a UX researcher can be pretty tough. When you're a UX researcher, you're often confronted with really tight time frames to get research work done, no one really understanding what the point of you is, and your colleagues getting sick of you telling them that their direction or their design needs to change. It can be a lonely world being a researcher, and researchers just need to feel the love. They want to feel loved.

[00:06:45] We touched on embedding researchers in the previous slide, but that needs to be balanced with the researchers having a home base of sorts: fellow researchers with whom they can swap craft tips and who they can rely on for support. We are humans at the end of the day, and humans need other humans to empathize with their experiences.

[00:07:04] Let's look at how this plays out in practice. Imagine you're working for a university, on their website, and there are five product teams: product teams red, blue, purple, green and orange. Product teams red and blue work on the student side of the experience, while the other product teams work on the university staff side of the experience. Here they are. In this example we've got our team of researchers over here, and we've got the product teams; as I mentioned earlier, there are five of them. Say in this team you have two researchers, these little folks in yellow.

[00:07:44] Up until now, in your centralized model, they've been jumping between projects and product teams as demand dictated. However, things are getting really busy now, and they can't keep up with the demand. You've reached your tipping point. You've chatted with your researchers, and you all agree that training the team to take on some research work will alleviate demand. But where does that leave the research practice? This is where you create domains, or product spaces, for your researchers to hang out in. Let's look at what that's like. Here's one, and here's the other.

[00:08:17] Doing this has a few benefits. First, by sticking to a single product space or domain, researchers can take more time to become really familiar with the product, its strategy and its stakeholders, as well as the current work. This means that the researchers ultimately deliver stronger, more actionable insights that aren't a repetition of what the product team already knows, or thinks they know. The researcher's increased product knowledge means that they can talk the talk and ensure the insights are applicable at the right time in the product's life cycle.

[00:08:48] Finally, the researcher has a product team to belong to, to do product-y things with. If the researcher in question wants to learn more about UX techniques or product management techniques, they can really easily do that. More importantly, they're included in stuff like team events, workshops, emails and, most importantly, any key decision-making forums about that product. So in that diagram from earlier, the one with the different team types, you're now at the midpoint, where you're starting to have your researchers partner with product teams, and there are more researchers.

Defining who does which research

[00:09:23] Let's look at the next steps to make that partnership a success. We've allocated our researchers to product domains, so they have a defined focus and they can deepen their relationships with those stakeholders. Next, you want to define what research work the researchers do and what research the designers or the product folk can do. Sorry, I wasn't expecting that. We're not expecting non-researchers to become researchers overnight. It's a specialist skill, after all.

[00:09:54] Ensure you term this as a partnership between UX research and design or product. Using this terminology sets the expectation that designers and product managers seek guidance from UX research on executing any tactical work, and it's therefore of a high quality, and it reduces the risk of findings being invalid.

[00:10:12] If we think about the Double Diamond as it pertains to research, there are two broad categories here: research to build the right thing, and research to build the thing right. The model I established at my past organizations mirrored this. I came into the practice at a time when designers were beginning their foray into UX research for tactical pieces of work, focusing just within their product. At the same time, continuous discovery was being implemented in the product space.

[00:10:40] Looking at all this research that was going on, the gap that became immediately obvious to me as a dedicated researcher was exploratory research that spanned not only a particular product, but also multiple products, looking at the horizontal experience or journey of a customer rather than just the product vertical. Because designers were conducting research into their own products for the most part, it left a gap for exploratory, future-focused research looking across products and beyond the SEEK ecosystem.

[00:11:11] In addition, specialist UX researchers are typically better equipped to handle the sorts of questions exploratory research asks, as well as to tackle the tricky methods that you might want to utilize at this stage, and they also have more time to do it, because they're not trying to do design work or product work on top of that. Ensuring there's a partnership between UX research and product or design in executing the tactical stuff also means your researchers keep their research skills sharp.

[00:11:36] Summarizing that: you need to give your researchers a place to exist and a purpose if there are other people in the organization conducting research too. This enables your researchers to do their best work and enables you to retain them. It also benefits the business by providing it with a wide range of data from which it can build great products.

Education: setting expectations and training

[00:11:56] We've covered when to recognize the need to democratize and how to structure your team for success. Let's look at how we build the right infrastructure to support that. Infrastructure, in my view, comprises education, processes and tools. I'm going to talk in detail about education and processes.

[00:12:14] Education comprises setting expectations and running some formalized training sessions. First, you need to set the expectation within the broader design and product function around what the UX research practice will do. Outline the structure of the UX research team and what types of research they lead, compared with what types product and design folk can conduct, all in partnership with their embedded researcher, of course. Make sure to include a point around the researchers being available for coaching and support. That's essential to making democratized research work. Do this as part of an existing ritual or meeting, so people have to listen to you, and follow it up with an email or a blog post people can refer back to so they don't forget.

[00:12:54] Once you've set those expectations, you need to conduct some formalized training, just a little bit to start people off. This ensures that everyone who does research has the same base-level understanding of what good should look like. You can start by assessing people's understanding of UX research techniques, and then use that as a baseline to run some standardized training and some targeted training for specific groups. For example, at SEEK I ran some broad training for product managers around research fundamentals, and some basic note-taking training for one particular squad who wanted to upskill in that area.

[00:13:27] If you're really pressed for time, train people up on facilitating user interviews and conducting basic concept testing. In my view this is typically what designers and product managers end up doing a lot of. For facilitating user interviews, you can create a basic exploratory interview template for product and design to use. They can then copy that and adapt the questions to their context.

[00:13:49] Finally, encourage a culture of peer review and sharing unfinished work often. Ensure the researchers are setting the example by doing this as well, so designers feel safe to do so too. That peer review culture again ensures research quality and reduces the risk of research being invalid. Set a guideline that no research happens without it being peer reviewed. I've got here some examples of some education I did at my past organizations. This is explaining types of research at SEEK, and this one's from training people how to ask good interview questions.

Processes: how-to guides and templates

[00:14:25] With education out of the way, let's talk about processes. Once you've set expectations and run some basic training to get things up and running, you need to think about enabling your designers and PMs to self-serve. This takes the pressure off your researchers over the long run. They can spend less time supporting on the basics and more time supporting people to answer questions that are less clear-cut. In addition, enabling your product teams to self-serve uplifts their capability even further.

[00:14:52] First, we need to create some how-to guides. This is a great activity for that weird time between Christmas and New Year's, when we're not really sure what time means, and it's also great for Friday afternoons. At SEEK I created some basic how-tos for note-taking, usability testing, writing an interview guide, choosing the right research method, and the process for storing and sharing research, as well as ethics and privacy.

[00:15:16] Sharing and storing research is pretty important. One of the risks around democratization is that research just ends up anywhere and you can't find it again, and that ultimately costs you time and money. Think about a low-barrier way to share and store research so you can use it again. It needs to be low barrier so the people in your org who aren't researchers actually follow the process. For example, at SEEK and Coles I created a research report template in Confluence that acted as the source of truth for the research. The report template only took a few minutes to fill out, which meant that people actually did it.

[00:15:50] Another important point is ensuring there's some ethics in place around the collection and usage of research data. This will look different depending on what country you're in, so I suggest talking to your legal team and asking them how to go about storing participants' personal information. Once you've done that, write it up somewhere. Once you've got some how-to guides sorted, tell people about them and keep reminding them that they're there, so you're creating less work for yourself.

[00:16:16] Next, you need to create some templates. Your designers and product folk will likely spend the bulk of their time conducting user interviews and concept tests, like I've said, so start off with those. You can also have some basic survey templates too, because surveys are tricky things to get right. Include in each template some basic types of questions that people can then adapt for their context.

[00:16:36] Now, if you don't have a UX researcher handy, that's okay. There are plenty of resources out there... Well, still going. There we go. There are plenty of resources out there to help you build out some of these things, and personally I've used the Nielsen Norman Group website a lot. Regarding tools, I'm not going to go into massive amounts of detail, like I said, but I would encourage you to have some kind of intranet to house your how-to guides, some way to store research so you can find it again, and some kind of survey tool, just to start you off. The rest you can get done on a shoestring.

Embedding research in the right places

[00:17:09] Alrighty, so we've covered the first three steps in our recipe, and we're up to the last step, which is embedding the work in the right places. By this point your researchers, designers and product folks are all happily partnering up and conducting awesome research. Now you need to figure out how to best embed that research so your organization is making customer-led decisions. This gets you the successful roadmaps outlined in the description of this talk.

[00:17:35] This is the hardest part of the job. Doing research is easy; getting it used is a lot harder. Now that lots of different types of research are happening, you need to spend some time working out where they'll have the most impact and ensuring that research gets heard. Here's our Double Diamond again. Now that the designers and product folks are conducting research to build the thing right, it's pretty straightforward where this research ends up: things like shipped designs, shipped products, or journey maps for a particular product. That's easy.

[00:18:04] The type of research that your researchers are now responsible for is less clear. Because it's often exploratory and future focused, it's hard to understand how it impacts the products directly, and sometimes there might even be opportunities to explore new products, stuff your organization doesn't even do yet. Its impact is longer term, and it's therefore less obvious. The artifacts in which this type of research lives include product roadmaps, strategy documentation, or even journey maps relating to experiences across a particular organization rather than just one product.

[00:18:42] Get really friendly with your stakeholders and get them to share these artifacts with you, or better yet, run some co-creation sessions and make them together, so you both have ownership over them. At SEEK I would partner with product teams to embed exploratory research insights into their opportunity solution trees, for example. At Coles I'd work with the group product manager to embed opportunities from research into their forward planning for each quarter. Get really in people's faces and find where those documents live.

Getting in front of decision makers

[00:19:11] However, it's not just artifacts we need to worry about. This research has a long life span, and we need to breathe life into it beyond the artifact to ensure it gets used in the right places. Once you've figured out the artifacts in which your researchers' work lives, you then need to get that work in front of the right people. Instead of influencing what the product team works on in the here and now, this type of research influences the future product or problem spaces for the business to then investigate.

[00:19:40] "Research it and they will come" as a vibe is not going to work. People are busy, and you need to help them draw conclusions. You need to put that research in front of them and really spell out what it means. That doesn't work anymore. Bring those decision makers and stakeholders along the journey with you, and link the insights to the business context. Demonstrate that by incorporating research data, we're increasing the certainty that our strategies and roadmaps are correct.

[00:20:07] Doing this means that those decision-making stakeholders will trust that you know what you're doing. The flow-on effect of this is that you get invited to more and more important meetings where decisions are taking place, or you can set that expectation way back when you're educating your team. Keep being persistent. It's really hard, but don't give up. Keep asking, keep showing your work, keep proving why it's valuable, keep explaining how it fits in, and the invites will come.

[00:20:32] Keep an eye and an ear open for meeting titles like quarterly planning, jobs to be done, journey mapping, vision or strategy. That's likely a place where your research will have value, so get into those meetings. After that, both the research work the product teams do and the research work the researchers do will have a place in your company's product roadmaps and also its shipped products. It's about finding the artifact and getting time with the people who own those artifacts.

[00:21:02] Phew. Thanks for coming along on that journey with me. Wrapping up what we've learned today: we talked about recognizing the need to democratize and looking for that tipping point. We talked about ensuring your researchers have a home and a purpose. We talked about ensuring the right infrastructure is in place, education and processes. And we talked about embedding your great work in the right places for maximum impact. Lastly, start kicking goals. Thanks for listening.