The Secrets And Synthesis Of Successful UX Research

Gianni Clifford at UXDX Europe. Video: https://youtu.be/pp4IMvkgDGs

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction

[00:00:01] My name is Gianni Clifford. I'm going to talk about the secrets and synthesis of successful UX research. I particularly picked a tongue-twister of a title, so I figured if I got to the title slide, I should be coasting for the rest of this whole thing.

[00:00:14] Before we kick on, I just want to introduce myself. I know Nick gave me a little bit of an introduction, but best to hear it from the horse's mouth. My name is Gianni Clifford, and as he said, I'm a product design manager at Zendesk. Zendesk is a customer [?] company. We were founded originally in Denmark about eleven years ago, and our three founders set about to improve the customer experience, particularly at the point of support software. But as we've grown, we've moved into sales, we have our own platform, and we're trying to give a full customer experience end to end, from customer acquisition all the way to owning that customer experience.

[00:00:55] Before I was at Zendesk, I worked with various agencies and design studios, as Nick said. I also founded my own startup called Phyllis [?] in 2015, and then I moved on to Zendesk, so I've been here about two years. The reason why I've done this talk about UX research is that it's going to be getting really into the weeds, so I hope I don't bore anyone. But I find that any time I go to conferences, I'm more engaged with the speakers when they show what they actually do and how they work, rather than just showing "this is what we do," the very end of the journey. I want to show what it's like as part of the journey. I want to pull back the curtain and give you guys an idea of how we do UX research.

[00:01:37] In the talk, this is what I'm going to cover: how we do discovery research. I particularly call it discovery research because there are lots of different types of research, whether it's validation or all different types. Hypothetically, and the slides aren't great, we're looking at discovery research if we're building a new product, a new feature, or expanding a product into a new area. That's what discovery research is. We want to know who is key to this research, who we need to collaborate with to make this UX research successful, and finally, what makes it successful.

The research plan

[00:02:18] First of all, we start off with a research plan, and this is essentially the equivalent of a design brief. Normally when a big discovery project sets off, it may come as a request from a VP of product design, or a GM of one of our products, or maybe both at the same time. So it's very important that we collect all these thoughts in this research plan: know who's involved in the project, what their responsibility is, and what's going to be delivered.

[00:02:46] The things that we would put into a research plan: first and foremost, who owns what, who's going to be involved in this project. Quite frequently it's not always a UX researcher at Zendesk who's involved. It could just be a product designer with their PM partner as the key people, and then they would consult with a UX researcher. Or there could be a UX researcher involved in the project, and it would be a trifecta between those three people. So it's important to call out who's involved and what their exact role is.

[00:03:16] What problems are we trying to understand? Why are we doing this? What are we looking to figure out? It's really important to call this out from the start, to make sure everyone's aligned on what those problems that we're trying to explore are. What decisions are we trying to inform? Are we looking to add a new feature? Are we looking to expand a product? What exactly are we hoping to inform at the end of this?

[00:03:39] What assumptions and hypotheses do we already have? No matter what the subject that you're researching is, people will always have assumptions, and it's more than likely that the people involved in this project, whether they're stakeholders or the people running the research, are going to have differences in their assumptions. So it's good to call these out, and through the process of this project you want to be proving them or disproving them.

[00:04:00] What would our output be? At the end of this project, what are we going to hand over and say, "Okay, this is the discovery research done"? It's very important to call this out, because sometimes people get a misconception of what the end result of the research will be. Stakeholder interviews: as I said, there could be various stakeholders. It's really important that you interview them and get much deeper, because the first request could be just a sentence saying, "I think we should explore X." But until you get time with this person, interview them, ask some questions, really understand and get a little bit deeper with them, you may be off the page with them. And if there are numerous stakeholders, they could be totally misaligned. Next, obviously: who are the stakeholders, the people we're going to interview? And finally, what does success look like? At the end of this project, what are we going to say: "Okay, we've done it, we've done a good job, we've informed X, Y and Z"?

[00:04:57] Once we have this research plan, I think a research plan should be a bit of a living document. You don't just do it at the start of phase one and go, "Done, it's finished with." This is something that can evolve and meander a little bit as the project goes on. But once it's used as the source of truth, and all the key members and all the stakeholders have access to it, whether it's a living Google Doc or something, and they can all check back, then it's okay for it to meander. Because if you're doing discovery research for a quarter, for three months, you're going to find out things halfway through that are going to shape the results of this. So you need to keep that research plan as a source of truth.

[00:05:33] Another source of truth, a living document that we use, is what we would call a mothership document. As you can see on the right-hand side, there's a Gantt chart. Some people hate them; I'm a huge fan of them. I can see Graham [?] in the audience; he's on my team, and he loves them. It's really important for a large project like this when you have different stakeholders, because you need to know who's gone on PTO when. If you need to travel to customers' offices that could be in a different country, you need to line up everyone's calendar, and this is just a great, easy tool to do so. Also in our mothership document we would capture stuff like a reading list, to-do lists and loads of different things. Again, you start this at the start of the project, but it will live throughout.

Desk research

[00:06:19] Next, we kick on with some desk research. This is the first time you can actually roll up your sleeves and get a bit stuck in. What you want to do is exhaust what's already out there. First of all, you want to be interviewing the stakeholders, as I mentioned. You want to talk to sales and success. Sales and success people own the relationship with the customers a lot of the time, so check if they have a VoC document, which is capturing the voice of the customer, and go through that and see if there's anything in the space you're exploring. Past research: has somebody else explored this on your team, or on your global team? Data: depending on what you're looking to figure out, there could be some product data that will help you.

[00:06:59] And then the community. We're very lucky at Zendesk to have community boards. We just launched a product last week called Gather, which is a community page. So if you're lucky enough that your company has a community board where active customers are talking, I would definitely recommend checking that for anything that may kick-start your research. These are essentially all internal things that you can start with in desk research.

[00:07:20] There are also heaps of external things. Articles, books, blogs and videos: what's out there? Is there anything that I need to be aware of? Reports: is anybody not in my company doing some research on this topic? Demographic information: who am I trying to solve this for? What do I need to know about them, and can I capture that information back in my research plan? And competitor analysis: if I'm looking to do a new feature, or a new product, does it exist already? How did they tackle this problem? It's good to be aware, because it's more than likely your customers are aware of this, so you should be too when you go to talk to them.

Who to talk to and how to recruit them

[00:08:03] Now, who do we talk to? This is where I get a bit woolly [?]. Who do we talk to? It really depends on what you're trying to solve. I can't tell you exactly who you should talk to; it really depends on what you're looking at. At Zendesk, a customer we may look for, I'll give you an example: they could be using, let's say, three of our products, Zendesk Support, Talk and Chat. They could be based in North America. They could need to have 50-plus agents, and they could need to be solving a thousand customer support tickets a week. That's the type of customer we may be looking for. But it really depends what you're exploring, and it's up to you, with your key partners and your stakeholders, to decide who the customers are that you want to talk to.

[00:08:54] What I can say is that at a very, very minimum you should be talking to six. That is by far the least you should be talking to. Any lower than that, you're going to have a higher chance of totally skewing your data. A more comfortable number as a minimum would be twelve to fifteen. Then you're in a nice place.

[00:09:12] To recruit the customers, this is how I wish it would look: do a bit of investigation, arrange it and confirm. This is the happy place, and it doesn't exist currently. We're working really hard on this area, because it's a total blocker for us. We're trying to build an internal tool that will make this a little bit easier, and we have some research coordinators who are absolutely brilliant, to help us facilitate this recruitment step. But it still doesn't look like this, and I want to show you what it does look like. I have a couple of examples here, because for us this is a real blocker, for various reasons: sales and success, GDPR and a number of other things.

[00:09:53] One way we could recruit customers is, first and foremost, check our community pages, like I mentioned. Then we listen and read them, absorb that information, maybe start conversations on the community board with the customers who are talking in the area you're looking to do UX research in. We do some investigation in our internal systems: how many seats do these people have, what products are they using? To understand: is this customer the right fit for our project? Then talk to sales and success again. They own the relationship with the customer, so see if they can give you an introduction, see if they're happy with you talking to their customer, and make sure it's a good time. Throughout the customer life cycle there are times that are better than others to talk to a customer, and sales and success can help you inform that. Then you want to email and align some calendars. This step alone could take a couple of weeks, especially if the customers you're looking to talk to are in a different time zone, or there are holidays or weekends that pop up. And then finally, arrange and confirm. So it's very laborious.

[00:10:52] Another way we could do this, which I'll fly through, is make pals with our data and engineering partners [?] and ask them to write a query first. That query may correlate the things that we wanted before: they're using X products, they have X many agents, they're solving this many tickets. We could run that query, get a list of our customers, check our internal systems, talk to sales and success, email, line up, and then eventually arrange and confirm. So you can see, these are just two examples. There are obviously heaps of other ways we can go about recruiting, but there's a repetition in these final steps: we need to talk to sales and success, and we need to check our internal systems.

On-site: admin interviews and shadowing

[00:11:31] Once you have all that lined up, I'm going to talk about what we like to do on-site. This will vary again depending on what your research is, but I'm going to show you one type of what we do. First of all, when we get on-site we do admin interviews. Admins are normally the more senior people in our customer support companies. They normally manage the agents; they could be executives or senior management. We talk to them first, not just because we want to hear what they have to say, though obviously their opinion is vital to us, but also so that later on in the on-site visit they'll give us access to their agents without the admins being there. That access to agents without the admins being there is really important, because when the agents are sitting with their managers, they may not always be at their most honest or most comfortable to talk to an external person.

[00:12:22] So first of all we'll interview these admins. There'll be at least two people from Zendesk there: one taking notes at all times, one asking the questions, and they will switch, just to make sure there's always someone engaged with the customer. We'll record these sessions if possible. Sometimes that's not okay, but we aim to do that. Roughly 45 minutes is a good time for this. Sometimes it can run over to an hour, depending on whether you have two or three admins, if they like to talk or they don't, if they have feature requests, et cetera.

[00:12:48] Next we do shadowing. We shadow two agents. Agents in the Zendesk world are the people who use the software, and we aim to shadow two people. We just sit beside them. We're very light touch; we ask a couple of questions, but basically we're just watching the screen, watching what they're doing. Maybe we'll ask them a question like, "I see you did X. Can you tell me why you did that? What was the reason behind your decision there?" just to really understand the root of what they are doing with the product.

[00:13:16] We'd also aim to do two of these separately. The first person may be a person who's newer in the company, maybe someone who's been there six months. They're fresh from the training and haven't picked up any unusual habits. Then the second person has maybe been there a little bit longer. They may be a little bit more fluent in Zendesk, and they may be quicker doing things. So you get a real good balance on every different customer that we meet.

On-site: rose, thorn, bud and the ladder of whys and hows

[00:13:40] And finally, we do a design thinking session, and I'm going to show you what we do here. We run two different ones: rose, thorn, bud, and bud blossoming. Let me get this up. First of all, we run something called rose, thorn, bud, and I'm sure a lot of people have done this before. It's a retrospective thing; it can also be called stop, start, continue, and various different names, but I'll explain how it works. We have in the room with us the two or three admins we've already interviewed and the two agents that we've already shadowed, all in the room together.

[00:14:20] We dedicate three minutes for everyone in the room to capture at least two roses on Post-its. Roses are things that they like in the product, things in the product that make their job that little bit easier. We repeat this with thorns, which, as you may guess, are their annoyances or blockages in the product. And finally we give another three minutes at the end, where we ask them to give us buds. Buds are areas in the product that have potential, or ideas that they have to improve the product. It's important we do the buds after the thorns, because essentially the customers are looking at the pain points and then they're coming up with solutions. If we came in off the bat and said, "Give us some solutions for our product," we'd get radio silence. But this is a clever way to get them really creative and thinking, and they normally really enjoy this. What we then do is read out all the roses, thorns and buds and get everyone to share that in the room. This should take about half an hour or so.

[00:15:17] Next step: we keep these, and we'll loop back round to what we're going to do with them later, but we focus on the buds. What we're doing here is we get everyone in the room to vote for their favorite bud: which is the best idea that we've had, which is the best solution that somebody mentioned. Let's say the bud might be "I would like an away status." So let's say it's a status for when they go away from the desk and don't have to select offline. What we'd ask them is: why do you want this status?

[00:15:49] We make it very clear to them that there are no silly answers. The questions that we may ask them may seem like we're trying to trick them, but we're really not. We just want them to give the first thing that comes into their head, over and over again. Once they say, "I want an away status because I don't like having to select offline when I go to a meeting; it changes my hours [?] as an agent," we'll say, "Well, why is that the case? Why do you not want that to happen?" And then we'll ask them again: why, why, why. We do these five whys, which many people are familiar with, I'm sure, to get into the root, or the job, that they're trying to do.

[00:16:23] As we're doing this, we simultaneously use the hows, moving up the ladder: how would you like that to work? How would you like that to actually work? It gives us this flexibility to go why or how, and we can do the hows off the whys and the whys off the hows. We continue this process and get everyone in the room really brainstorming, and we can start coming up with ideas [?]. We end up with something like this, with loads of ideas. Again, if we came in off the bat and said, "Give us 16 ideas for a product," we could get radio silence. But if you do this kind of controlled brainstorm, or encouraged brainstorm, where you bring them on the journey, they really engage with it, and they give you lots of ideas and lots of great content.

Readout sessions

[00:17:05] After doing the on-sites, you're going to be pretty wrecked. You've just been to 15 different offices and talked to a heap of different people. You're going to have loads and loads of information, and it's going to be a mess. You're going to be juggling like this. So what we want to do now is analyze and synthesize that information into something a little bit more digestible. We want to get it like this: something that people can just pick up and read, and something that's going to be valuable for our whole organization.

[00:17:36] This is where we do the readout sessions, and again I'm going to talk you all through how exactly we do this. The first thing we do is recruit cross-functional colleagues. People suggest the larger the number the better; personally, I prefer groups of around three. In the readout sessions we'll have two people who are going to facilitate the sessions, and I think the number three is good for the people who are going to be involved, so recruiting groups of three is really good. The most important thing with those groups of three is to get cross-functional people. You might have a group with an advocate, an engineer and a salesperson, or a PM, a product designer and a researcher. Make sure there's a good mix in every one of the groups, and recruit a different group for every customer that you've been on-site with. For some of the people who are really engaged in being involved, it's totally fine if they're in numerous groups, but just make sure you don't have three engineers or three product designers, because that's going to skew your data.

[00:18:49] Next, what you want to do is build empathy into your notes. When you're doing the readout sessions, it's really important that the people in these sessions understand what it's like to be on-site. I was visiting a customer of ours called the black talks [?] in Los Angeles. When myself and my project partner Jenny turned up, they had two little baskets for us, with our names colored in, and loads of goodies in them. It was just such a nice welcome, and for the whole four or five hours that we were there on-site with them, they were just the nicest people ever. So when we were telling the story back of the information that we captured, it was really important that the people in the room were aware of this. What were their offices like? What were the people like? What was the environment the agents work in? We normally capture photographs of the agents' desks, and we'd show those, to make sure people really understand what it's like and get in the headspace of that customer.

[00:19:41] We would gather the groups together. Don't underestimate this step, because you have to get everyone's calendars aligned and all sorts. You're looking to do the sessions for about an hour and a half per customer. You dish out the roles. Of the two people I mentioned who are facilitating the session, one will read out the notes, and the other will be facilitating: making sure it's running along at the right speed, making sure the notes that they capture are digestible, and making sure that they're in the first person, so when removed from the context of that customer they still make sense. "I want an away status" is very digestible, even when it's removed totally from all the other facts about that customer. Then the people you've recruited: all three of them will be capturing the data, saying, "I think that's a data point," but one of them will be responsible for documenting that in a Google Sheet. Then finally we'll read out the notes in the hour-and-a-half session and capture those lines of insight.

[00:20:39] When you do this, you'll be left with something like this. This is a document with about eleven hundred lines of data that we captured from talking to twelve different customers on a research project. It's super, super valuable, but we're not finished with it yet. We want to go one step deeper with it. It's still lines of insight; it's still not digestible enough.

Affinity mapping

[00:21:05] This is where we do an affinity mapping exercise. I'm sure people have done affinity mapping and are very aware of it. There are two key ways to do this. You can either do it in the physical space, by turning all those lines of insight into Post-its, printing them out on a whole wall and getting your cross-functional colleagues in to help you digest it. Or you could do it in a software program like Mural or Miro. My preference is to do it in the software. There are pros and cons for each version, but with the software version it's just so easy to search through all the lines of insight. It's so easy to export from your sheets and import back into your sheets. It saves you heaps of time. It does have the downside that you can't look across a whole wall, but the software tools are also great for collaboration, so you should have your cross-functional colleagues in there helping you break down this information.

[00:21:59] So what we have is a thousand or so pieces of insight as Post-its. The first thing we want to do is put some sort of filtering on them. What's the difference between these? Who said what? We'll flag the ones that were said by agents and make them yellow. We flag the ones made by admins and make them pink. And the different ideas that came up from Zendesk employees as part of the readout session, let's make those green. Now we have at least one layer of filtering on top of all these lines of insight, but it's still not enough. We need to break them down into smaller groups.

[00:22:35] At Zendesk we're really focused on the experience. The agent experience: they're the people, as I talked about, who use the product. The admins, the people who set it up. The developer experience, the business analyst's experience, the team lead experience. So what we want to do as we're breaking these down, at the highest level, is, no matter who said it, put it into an experience bucket. All these Post-its are now about the agent experience: what does the agent do? The other Post-its we're going to sort through in the same way.

[00:23:11] Once we break it down into this experience, we look for the different themes that are in it. Let's say we break it down to another level: one of the themes might be workflow. What does the agent do at work? What are the different important factors of that? Let's break it down again, into category. Let's say we have a heap of different data points from agents at work on calls. Can we break that down again? Yes, we'll break it down into a subcategory. What you're aiming to do is break it down to a level where all the Post-its in each of these buckets are concise and directly related to each other. It's a time-consuming thing, but the value from it is enormous.

[00:23:50] You should be left with something like this. An example of what one of the Post-its may say: the number at the top is a unique number, so you can search through your Miro or Mural boards. "Acne Inc. [?] would like an agent status strictly for escalations." As you go through and do your affinity mapping exercise, it'll make it much easier for you to highlight the buckets that have lots and lots of consistent themes. This lets you know this is an area of opportunity: there could be an opportunity to do a new feature in that space. Sometimes the data will push you somewhere you didn't think you were going to go initially.

Sharing the value: the insight document and findings document

[00:24:30] Next, what you do with this, as you have the breakdown of the data, is pop it back into your document, and all of a sudden the document of eleven hundred lines of insight is completely searchable by theme, by experience, by category, by subcategory. So if you're a product designer, let's say working in our Melbourne office, focused on our developer experience, you don't need to read through eleven hundred lines of insight that may not be relevant to what you're working on. You just apply some filtering, and then there are 20 that are exactly about what you work on. The value from this goes far beyond the project that you initially set out to do. You're still getting the value for the project you're doing, but you're sharing that value widely with the rest of your org. So that's one artifact that you want to produce as part of the project.

[00:25:21] Another artifact that you want to produce is a findings document. In a way, the findings document should be a bit of a TL;DR, a bit of an executive summary. I like to have at the start a very short paragraph, a couple of sentences, on why you are doing this research, and then two paragraphs max with the findings, and a couple of bullet points. You need to expect that this document is going to be digested by numerous people in your company, and maybe people at executive level who just want to scan through. They don't want a 20-page research findings doc. You do have all that information, but you want to give them that executive summary. You want to use maybe a Google Doc or something similar, so as they're reading through your findings document and you have pull quotes from a customer, they can simply click and they're deep-linked into the affinity board, or into the raw notes from that customer visit.

[00:26:19] When you do this, you'll start to see the value beyond the data. This is going to be hugely valuable for product designers. It's going to inform them what they're going to do, what decisions they're going to be making as they're designing this new product feature, or anything else. It's going to help PMs decide on the roadmap, and it's going to help back up some of the decisions that they have, or inform some of the assumptions they previously had. It's going to help engineers. Engineers don't like to be left in the dark. They want to know why they're building something and what they're building next. Sharing this kind of data with them, after they helped to produce it, is hugely valuable.

[00:26:57] Same with sales and success. As I mentioned earlier, they own the customer relationship a lot of the time, so make sure we're sharing that information with them. Again, they helped in the readout sessions, so they should get some value from it as well. And finally, leadership. They're the ones doing the long-range planning. They're the ones who make some of the biggest decisions, and they need this kind of data from our customers to make decisions. So in short, everyone gets some value. Everyone should take this document and the previous data document and get value from it.

The secret: collaboration

[00:27:31] So it comes to the secret. I hope, as I've talked through this, there's a clear recurring theme over and over and over again, and hopefully it's something that you all exercise all the time. For me, the secret of successful UX research is collaboration. It's getting the involvement of other product designers, engineers, PMs, researchers, sales, success, advocates: all of your company involved, helping to digest it in a different way. It removes the bias that you may have. It gives different insights on top of what the customer said. It extracts what the customer said in a different way, and it helps you understand it in a fair way. So love your colleagues, in an HR-appropriate way. Thanks very much.