Beyond Interviews: UX Research Methods 4 Everyone

Nadine Piecha at UXDX EMEA. Video: https://youtu.be/7_qsJLpibAk

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

UX research for everyone, not just researchers

[00:00:08] Thank you. Good morning everyone, good morning everyone in the room and good morning everyone online, and thanks for the nice introduction, so I don't have to say that much about myself anymore. Everything was said already, so let's jump right into the topic. What is my objective today? I want to talk about UX research for everyone, not just for researchers. We'll look into different methodologies and how to involve people that are not just the researchers but maybe designers, product people, everyone who builds products for end users. We'll look at different methodologies and we'll look at how to overcome obstacles like no time, no budget, no people. We know all these different obstacles. With a bit of your input, with a bit of your help, we'll look at the most relevant ones.

[00:00:53] Just a quick reminder. Probably everyone online, everyone in the room knows what user research is. I just wanted to point out once more it's the systematic investigation of existing or potential users to understand their needs, pains and context. I've underlined the words that I personally care about: systematic investigation. It's not just talking to users. That's also good, you always learn something, but it's systematic. You always have a purpose, you always have a goal. Product and tech teams do UX research in order to understand for whom to build their products, what to build and how to build it. That's what we're going to talk about, and I'll be focusing a bit more on the traditional user research like talking to users, involving users. I know data analysis is also very important and it always goes in conjunction. Nevertheless, I'll focus more on the qualitative part of research.

[00:01:46] First thing, because I see you a little bit in the distance and I always like to feel a bit the vibe of the room. Sorry for the people online, you can do the stretching exercise as well, but I want to see hands up in the room. Hands up everyone who is a researcher. Do we have researchers in the room? I think I saw two, three hands, so not that many. I hope there's some inspiration for you as well: how to involve others, what other methodologies to use or why to use different methodologies. Do we have data analysts in the room? No one. That's a shame, I would like to have you more involved. I hope online we have some data analysts. Do we have designers, product managers in the room? Okay, now I can see the hands going up. Also welcome to the designers, product managers online. For you, I hope, I have some input and some ideas on how to get more involved with research. And then obviously welcome everyone else as well. If you work in sales and customer support, wherever you are, we need you, because research is a team sport.

[00:02:50] The first question is a poll that I would like you to take part in online: who typically does user research in your company? Just go back into the polling tool, and if we could see the results here as well, that would be fantastic. Who typically does user research in your company? Okay, we can see the results coming in. That's amazing. We can see a few lucky ones who have researchers in their companies, in their teams. We can see a lot of designers doing research. Surprisingly few PMs and POs, sorry about the typo there, PMs and POs doing research. I would dream of more. I would dream of more writers being involved as well. Copy is a super important part of the product, of the end result of your design, of your product, so if writers can be doing research as well. Engineers as well, if you can get engineers involved, better.

Who does research: the team sport

[00:03:53] Great, if we could jump back to the slides. This is probably a visual that everyone has seen before. Everyone out there is also talking about it: it's not just the UX researchers, like the top circle, that are doing research, but it's also the designers, it's also the PMs that are building products that we ultimately want users to love. But in my mind the picture should actually look like this. It shouldn't be just the PM and the designer and the researcher. We also have people from data. At FREENOW I'm very lucky, data engineers are very involved also in research. They attend interviews, they sometimes even lead interviews, they take notes, they follow up then with data analysis on what we've heard in interviews. So really great to work very closely with people from data. Engineers as well, if they can join, if they can see firsthand what's happening in interviews, what are users really telling us, what are they really thinking. Super important.

[00:04:51] Designers obviously involved, PMs, marketing, and I also put senior leadership somewhere there in the middle. If you can get the senior leadership into the room, even better. And then there are a lot more roles that are not even on the picture. It's been very helpful to involve sales teams, especially if you work in B2B products. I have salespeople quite closely involved sometimes. Customer support teams, they have so much knowledge, they know so much about the end users and their problems, so if you can get them involved as well. It's what I said before: research is a team sport and we should be all in it together.

[00:05:26] Okay, next thing I want to do is another poll that is about the methods that are used in your case. Which are the methods that you have used most, or that were used in the past few months in your company? If you could take the poll again and then we'll see the results here. Nice, I can see the results coming in and it looks pretty much what I expected. User interviews, the number one methodology that is used by everyone, and that's usually our go-to method. That's also why I called the talk today Beyond User Interviews, because I also have the feeling that even as researchers we always default to let's just talk to people, let's just do an interview. But there are other methodologies that we can also use, and we'll talk about a few.

Beyond interviews: surveys, focus groups, co-creation and field visits

[00:06:24] Online surveys. I've also expected online surveys coming up quite high. There's a lot of different ways of doing surveys. We all know probably the NPS, the net promoter score, is one of the most famous survey methodologies. There are on-site surveys, there are surveys you send via emails, there are surveys you can send to your own database, or as blind format surveys via a panel or via an external provider. A lot you can do with surveys. Just be careful, surveys are also sometimes the false friends. There's this saying, I don't know if I'm allowed to say that here, about garbage in, garbage out. If you ask the wrong questions or if you ask questions the wrong way, you'll get results, you interpret them because you think you've done it right, and if you do it wrong you'll never know. If you do an interview wrong, the user will tell you: I don't know what you mean, are you referring to this or that? In a survey they can't, so they'll give you answers. So you need to be sure you got the questions right. That's one of the methodologies where I'd recommend to really include a researcher. Interviews are much easier to also go with the flow and learn and see and get some feedback from users. Surveys, you have to get them right to get the right responses.

[00:07:36] Focus groups, also a very nice methodology. I've personally seen it a bit more on marketing topics than on UX topics. When I do them for UX research I used to call them workshops, to distinguish a little bit. It's a little bit less people discussing concepts or discussing designs, but it's more about building something together, working on some tasks, having specific questions, and you see the interaction, you see how users explain each other things, you can learn from the conversations that are going on in the middle. So that's quite nice.

[00:08:11] Co-creation, glad to see that's also coming up here. We've done some in FREENOW and they were quite insightful. We've done some with drivers on how to build the designs, the navigation, also elements that are not while the driver's riding but even on non-riding moments. So quite an interesting methodology as well. And ethnography, field visits. I can only encourage everyone to do that. I'm doing that right here in Dublin. Dublin is actually one of the biggest, most important markets for FREENOW, so I'll invite you all also to test FREENOW, test the product. If you learn something, let me know afterwards. But it's really different. I've seen, for example, we took a taxi yesterday and I saw there was a camera recording the passengers in the back, and I didn't know because my interviews never included that. I spoke to so many drivers from Ireland and I never asked what else is in your car, because I always have another concept. So seeing the context, seeing how the users actually use the product in real life, very, very valuable.

[00:09:18] And then some other methods. Hit me up later if you have some other cool methods that you can share with me.

Why a large toolbox: better insights, customer centricity and team growth

[00:09:24] If we go back to the slides, just a quick summary of why I really insist on using a large toolbox in research in your companies. One is better insights. I already mentioned, if you can combine what you see in context, what you learn in context, with interviews that you've done before, if you can combine qualitative insights with data, with maybe quantifying via surveys, you just learn much more and you have a better, more complete picture.

[00:09:50] Customer centricity. I've also observed that in the companies, whenever you switch to a new method, you activate more interest. You get more people joining, you get more people, even senior leadership, sitting in the room. The first time we did some focus groups in FREENOW, everyone was there because it was so cool behind this one-way mirror and you could observe and you could see that. It just gets more interest when you try something new, compared to the 200th interview that you want everyone to dial into. So to foster customer centricity and get more interest, that really helps. And then also there's a nice side effect that you encourage your researchers, your team, also to grow. It's important that everyone also every now and then goes out of their comfort zone, tries something new. So new methods also really help to get everyone or keep everyone engaged.

The obstacles: polling the room

[00:10:45] All righty, and last but probably most important topic, the obstacles that you face when looking at research. I have a list of several obstacles. I'll also have an open-ended "other" question, happy to hear from you what other obstacles you might face, but there are a few you can select from in the list already. So there's another poll that I would like everyone online and in the room to take, and then we can see what is happening and we'll talk about the most voted ones. Vote for the ones that are most relevant for you and then I'll give you some of my ideas on how to address them.

[00:11:36] Okay, instead of "enterprise" it's supposed to say "expertise", sorry about that. And it's a bit hard to read that one, so we have exactly one third voting for no research expertise, not enough researchers or budget, and speed. And nothing is moving. Is that poll working correctly? Okay, it looks like it's a little bit stuck. Maybe we can do a bit of hand voting again. No research expertise, who has that problem? No researchers in the team? Ah, there we go. Okay. Resources, not enough researchers or budget, that definitely seems to be the problem number one for people online and for people in the room. What is the problem number two? Buy-in, leadership doesn't listen or act. That's a big one, I can relate to that. Speed, research slows us down. That's unfortunately something I still hear quite a lot. Let's see how we can solve that.

Lack of resources: prioritize, start early, get trained, outsource

[00:12:53] All righty, so let's pick the first topic. Sorry, I need to see the results again. What was the first topic? The not enough budget. Can we go to the slide with not enough budget? There's the link from there. Lack of resources. All righty, those are my tips, but I'm happy to have more conversations and also learn from all of you afterwards. Maybe we can exchange different ideas.

[00:13:39] For the lack of resources, the number one advice is prioritize rigorously. It's super important that you have very clear where to spend your resources. If you don't have unlimited resources, which is probably the case for most of us, focus on what will have most impact. That can be small or big projects, but where you actually see that people are going to act on it. And then also focus on answering questions early in the process, because those learnings that you get very early in the process, you'll carry them through the whole rest of your project. If you learn very well who is the user, what do they need, how do they use the product in context, if you learn that very well at the very beginning when you're in early discovery, you know that when you start designing, you know that when you assess different ways of solving the problem. So you might need less research later on, versus if you only test your solutions, if you go right away into usability testing, you don't know really very well what other solutions might have been there. So I would always prioritize early research rather than the late projects.

[00:14:43] Get trained. There are lots of trainings out there. I've also trained different companies. It's very easy to get some basic trainings for PMs, for designers, for people in customer support on how to also help with research. And then outsource what is possible. Outsourcing, the same, like big projects or surveys, that is something that you typically can do. You can outsource and you can consume the results and use the results in house. What I wouldn't probably recommend to outsource that much is everything that's very agile and that's very like, we need feedback for the designs, adjust the designs, test again. All this very interactive, immediate research is probably less easy to outsource.

Speed: research doesn't have to be slow

[00:15:30] Let's go to the speed slide maybe as well. That sentence that I heard, research is too slow, that's actually not true. Research doesn't have to be slow at all. In our case, for example in FREENOW, it's as easy as going to the street and taking six cabs, and then every time I go with the driver for 15 minutes, 20 minutes, I can ask questions. So that can be done in half a day and we have some insights. There are methodologies that are quick and dirty but very easy to get the most important insights you need. Same advice here: start early, start strategic research, start learning about the user, their context, their behaviors early in the process, because then you have that knowledge throughout the rest of the process.

[00:16:13] The pragmatic approach. Keep everyone involved, discuss compromises in research approaches. Not everything has to be perfect. I'm someone who really doesn't like to work that much by the book, so it's like, yeah, the book would say maybe I need six focus groups and everything needs to be perfectly balanced. If I don't have time for that, we'll do something else, but we'll learn. We'll have to learn, because otherwise you'll build products that no one will use. And the guerrilla research I just mentioned, for us it's easy, like taking the taxis. I worked before in the travel industry. Almost everyone travels via a plane one way or another, so it's as easy as going to the street, asking someone, do you have five minutes for me, I'll give you a Snickers or whatever, Mars or Bounty. It's really easy. Just get started and you'll see how easily you can learn just on the street, talking to people on the street.

Buy-in from leadership: relationships, format and persistence

[00:17:11] What was the third topic that came up very high? I think, ah, the leadership one. Buy-in from leadership. To get buy-in from leadership, all the way up, the most important is building relationships, from my point of view, from my experience. So really meet them, find occasions where you can talk to them, try to understand, and then also adjust your format to them. Once you know what is their language, is it data, is it spreadsheets, is it videos, is it examples, anecdotes, how are they thinking, where are their minds, then adjust your format. It's not always 20 slides with results, but adjust your format to them.

[00:17:53] And persistence. Just never give up. If five times they didn't act, maybe the sixth time they will listen and they will act. We have one thing, for example, in FREENOW that we call User Voices. That's a Slack channel where we post one quote every day, only one, so it's not overwhelming, it doesn't take up a lot of time from everyone. But one quote per day from a user interview, a short video, a quote from a survey, or something interesting that catches people's attention. And we do see the CEO going in there, commenting on the quotes, putting their emojis, reading them. So being persistent, being there every day, talking about the user every day, in my case that really helps.

[00:18:35] I think we are about to finish with the main talk. If you took something away, I would hope that it's this. Everyone can learn user research. There's more than just interviews and surveys, which is what most companies probably do most of the time when they need to involve users. And user research is a team sport. We are all in it together, we all want to build products for users that users love, so we all need to speak to users, we all need to learn from and about users. And I guess it's time for Q&A.

Q&A

[00:19:11] Nadine: Okay, let me try to see the questions here. "My company doesn't have researchers, but everyone thinks they are doing research, while in reality they're only chatting with customers randomly and without a plan."

[00:19:25] Host: Hello. How would you go about changing that? I'm here to make it less awkward for you.

[00:19:30] Nadine: Thank you, that felt a little bit awkward.

[00:19:32] Host: All right, do you want to go with the first question? If you don't have research in your company, how do you deal with that, how do you change that?

[00:19:40] Nadine: Yeah, if you don't have anyone that is in charge of research, maybe you need to find some kind of volunteer, someone who loves it. There's always someone. There's a designer who knows more about it, there's a PM who knows more about it, or maybe you can find two or three people that can team up and be the ambassadors and teach a little bit at least about it. That's also what I said in the very beginning: research is not just randomly talking to users but somehow systematic, and the most important part to make it systematic, a systematic investigation of users, is to have a plan. What do you want to learn? If you write that down and you really know what are the questions that we want to learn about, that we internally need to answer, you automatically have a bit more of a script, of a guide, of a plan on how to answer questions. But someone needs to own that and guide that in the company. So whoever wrote the question, I guess it's you, you self-volunteered for that now.

[00:20:38] Host: So you don't have researchers, but you're trying to integrate research. How do you convince your stakeholders to actually do the research? How do you sell the value and make sure that they don't see it as extra time that they need to put into their development process? How do you convince them that this is valuable, this is needed, it's a good thing for your product or your service?

[00:21:03] Nadine: Ideally you'll have some case studies or some examples that you can use from the research you have conducted in this company or previous companies. Ideally you can even link it to revenue, to return on investment. You can even say, look, thanks to this research, and that happened to me in a previous company, we identified this problem, we fixed this problem, now we have conversion up that much, that equals x amount of million per year. If you have those cases, they always work. If you don't have those cases yet, another way could be to find ambassadors. Find the PMs, find the designers that can actually share, when they share about their product, when they say what they've worked on, that they always say we learned this in research, we heard this from users. So that you team up with others that are also talking to senior leadership about why research was so important, why they shaped their product or their plan, their feature, to a better solution thanks to research. I think those would be maybe the two pieces of advice I can give you.

[00:22:08] Host: Good, thank you. I'm going to incorporate the next two questions into one question, talking about the user research subjects, who you include in your research. One, how do you incorporate people with disabilities, and how do you find those users to be able to solve for them specifically, making a mindful decision? And then two, how do you do your research if you can't actually access your end users, or you have limited access to them?

[00:22:41] Nadine: I guess for the second part I would need a more concrete example.

[00:22:47] Host: Let's say you're in a product team, but your product team doesn't get to interact with your end user. You're five layers removed through abstraction and other systems from your actual end users. Maybe you're building a platform for applications for end users. How do you get that close connection to be able to do that end-impact research on your users?

[00:23:14] Nadine: There are many different ways, but you have to find a way. I think there's no way around talking with end users one way or another. If you can't directly get access via your own database, because maybe you don't have them registered with you and they are registered with another product, there are panels out there, there are recruiting companies, there are freelance recruiters that can go out. Maybe these people you can find on LinkedIn, maybe you find them on the street, maybe you find them in any other platform, social media or wherever. But you need to find a way to talk to end users. There's no way around it, I would say, if you actually want to understand who your customers are and what they need.

[00:23:52] Nadine: About the disability question, the accessibility, in the end, I've been listening very carefully also to the first talk, and I have to admit I need to get better at that as well. It's not been the top priority. Maybe we all need to make it top priority, myself included. We just go with the most relevant users, the largest user segment that we aim for, and we try to recruit for those with a little bit of controlling for gender and age and those aspects.

[00:24:22] Host: And for our last question, I'm actually going to ask a question that has disappeared from here, but I think it's super important. A lot of the time our user researchers are a little bit isolated. They might be the only person in a team that is a user researcher, or they might be distributed across the organization, so you might have 20 in a company of 2,000. How do you recommend that they keep up with the trends, with the research, they keep upskilling themselves? From my personal recommendation, communities of practice are great, something like Dublin UX, something like the UXDX community. They're really good for you if you don't have that community in your own company. But what have you seen work to still help them and nurture them and grow them, even though they might be the only UX researcher in their team?

[00:25:11] Nadine: Even if you're the only one with the title of UX research, you are probably not the only one who loves UX research, because amongst designers and PMs and engineers, data analysts, I find a lot of people with a true internal interest, really engaged in user research. So you might only be the only one with the title, but not the only one who feels like a researcher and thinks like a researcher. Try to identify those people and team up with them. The other thing is obviously conferences like this, that always helps with learning. It's also as easy as contacting someone on LinkedIn. Find other people, meet other people, maybe you find someone in the same city that you can talk to. I also have very close relationships still with all the other researchers I worked with in other companies, so try to keep the relationships while you move on.

[00:26:00] Host: Amazing, thank you so much, Nadine. If you do have some questions we haven't answered, please go and find Nadine after we close up in the next five minutes. So thank you again, a round of applause for Nadine.