Simplifying User Interviews for UX Teams

Tom Higgins, Katie Walker at UXDX Community: Simplifying User Interviews for UX Teams. Video: https://youtu.be/oUmeAX1Jz3E

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions and the power of the conversation

[00:00:08] Tom: Thank you so much, Rory, appreciate the very warm introduction, and good day everyone here listening. It's an absolute pleasure to be here with the UXDX community today, and I'm super excited to share why UX teams are switching their user interviews to Voxpopme. But before we get to all that good stuff, I want to quickly introduce ourselves, although you already did a great job. Very briefly, I am Tom Higgins, Voxpopme's senior product marketing manager, and I spend my days trying to understand how researchers go about understanding the audiences that they care about, and sharing how Voxpopme can help them in that process with video-based qualitative research. And yourself, Katie?

[00:00:49] Katie: Cool, thanks, Tom. I'm Katie, I'm one of the product managers here at Voxpopme, so I tend to focus on building the tools to help you do the interviews. I previously worked in the customer experience team in a major high street bank, so I know a bit of the stuff you were talking about, Rory. I've got experience from both angles.

[00:01:08] Tom: That's awesome, you can share banking stories after the presentation.

[00:01:15] For those of you that don't know Voxpopme, we have spent the last decade scaling customer conversations with our video-first qualitative research platform, and the last two years applying AI to that process to enable video conversations at unbelievable scales, and really just increasing the access to human stories for businesses across the world. Traditionally we've worked with consumer insights teams, some of the world's largest brands, a handful of which you can see on the slide there. But over the past six months we've started to see more and more UX teams coming to work with Voxpopme, because they want to access real user stories to build their understanding of users, understand why they behave the way they do, and really get actionable, emotive insights that they can share within their businesses.

[00:02:05] The shared or common thing across both consumer insights teams and UX research teams is that they all really value the power of the conversation, and know it's an unbelievably powerful way to unlock deep user understanding, and an opportunity to connect and hear what users are saying, what they're feeling, the rationale behind the way they behave. So it's a really incredible way to build a deeper understanding, but also a really powerful way to share user stories within their organizations as well. Taking a conversation that's been had with a user and distributing that to other teams that perhaps don't have direct access to users is just an incredibly rich way to share those stories, build empathy for users and help drive decisions at the end of the day.

The spectrum of ways to talk to users

[00:02:56] But sadly, conversations don't always come easily. I like to think of this as a spectrum on this slide, and there are different ways that UX teams can go about having conversations with their users. On one end of the scale is the in-person user interview, which is an incredible way to connect with real people, as I say, in person. But it has limitations when it comes to geography and scalability. You can only have so many people there in person, and there's the sheer willpower to organize and orchestrate an in-person event: find a venue, get people to travel there, travel there yourself. It ends up creating this incredibly long to-do list, and it really is a lot of hard work to just have a handful of conversations.

[00:03:43] What a lot of people have done is moved to the middle stage of this, the online multi-tool or DIY approach, if you like. Here people are stringing together a series of different software tools: perhaps using Gmail or Outlook to communicate with respondents and try and engage with them, and finding time to schedule their user interviews through Google Calendar or Calendly. Then when they conduct them, they turn to Microsoft Teams or Zoom to host those video calls and record them. But then they've got to work with the data that they've collected and store it in OneDrive or Google Drive, depending on which platform you use.

[00:04:22] And finally, when they get to the point where they want to share those user stories with the business, they're having to turn to things like iMovie, or Premiere Pro if they've got a bit more video editing expertise, just to create those stories and tell them to their business. The alternative is that they just pull text verbatims, which degrades the quality of the research that they've collected, because they've got this rich human story with a real face or a real screen to show the business, but you've distilled it down to just a text verbatim quote, which, like I said, loses some of that power.

[00:04:55] Where there are problems like that, quite often there are software solutions. There are certainly some online user research platforms that give you the ability to conduct UX research out there today. But right now we feel that UX teams aren't getting a fair deal, particularly with the market-leading software in the space for conducting user interviews. In the conversations that we've had with UX teams, we've noticed that those platforms have become incredibly expensive and are putting on pricing pressure, particularly when it comes to negotiating second- and third-year terms of subscriptions. It's just become an unpleasant experience, when really all you want to do is have conversations with users.

[00:05:38] They're enacting harsh use-it-or-lose-it terms when it comes to credits for conducting research, and it seems that all common sense has gone out the window of just having good terms and good business partners to conduct research with. And lastly, real minimal AI capabilities. There's such a feature spread that they're struggling to really embrace AI and use that to make your research process more efficient and more effective, and they're really, like I said, just doing the bare minimum when it comes to integrating AI into those research platforms.

Why UX teams are switching

[00:06:15] So why are UX teams switching to Voxpopme? Well, first and foremost, when it comes to qualitative research, we meet core UX needs. We have everything you need to recruit respondents and run asynchronous, unmoderated video surveys to collect feedback. We are able to do live interviews in secure video calls, so you can do that moderated or unmoderated. You can share and record screens in both those versions as well, so you're able to both see and hear respondents as well as see what they're doing on their screen at the same time. So it's incredibly powerful from that perspective. And then once we've collected all that data, we enable you to use AI to analyze it at speed and scale, and also edit the footage that you've collected and create showreels that you can distribute around your business. So again, the rest of the company gets to see and hear your users based on the research that you've conducted.

[00:07:15] We offer friendlier pricing and terms. There are certainly no harsh use-it-or-lose-it policies when it comes to Voxpopme. We have flexible pricing for everything from incentives for the respondents themselves through to how you access Voxpopme. Many of those brands that I shared earlier in the presentation are on annual subscriptions with Voxpopme, but there is even an option to do ad hoc projects with Voxpopme. Like I said, the natural disposition is for people to move to that subscription so they have that license year long, but again, friendlier pricing and friendlier terms, because we see our customers as partners.

[00:07:52] And lastly, advanced AI analytics. The UX teams that are switching to Voxpopme are always blown away by how advanced our AI capabilities are, specifically when it comes down to analytics. In a moment I'm going to hand over to Katie, who can walk you through a handful of those features.

Interview rooms, video surveys and screen recording

[00:08:11] Katie: Perfect, thanks, Tom. Tom talked a bit about the why; I'm going to show you a little bit about how it works and show you some snippets. If you don't mind skipping on. Perfect. As Tom said, we talked about the different types of things people need, and we've developed software that will effectively cover all the different elements. We look at respondent recruitment, we cover the moderated user interviews, the unmoderated screen recording tasks and also unmoderated video surveys, the AI analysis, and then even the sharing of the insights as well. I'll dig into each one of those in a bit more detail in the next slides.

[00:08:54] Perfect. If we start with the interview route, we've developed a tool called Interview Rooms. From there you can host your user interviews or your focus groups within that platform. That is a secure environment, and it allows you to have conversations with your users. You have moderator links, you have participant links and you have observer links. That means that you can have people watching those interviews and being able to converse with the moderator, saying, oh, can you ask them more about this, and the participant's unaware that that person is watching. It really brings everything into one place.

[00:09:39] Before that, you've got the scheduling side, so anything you need to do to be able to schedule anyone in, you can do that through our platform as well. They'll join through the participant links, you join through the moderator links, and then you conduct those interviews. The best part of it, I think, is the fact that it's all secure. It means that any recordings you do within that interview will go straight through into our platform, and the analysis will start automatically. It means that you don't have to have that problem of running it in Zoom, for instance, downloading the recording, saving it on your computer and then having to upload it again to start that analysis. It's all straightforward and goes straight through to the platform.

[00:10:21] We then also have the ability to scale those conversations. Obviously not everyone has time to be able to do the moderated interviews. If you wanted to do short, sharp surveys, you can do that using the video surveys: literally 60-second little snippet conversations, to either do a precursor to those interviews or a follow-up to those interviews, depending on what you need. So that's that one.

[00:10:45] Perfect. The next one is around interacting with the screen, mainly from an unmoderated perspective. This is where you can see users interacting with your prototypes or your products, and you can see it firsthand, in real time. You can ask them to perform a task and ask them to talk through what they're doing as they're doing it, so you can get that firsthand reaction to what people are doing and whether your prototype or your products are working as you expect them to. That can be done on different devices, so you can do it on desktop, mobile, etc., depending on what you need. You can do that kind of thing through a moderated live interview as well, since we've got the screen share functionality, but the main part of this is the unmoderated side and the ability to see and interact with the customers and the products.

AI analysis, chat and showreels

[00:11:43] Perfect. So you've then done all those interviews and the screen recordings, but you haven't got the time to spend watching all those hours of footage and having to manually review them. This is where the AI comes into play. Tom mentioned a bit about what we've been doing over the last couple of years. The main focus has been how we can reduce that time and that effort required to do that analysis. With Voxpopme we have the AI tool, something we've built, partnered with Microsoft, and that will effectively just start analyzing it for you. It will pull in all the data, all the recordings that you've got, and generate automatic themes, summaries, recommendations.

[00:12:27] And then even if you say, it doesn't quite answer the questions that I wanted it to, you can probe it further. We have a chat function to allow you to ask follow-up questions and ask for a bit more detail, just to make sure you really get the outputs you need. The main aim of it is to reduce the amount of time that you're having to do the analysis, so it means you can then focus on either creating new studies or doing more interviews, or even sharing those outputs with your colleagues to be able to get the buy-in you need. It's all aimed at that reduction in time. On the next slide I'll show you a little bit of how that works.

[00:13:12] Perfect. Excuse the typing noise in the background, but this is our AI chat function. This is a project that we did; it was a survey on Netflix, and it was looking at user behavior in terms of watching power. We've asked it to give a bit more information, and, assuming you can see it all well enough on the screen, you can see that it's coming back with different quotes. All the information that's coming back from the analysis tool is real information, because it's proved by the fact that you've got the quote there. You can click on that quote to be able to listen to it as well. So you can be confident the information that's come back is genuine; it's from the recordings you've come through with.

[00:13:56] There's an example of where you're playing that snippet as well. As you can see, there are the quotes above, and then you have the snippets, the actual recordings, below. From there you can even put them into a showreel. You'll see in a second the little add-to-showreel button that will take it straight through into a showreel, so that means you can then share it with your stakeholders as you need.

[00:14:28] And then I think on the next slide we've got, perfect, just that summary. This is where you can share the insight. Again, we want to make sure we've got that full journey of insights, from collection to outputs. You can take the snippets from what you've got from the AI analysis and put that into a showreel. You can then tailor that showreel for what you need, to make sure that those insights are delivering the impactful results you wanted. That's quite a versatile tool: you can change the layout, change the color schemes, change the branding, etc., as you need.

Stories from switching teams

[00:15:05] Tom: Thank you so much, Katie. I fully take the blame for resetting your video there. But thank you, I really appreciate that walkthrough and seeing those cool features as well. It's incredibly helpful. Before we wrap up, I just wanted to quickly share some stories from the UX teams that are switching to Voxpopme to conduct their user interviews. Very briefly, in the last six months we've seen a huge multinational with dozens of product lines across TVs, personal care, household appliances and more, $15 billion plus annual sales, switch to Voxpopme. A huge international clothing brand with both online and in-store retail locations, again a huge company. And then a member of the tech Magnificent Seven, $200 billion annual sales. I'll make anyone guess which one that is. Another huge company conducting tons of UX research, all switched their user interviews to Voxpopme.

[00:16:08] They're doing that because where their current provider has offered seemingly complex and ever-increasing pricing, Voxpopme has been able to offer them the same capabilities, with those enhanced AI tools as well, but with transparent pricing that's easy to work with. We give them full control of their research budget, and they can start working with Voxpopme with confidence on how far their research budget will go.

[00:16:36] Also fair commercial terms. I hate even having to list this one. I feel like it should just be common sense. As I said earlier, we treat our customers like partners, and we don't enforce harsh use-it-or-lose-it policies that are going to result in you either running around doing research just to use up the last few credits, or feeling like, hey, I've lost out because of credits I didn't get to use. Again, we just use common sense, fair commercial terms with our customers in that instance.

[00:17:07] And lastly, those superior AI capabilities. As Katie just showed very briefly there, you have the ability to sit there and have conversations about your data: really probe and ask questions, ask for unexpected findings, even ask the AI to count how many times people mention specific things. It's just an incredibly powerful way to turn hours and hours of video research into digestible data that can then easily be shared with the other teams in your organization.

[00:17:36] If you would like to learn more and are considering a switch to Voxpopme, I would absolutely encourage you to take out your phone and scan this QR code, and you can book a really short demo with our team, absolutely zero strings attached, but you get to take a closer look at the technology that we've shared there. I appreciate in a short webinar we can only give you a glimpse of what's possible. And most importantly, we love having conversations with UX researchers, so please do reach out to us. I'll hand it back over to Rory for Q&A. I'm excited to answer some of your questions.

Q&A

[00:18:10] Rory: Excellent, thank you very much, Tom and Katie. Just to reiterate, everybody out there, this is your opportunity to ask any questions that you want, so please do write them in on whichever platform that you're watching this on, and I'll pass those questions along to the team here. But I'm going to kick it off with a question, because, call me a little bit skeptical, I'm always a bit nervous about those AI summaries. How do you know that it's picking out the best things? How do you verify that, to make sure that it's not just picking some random bits of information in those summaries and it's picking the really good, useful stuff?

[00:18:54] Tom: Cheers, Rory, that's a great question. I'll go first, and then, Katie, feel free to add to my answer if you've got additional thoughts on it. From my perspective, I think the beauty, particularly when it comes to video-based research, is that the AI can back up the findings that it's presenting with associated clips. If it's saying users get stuck at this point, or, let's take the Netflix example, users choose which movie to watch based on the cast, it's going to show you associated clips. Once you've got that data and you can see those associated clips, you can further validate that point.

[00:19:35] One of the most recent releases that we have is the ability for AI to count mentions of words and phrases within that research as well. So if you're unsure and you're like, hey, this is an interesting finding but I need to probe further, ask it to count the number of times people mention using the cast to choose which movie they're going to watch next. Katie, I don't know if you've got anything to add to that one.

[00:19:57] Katie: Yeah, I just agree with you, Tom. The key thing is the fact that it provides that evidence back to you, so you can verify it yourself and sense-check it. As I said, we partnered with Microsoft to be able to build it, and we've had a lot of customers using it, and generally everyone that's used it has gone, oh wow, okay, it does bring it back. I think when you first try it, you're probably going to do a bit of sense-checking and go, does this match what I would do if I was going to do it myself? And everyone's been coming back saying, yeah, it's exactly what I would do, but it takes 2% of the time versus so much longer.

[00:20:35] Rory: Brilliant, thanks. And on that sense-checking, how much do you propose having a human in the loop? Is all the analysis done, or is there some sense-checking? What's your recommended approach for the human-in-the-loop element?

[00:20:52] Tom: That's a great question, Rory. I know there are probably lots of strong opinions out there on how much we should rely on AI to do a whole manner of different disciplines for us. I always see AI, particularly with Voxpopme's product, as an assistant. The human is the essential part of the process. You've got tons of domain expertise as well as just human intuition. So I would say that the AI can give you an incredible jumping-off point. If you want to jump into a project that's got 100 video responses in it, or 15 IDIs, and you want to quickly understand some of the key themes and see the associated clips, then AI can help you get started and give you some ideas. But then use your own expertise to sense-check the evidence that it's providing, the commentary that's being provided.

[00:21:51] You can also go the other way around as well. You might come in with a preconceived idea that you want to explore and verify. Ask questions to the AI chat and see if it comes back and agrees with the theory or hypothesis that you have, or if it's able to counter the hypothesis that you have. So I think it can be used either at the start of research to come up with new ideas, or at the end to validate the ideas that you do have, or disprove them as well. The short answer is the human is the essential piece of this, and being able to verify the quality of that research is very much a human job still.

[00:22:42] Rory: Great. And just one thing from your example, because it's always hard to really gauge when you're looking at a preset example. Can you give me an idea of timelines? Let's say I want to run one of these sessions. What are we talking about? Is this a day's setup, is it a week's setup? I'm just looking at the initial setup to get my responses, and then we'll dig into how we deal with those responses.

[00:23:12] Tom: That's a great question. I have to give the "it depends" answer a little bit. Obviously, when it comes to user research and all kinds of research, the audience that you want to speak to matters. The more niche the audience, the more difficult it may become to reach them, so that can extend the time. But for pure setup, Voxpopme is a DIY setup, so you can be doing that in less than a few hours. Then you can kick off the recruitment process for something like an unmoderated study with video survey questions, and you can start collecting responses the same day. It might be 24 to 48 hours to complete a project, again depending on which audience you use.

[00:23:58] Voxpopme has got its own on-demand video feedback communities, but we also have access to a global panel as well. So it can be incredibly quick, getting responses straight away. Like I said, if you want to reach an incredibly niche audience, then you might be looking at a few days for a bit more of a specialist recruit. But we're able to fulfill a hell of a lot of research needs with the panels that we've got access to, both owned and through our panel partners. So I would say 24 to 48 hours for video surveys. And again, for unmoderated, you set it up and start the recruitment round the same day as well, and start seeing those potential user interview candidates within 24 to 48 hours as well. Katie, I don't know if you've got anything to add to that one.

[00:24:51] Katie: No, you hit the spot.

[00:24:55] Rory: And is there a limitation? It's fantastic having all of these summarization tools, but I know GPT has a context window and things like that. Is it that you can only have 10 videos? Is there a set limitation to this?

[00:25:12] Tom: I love that question, because it's a journey that we've been on. We partnered with Microsoft to build AI analytics capabilities about three months after OpenAI broke the internet, and I think that was November 2022, if my memory serves me right. We started working with Microsoft, and initially it was, hey, can we summarize one video? Then can we summarize five videos and present some usable findings off the back of that? That quickly scaled, and we were pushing the boundaries more and more to, hey, can we do an entire project of 100 videos? I think the largest study that we ran analysis against was 600 videos with Microsoft. Those were the short-form videos, but we also did a big IDI project as well.

[00:26:07] So we really went on the journey of going from a handful of videos to now what feels like unlimited videos. I'd be careful using that word specifically, but it's got to a point now where you'd be working hard to push beyond the boundaries, and you'd have to get a lot of video in there. Our evolution has seen us go from, hey, can we analyze a handful of videos, to entire projects of videos, then to looking at multiple projects. I did a Christmas study last year and I did a Christmas study this year, and I could compare the two and get summaries across both those projects.

[00:26:44] And now we're at the stage where you can use AI to search across your entire database of qualitative research. We have what we call a research repository, so you could have user interviews stretching across the last year and very quickly search all of that research in a moment, which is incredibly powerful. We see a few benefits of that. One, you can give immediate answers. When the CMO asks you an awkward question in a meeting and you need to give an answer right there and then, you can quickly search across historical research.

[00:27:23] It also does a great job of reducing redundant user research. If you've got an answer to that question from existing user research, then why spend all the time, energy and resources going out conducting further research to verify something that you already know? You can prevent research duplication: if you already know it, then you don't need to do that duplicative effort. And lastly, if you do want to go out and do more research, maybe you've got half an answer to what you need, you're going into that piece of research with a stronger foundation, because you've got more evidence to rely on to build whatever your next piece of research is, whether that's user interviews or something quantitative perhaps. So that's that one. Again, Katie, sorry, hogging all the questions here.

[00:28:16] Katie: I think you gave a very, very thorough answer there, Tom.

[00:28:25] Rory: Well, that brings us to time. I just want to thank you once again, Tom and Katie, for showing us what's new and what's out there. It's very much focused on the Voxpopme tool, but I think what's very useful there is also just expanding people's horizons on what's possible in the market now. Maybe if you're doing a lot of this manual stuff, you could save some time and effort by adopting one of these tools out there. So hopefully that's valuable for everybody there in the audience as well.

[00:28:58] Tom: Yeah, absolutely. And like I say, we would love to have conversations. If anyone that's joined today has a little bit of interest to learn more, give us a shout. We'll happily give you a more in-depth tour of the product or answer any of the questions that we didn't get to in the Q&A. More than happy to pick it up offline.

[00:29:16] Rory: Thank you very much, Katie and Tom.

[00:29:21] Tom: Cheers, Rory, thank you. Thank you, everyone. Bye-bye.