UX Research In Times Of Covid-19

Henny Melvina, Nazish Zafar at UXDX Community: Singapore. Video: https://youtu.be/8XVs4EaUKO0

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Research at Grab before the pandemic

[00:00:00] Nazish: Hello everyone, my name is Nazish. I lead the design strategy and research team at Grab. I'm based in Singapore, but we do have team members in Indonesia, and the research we do is all around Southeast Asia.

[00:00:12] Henny: Hello, my name is Henny. I run research ops in design strategy and research. I'm based in Jakarta.

[00:00:19] Nazish: Today we're going to give you a flavor of the types of research that we do at Grab, and more importantly, how we adapted it when the pandemic struck. Our key takeaways will be summarized in five points: simplicity, scalability, security, sensitivity and survival. We'll go through them one by one, but first let's get started on what research looks like at Grab.

[00:00:40] At Grab we place a huge premium on doing exploratory research to discover the unknown unknowns. This is beyond the usual usability studies. Our team is obsessed with defining problem spaces and opportunity spaces as clearly as possible. And of course, this type of discovery work requires us to be on the ground to sniff out these unexpected possibilities. We need to know how people make decisions in the context of their environment. For example, how do merchants respond to online food orders while at the same time customers are walking in to dine in? We want to be there to observe the dynamics, and this influences how we design the app. We absolutely want to know the 360 of a person's experience, so that we can reduce friction across the journey.

[00:01:25] As you know, people don't always articulate their challenges in a one-hour spoken interview, because it might not seem relevant until it's in context. Here you can see Henny looking quite worried, and the driver's brows are furrowed quite anxiously. We could only detect how severe this very specific pain point was to their experience because we were right there in the moment in the journey, and we could immediately empathize from their point of view. We were feeling what they were feeling. So this requires being on the ground and literally shadowing the experience, to challenge our view of the existing solutions.

[00:02:01] And we don't do this alone. We often, in fact almost always, do research with our stakeholders, so we don't need to over-explain our findings, because they're right there, and we can align almost immediately on next steps. This really helps with research influence. And wherever possible, we don't just collect data; we like to co-create with our participants, for example on how we should organize the home screen of our app. After all, you are also the expert of your own experiences.

[00:02:30] Then COVID struck. In Singapore, by February of last year, we had already become quite cautious, and we were starting to move our research online. But soon we were all working from home, not just us but even our participants. It felt like flipping a switch, literally one switch, and everything we do, every study we planned, had to go online. And just as immediately, we confronted a lot of challenges. Henny's going to go over some of these challenges with us.

What we lost when research went online

[00:03:00] Henny: Okay, thank you, Nas. One of the challenges is that we lost a crucial source of qualitative data, which was the environment. This is especially important for seeing the interactions between different players in the marketplace, like merchants and drivers. When we switched to online interviews, we realized that our stakeholders stopped turning up as much as they used to. Why? It could be because we limited how many observers could join the Zoom call, because we didn't want to overwhelm the participants. Instead, we streamed the interview on Workplace for them to watch. However, watching on a separate platform made them lose their direct connection to the participant.

[00:03:49] Even when we used a two-way mirror, we could still comment whenever the participants said something interesting and share our thoughts during interviews. Now, if the stakeholders had questions, we asked them to type them over Slack, and we would pull them together at the end. Our interview sessions were no longer interactive. And even though we limited the observers in our Zoom call, users still seemed to feel uncomfortable. I think they were feeling overwhelmed; Zoom was a new format for them. They looked like they were being interrogated, and it made them more cautious and less candid.

[00:04:29] So not only did our stakeholders lose their direct connection with our participants, our participants also lost their connection with us. Previously we could have an open conversation with our users about their experiences, and we could build a relationship with them. For example, in this photo we are having lunch with drivers in Vietnam. It's simply more friendly. Rapport and trust were major things we needed to solve.

[00:05:02] Another problem was the internet. Most of our users in Southeast Asia rely on SIM card providers, and we were randomly losing people from the interview. Suddenly they would disappear, and we could not contact them. The interview would basically be aborted, and it wasn't a good use of time for anyone. And because of the internet connections, usability testing became difficult. We were screen sharing from tools like Figma, ProtoPie and InVision on a laptop, and giving participants mouse control over Zoom so they could take over the screen. When suddenly the prototype stopped loading, we needed to fix the issue on the spot, and everyone would have to wait. This was quite disruptive.

[00:05:56] We were also quite worried about how to protect the confidentiality of our prototypes. How would we prevent leakage online, and what if it went viral on social media before it was even launched? Of course, we had no choice but to adapt. There are five key things that we did, and Nas and I will go through them one by one.

Simplicity: diary studies and everyday tools

[00:06:20] Nazish: These are the challenges we had to confront immediately, and just as immediately we had to adapt. The first thing we did was simplify. We could not go to the users and explore their habitats, so some of our researchers empowered the users to be our eyes and ears on the ground. How did we do this? Through diary studies. Diary studies are not a new method, but we did find ourselves leaning on them a little bit more instead of running in-depth interviews.

[00:06:46] Why did it work? First, it simplified the conversation. Rather than having in-depth one-to-one interviews, we conversed with our participants just like we would with friends over WhatsApp. Think about how much you learn about your friends just by WhatsApping them or Telegramming them or SMSing them. You share pictures, videos, feelings, observations, you gossip, and you learn a ton. So we did the same. We used natural human language, we responded with emojis, we empathized when they complained, and we were right there in the moment when things felt surprising.

[00:07:21] I think we can safely say that the participants enjoyed the company of our researchers. I mean, how often do you have someone who's so attentive to you throughout the day? And our researchers also enjoyed the interactions. They said it's like making a temporary friend for six days. We learned way more than we expected, which really is what any honest conversation should be. It was very organic. I don't even know if you would call it a diary study; more like a WhatsApp study, or a Telegram study, a text study. Basically digital ethnography. And ultimately it helped us escape the awkwardness that we had confronted with the online interviews.

[00:07:59] Another reason we love this method is that it still helped us gain rich context. When we're on the ground ourselves, in cities, on streets, we're exposed to so much stimuli from so many different angles, and we lost all of this. Previously you could see myriad things, you could smell the pollution, the food, you could touch, and taste good food. It's the smorgasbord of stimuli, and that is where the aha moments also come from. So with a diary study, rather than one touchpoint, one interview, which deprives us of this exposure, we could have multiple touchpoints across the day and across many days. We were basically by the side of the participants, shadowing them virtually.

[00:08:48] This method doesn't completely replace being on the ground, but it was a good substitute, good enough for what we needed. The whole point of having many touchpoints is to open ourselves to data from the unexpected corners of people's experiences. How did it work? Very briefly, for participants, we would start with a briefing to discuss their preferences and their constraints, and then end with a debrief at the end of the 10 days to deep dive into curiosities that we couldn't really get hold of through text. Stakeholders would join the debrief at the end of each day. We would sit down, download what we learned, and feed their questions into the next day's conversations, day to day. So in a sense it's like the stakeholders were part of the research trip, a virtual excursion through this methodology.

[00:09:38] So the pandemic essentially forced us to lean into methods that were a bit more lean, a bit more agile, a bit more flexible. And we not only simplified our methods but also the tools, as you can see. Our default initially was to use Zoom or Google Meet, because that's what we use at work, but we were truly struggling, especially when users are not very tech savvy and also have poor internet connections. These two things were a disaster for any online interview. As Henny mentioned, people literally disappeared. Who is that person, where's that person? You have to regroup. It's quite disruptive.

[00:10:16] So when we're simply having a conversation and not sharing anything confidential, we just opted for everyday tools. This photo you see over here is actually a focus group that we ran with people from all around Indonesia on WhatsApp. It was that simple, and we just made it happen. Now, some of you might be wondering, what about all these other UX tools that exist? For streaming interviews there are quite a lot out there, and we've tried some of them, but here's our take. We prefer not to need to teach people how to use a new tool just so they can talk to us, even if it means saying, "Hey, click on this link," because believe it or not, Zoom has been hard enough to explain, and we end up biasing our sample group.

[00:10:58] Also, the internet connection gets laggy because people use SIM cards, and we realized that in Indonesia people are quite sensitive to how much memory they have on their phones. They uninstall and install apps almost daily. So our take is to use apps that people are already using as much as possible, rather than introducing new software. That could change, but that's where we are right now. Most of our studies, however, still take place over Zoom because of enterprise security, especially for prototypes, and Henny will walk us through how we dealt with security during the pandemic.

Security: an internal user panel

[00:11:38] Henny: Okay, the next challenges we needed to solve were security and speed. Grab has offices all across Southeast Asia, so we already have access to people and experiences around the region. Even Grabbers do things like go to the supermarket, look after their children and pay their bills. They come from all walks of life. We started this internal user panel by reaching out to the entire company through several communication channels, like email, Workplace and Slack, and asking people to do a short survey. This kept the sign-up process simple. We attracted them to join the panel by aligning this initiative with Grab's mission of elevating the quality of life for everyone.

[00:12:28] In setting it up, we also faced challenges. The first challenge was creating awareness of this initiative and getting people to join the panel. What we did was work with our HR team to help blast out communications. We even joined the new employee groups to engage them and encourage them to join the panel. The second challenge that we faced was how to operationalize the process. We ran several pilot tests to identify problems and concerns, and eventually streamlined the process.

[00:13:07] Now we use the panel mainly for rapid research, like quick guerrilla testing. We discovered that the internal user panel works well with unmoderated testing tools like Maze. Why? Because users can complete the tasks and answer whenever they want, and they can do it anywhere, which was great, especially during the lockdown period, when we could not just casually find someone to help test out designs. We are also looking at copying the model to build up user panels for the other user segments that Grab serves, our drivers and merchant partners. It takes a lot of effort, but it's useful. We believe having an internal user panel can help, one, test confidential prototypes; two, do super quick testing; and three, reduce costs, because you are not using any third party and giving incentives, and I don't have to go through a long process of quotations and invoices.

Scalability: scaling empathy across continents

[00:14:11] Nazish: The next challenge that we approached was scale. How do you scale empathy? We continued to get a lot of feedback from stakeholders about how they can't travel to the users anymore, and how they might be able to understand the context they're building for better. We have engineers in Beijing and Seattle, our regional team is in Singapore, and we're serving Southeast Asia. So how do we scale empathy across continents? One thing our team started to do was use rich video and photo media in our reports, to make the reports more cinematic. We can't bring stakeholders to the scene, so we bring the scenes to them. We make them feel like they're right in the middle of the operations, so they can see how the products are being used in context, and see and observe for themselves. But they can't eat.

[00:15:01] The next thing we've tried exploring is creating short, focused films, four to seven minutes of documentary. If you think about how moved we feel when we watch a movie, why can't we bring a similar storytelling style into our research reporting? We've done this before for drivers, and we're exploring it now with merchants. And again, we collaborate with other functions at Grab to distribute the videos more widely.

[00:15:28] Finally, one other area we're exploring is virtual immersion. We weren't even the ones to suggest this; it came from the product strategy team, and the idea is still evolving. We haven't actually done this yet, but the idea is to recruit people from different walks of life, say a small shop owner, and have a focus group, not for research, but simply for stakeholders in different cities to understand their outlook and experiences on the ground. So we're creating a channel for direct interaction, and then as a research team we would facilitate a debrief to take away the key observations from that.

Sensitivity: building rapport online

[00:16:01] Henny: We had to focus on rapport. When we do research in person, it's easier to feel a sense of connection and read people's body language. As mentioned earlier, people felt more awkward online. These are the four things that we did. First, we started experimenting with an interactive visual icebreaker, to make the interview less intimidating and help participants open up and voice what they are thinking. We adopted this exercise from a brand called Point of View. In this case, the participant chooses three cards that represent how they feel about COVID and what it means to them. This brings an emotional dimension to our research, and it helps people express themselves in other ways.

[00:16:56] Two, each researcher in our team has a different approach. One of our researchers likes to use a homey Zoom background that shows objects from the local area. It might seem like a small thing, but it helps to create a familiar atmosphere for the participants and shows a sense of understanding.

[00:17:22] Three, start by connecting as people, and treat participants like friends whose honest opinion we want. Rather than getting straight to the research questions, we ask people how everything is at home, how things are with their family. Basically, we spend time acknowledging that we are all stuck at home and coping with the pandemic, instead of brushing off this reality immediately. It gives us something in common, and it builds mutual empathy. In other words, we slowed down a little bit more at the beginning of the interview.

[00:18:00] And four, many of our participants are not familiar with how to use a laptop and a mouse. However, we still needed to use a laptop for our more confidential prototypes. We would share the prototype on our laptop and let the participant take over the screen; this way they would not be able to take screenshots. To help participants get familiar with clicking and hovering with the mouse, one of the designers created this exercise to help them get used to the mouse. It might not be ideal, because it is less intuitive, but it definitely helped us get feedback on our prototypes.

Survival and summary

[00:18:51] Nazish: And finally, we survived. We won't say too much on this, but look, research takes a lot of energy, and empathy is very emotionally consuming. We were all stuck at home, feeling the weight of uncertainty. As they say, you've got to put on your own oxygen mask before you can put it on anyone else. So as a team we spent more time connecting with each other, sharing our experiences and troubleshooting. If I can be completely honest here, we didn't adapt that fast. At first we did the bare minimum, because we were still figuring things out like everyone else. We literally flipped the switch and there were so many challenges. But we did figure it out, and we did it as a team. So our takeaway here is to recharge and be there for one another, because that's what community is ultimately about.

[00:19:35] To summarize, here are the five things that we did to adapt. First, we simplified, to overcome internet challenges, reduce cognitive load, and find a way to gain rich context virtually. Second, scalability, to help stakeholders feel engaged and socialize insights in more creative ways. Security, to run research without fear for confidentiality. Sensitivity, to build rapport with our users. And survival, because we deserve it, and we owed it to ourselves and to one another. So thank you so much for your time, and we look forward to chatting very soon.