Research Can't Stop: Lessons From N26

Shruti Ramiah at UXDX Community: Germany. Video: https://youtu.be/IZRY3E_onVw

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and two takeaways

[00:00:04] Good afternoon, everyone, thank you for joining today. This is indeed a somewhat surreal experience, but one that we'll all be getting used to, attending virtual conferences, I'm sure. My name is Shruti Ramiah and I lead user research at N26, which is a digital bank, a mobile bank based here in Berlin, and Barcelona and New York. I hope some of you are customers; otherwise you'll be seeing some of the screens today and I hope to get you excited.

[00:00:32] In the last year and nine months I have been at N26. I have been building out this discipline of user research and figuring out how to embed it well within our design and our product management practices. We've built an amazing team of researchers across our different offices, focusing on serving our extended customers across the US and Europe.

[00:00:58] A bit of background about me: I come from a design background. I'm trained as an interaction designer, and that's where I started off, and what you'll see is a bit of this design-led perspective throughout my talk that I bring to my research as well.

[00:01:16] What would I like you to take away today, if nothing else? These two things. First of all, failure in good measure is good for you. And secondly, that research is a creative activity. These two things will hopefully unravel here as I go through my talk.

Working in sprints and the pull of usability testing

[00:01:33] Let's jump right in. I'm going to use the reference of the product development company, the digital product company, because that's, I think, the context that most of us are working in, where most of us are working in an agile methodology and sprints. Oops, I think I need to be helpful with my deck, just a moment. We're seeing it here but not here. Sorry about that, I seem to have an issue with my slide, but I will just describe it in abstraction and work with the situation. It seems to be working out. Yes, there we go.

[00:02:29] We're usually working in sprints two to three weeks long. I'm speaking from the perspective of designers or researchers, but at the end of each sprint your aim is really to deliver a finished piece of the experience and have something, usually pixel-perfect, designed to send to development. That means that in those two to three weeks you're usually ideating, concepting, defining, detailing, and then potentially also evaluating that part of the experience, so you can land on this completed piece by the end of the sprint. And then you do all of that all over again the next sprint.

[00:03:08] It's understandably a really busy process. We have a lot to do: all the rituals to keep our team and our culture running, the extra work on top of all of this to keep an eye on product strategy and the vision, and also to ensure that there's constant quality within what we're delivering. It's a wonder that we fit in these moments for research with customers, but I really appreciate that we do, because it's so important to make customers and users a part of our process.

[00:03:41] But when we are working in this kind of compressed timeline, we tend to go to what we know, and that's natural. We go to the tools that we trust, and in the case of user research that tool is often usability testing. It makes a lot of sense. Usability testing is reliable, it's predictable and really easy to repeat, and it fits really well with our process in this kind of sprint structure, because it comes in and it helps us evaluate and get a sense of confidence for what we're going to be pushing out.

[00:04:15] It helps us know that we're hitting the mark in terms of the experience working well with customers, customers understanding what we're showing them and being able to use it well. It's a real confidence boost often, and it helps us feel reassured. But the risk is that it can get a bit cookie cutter.

When feedback starts to feel like failure

[00:04:37] I hear this sometimes. I hear people say, "Hmm, we did four or five rounds of research or usability testing now," because it doesn't take just one sprint to get a part of the experience done, "and at this point we're perhaps hearing some lukewarm feedback. It's not quite as insightful as it was at the start." And you're wondering if there's something you could have done to learn more or get more input. But you say, isn't some feedback better than no feedback at all?

[00:05:13] The problem arises really when there's negative feedback. When you go into usability testing, and perhaps it's a few iterations down the line, and you hear contradictory input, or the solution really doesn't succeed with customers: they really don't know how to use it, or they're challenging that the solution is right for them, they don't find it helpful. I have heard from many designers that this can feel like a real moment of meltdown, because it often means there needs to be a significant amount of rethinking, or at least taking a step back and looking at the solution that's in place. That means time, and that means perhaps pausing development, or taking a little break and coming back to the design with new eyes.

[00:06:05] This kind of feeling of frustration or disappointment, I think that's really unfortunate, because if feedback starts to feel like failure, perhaps it's really no longer a moment of learning. And isn't learning the essential goal of research?

[00:06:26] Think about it this way. If you've done a sequence of usability tests, or any kind of research with customers, and it's been constantly confirmatory feedback, positive input, everything is working well and ticking along, you've essentially confirmed your initial hypotheses or initial learnings through that journey and patted those down. They're good, you've confirmed them a number of times. It's only when you hear potentially slightly contradictory feedback, or unusual feedback, or things that are challenging what you are putting in front of people, that you are actually expanding your understanding of the user need or the user challenge that you're trying to solve.

[00:07:11] That little bit of discomfort and friction within the research process is actually really important to ensure that we're constantly learning. Evaluative research is really important. I'm definitely not saying it's not; usability testing definitely has its place in the process. The question is, if you're only doing that, are you potentially missing out on what user research can really offer you? That's what I'm going to talk about mainly today.

Building exploration into the whole process

[00:07:37] This is a well-understood model for defining the design process. I won't talk about it in super detail, but discover and define help us to design the right thing, the product definition, and develop and deliver help us design the thing right. This is really bringing the product and experience to life.

[00:08:02] Often we spend quite a bit of time understanding the problem and the solution space in this discovery phase. We do lots of qualitative interviews, we might go out and do ethnographic research, we review all the data we have, we look at competitors. We spend a lot of effort when we're in this moment. But really we spend most of our lives in digital product companies in the develop and deliver phase. For me the challenge is really then to figure out how we might build exploration and inspiration throughout the process. How can research continue to be inspiring and challenging throughout the process?

[00:08:45] Today I'd like to share three approaches that are very concrete, and I hope that you'll be able to literally take them out and do them in your next round of research. I back them up with a few examples from N26. They're just illustrative, and I'd be happy to talk about more in the Q&A.

Approach one: stop short

[00:09:02] The first approach I've called stop short. I'd like you to think about a prototype. A prototype is essentially a story, it's a narrative. We tell people: X thing happens, and then the next thing happens, and then the next thing happens, and if you do this one last step you have this final outcome. Whenever we show people prototypes and there's some missing information or something that isn't detailed out, they essentially fill it with whatever their experiences are of the world, what they understand, what expectations they have for how this experience will work.

[00:09:43] The thing is, when you intentionally leave a gap in that experience and then dig into it and explore it further, you're able to surface and understand what expectations, what assumptions, what comparisons people might be bringing to your product or your experience.

[00:10:03] I'm going to use the example of one of the features that we have in our product that was launched earlier last year, called Spaces. These are some older screens, so you'll recognize that it's changed since then if you're an N26 customer. Spaces are essentially sub-accounts that are really easy to set up within an N26 account. They're intended to help you easily put away money for a savings goal or for a rainy day or whatever you might be wanting to save for. If you look at the leftmost screen, the Brazil vacation, Tracy's birthday, dining out: those are the spaces that this customer has.

[00:10:41] Once we'd set up this part of the product and rolled out Spaces, we started thinking about what it would mean to potentially share a space. In order to do some early exploratory research with our customers, we invited current users of Spaces to come and talk to us about what they were using their spaces for. But also we provoked them with these screens that you see here.

[00:11:11] In the two in the center, the space has a little link icon, the two looped rings, which are meant to indicate potentially sharing or having a connection. And in the middle screen you can see that there are little profile pictures attached. We literally just added those to existing screens, so it was a really quick exercise. Then we asked these customers, who were existing Spaces users, to look at this and say, "What do you think these little indicators might mean?" And they said perhaps sharing, or that it's shared with another person.

[00:11:44] Then we were able to dive in further and talk to them about: "What would you share this space with?" "A vacation, you might share that with a partner." That's interesting. "Dining out, you want to be splitting bills." Okay, what would that situation look like? When would you want to set up a shared space? How do you think it would work? With these little prompts we were able to start this longer narrative and get an understanding of the customers' needs and the use cases that they might have.

[00:12:14] In the screen to the right, the settings screen, you can see that bottom section called members. We just added that to the existing settings screen, and we asked people to describe to us what would happen once they clicked on new contributor. What would happen next? They often described to us the journey that would happen to invite another person into their shared space. And then eventually we talked also about what it would mean if they needed to manage that space, or get rid of that person if things went wrong.

[00:12:47] A really big learning for us from this really quick test was that even though shared spaces is a financial product, and you're going to be sharing money and control of money with other people, the bar that people had in terms of how the invitation process and the administration process should go was usually a reference to WhatsApp or other messaging channels. People expected the group administration experience, the invitation experience, to be as easy and as simple as a WhatsApp group. And this was a really good bar to set for us in terms of how easy we need to make it.

[00:13:30] I do want to say these kinds of provocation screens are not the only way to stop short. There are many different ways, and one of my favorites, which has fallen out of favor, is wireframes and sketches. I do encourage everyone to try those as well.

Approach two: layer the cake

[00:13:46] Moving to the second approach, I call this layer the cake. When we present great design, our completed piece of good design is usually the amalgamation of so many layers of complexity. There's the business concept or the product concept, there's information architecture and navigation, there's UI, the visual design, there's UX copy, graphics, illustration, there might be micro-interactions or motion design. There are so many things coming together to make a great experience.

[00:14:26] Now when you know that you have some problems or issues with an experience, and you try to diagnose them with all those layers sandwiched together, that can be really challenging, because you're not able to pinpoint exactly where your problem might lie. You might spot one problem but not diagnose others. The approach that is suggested here is really to pull apart those layers, and in order to do that you need to take a hypothesis-driven approach.

[00:14:57] Early last year we had these three tiers to our products. We had the free standard tier, that's the far left, the clear card. In the middle is our N26 Black, and to the right is N26 Metal, Black and Metal being our two paid tiers. What we were looking at with the membership tiers was that we weren't seeing that many upgrades. With product and design, we were thinking about whether we needed to rethink the product offering or the value proposition, whether we had the message right.

[00:15:30] But one of the designers had a hunch. He had a hunch that we were just not making this option visible enough for our customers. Of course on the website you see all the different options really clearly, but once you're a customer and you're in the app, this is where you would see the upgrade option. This is the my account screen, where you usually go to manage your account or manage your card, and it just had this little green box saying "Get more from your account" and upgrade. His hunch was that this is not where people were expecting to see memberships. This was not the place they would go to find out more about membership.

[00:16:10] In order to test this, we singled it out into the information architecture layer and did a card sorting exercise, which is actually quite simple to do, in order to see where people would actually put this upgrade option. Unsurprisingly, it wasn't in my account. The path to the solution was a little bit more complex and not as straightforward, as customers don't always just give you the answer. We did have to do a lot of iteration to find the right solution.

[00:16:43] But the solution we landed on was an Explore tab, a tab that's dedicated to helping you understand what the product can offer you and what else you can get from other tiers. Straight up, it's pretty easy to see that there's a lot more real estate, and we were able to tell a much better story about these two tiers that we have on offer.

[00:17:05] Customers who follow the N26 product will know that mid last year we launched a new product called N26 You, and that replaced N26 Black. So we did rethink the whole value proposition and the product offering. It doesn't mean that by diagnosing one problem you have diagnosed them all. But by pulling apart the layers and going deep into each one, laser-focused on trying to solve that one, you're able to ensure that you're not accidentally throwing the baby out with the bathwater.

Approach three: go to the extremes

[00:17:46] Then finally, the third approach I'd like to share is go to the extremes. When we build a product, we usually have an idea of what the typical usage would be like and who the typical customer is. When we do usability testing or user testing, we often focus on this ideal user or target customer. We're inviting the people that we imagine will love and use our product the most.

[00:18:14] In this approach I encourage you to go out and speak to the people who are absolutely on the other ends of the spectrum. Who are the people who are the rejecters and absolutely don't want to use your product? Or perhaps who are using it in ways that are unusual, or almost look like misuse to your eyes, looking at what you had imagined as the classic usage of the product?

[00:18:40] An example here is again with Spaces. Once we launched Spaces, we had this idea that the typical usage would potentially be a holiday space or something like that, a purchase goal that someone has, with a foreseeable date. They know when they need to get it done, and the amount of money that they're hoping to save for that. And potentially another bucket that's putting money away for a rainy day, for an emergency or surprise purchase.

[00:19:09] When we looked at how people were using the product, we saw these groups of people who had either tried Spaces out, and they were really engaged customers of N26, but they were just not using Spaces. Or we also saw people who had eight, nine spaces, and we were really curious: what are they doing with so many spaces? How is this actually working for them? This is definitely not our imagined case for the product. We invited those people in, just to come in and tell us: what is their use case, how do they find it, what are they finding helpful or not so helpful about the product, how would they like to change it and adapt it?

[00:19:51] This really helped us to surface potential use cases. Those maximizing users with eight to nine spaces helped us see the way that we could grow this product this year, by budgeting means. But on the other hand, the people who weren't using Spaces helped us see what are those little nudges, or those last features, that we need to provide these users in order to make Spaces something meaningful for them.

[00:20:18] One of the really fun, or I find really interesting, features that came out of this research was the ability to lock a space. Between your spaces, it's as easy as dragging and dropping, tap and move to another space. It's that straightforward to move money between your spaces, and usually we get really positive feedback about that. People are really impressed by how easy and intuitive that is.

[00:20:44] But from some of our users we heard, "That's too easy. It's really easy for me to go back on the commitments I've made to myself. I said I was going to save for this goal, but I'm going out tonight and it's too easy for me to drag money back into my main account and spend it." For that feedback, for those users, we introduced this lock space functionality. That means that you can always move money into that space, but you can't move money out unless you go into the settings and proactively release the lock.

[00:21:17] That just means that there's a little bit more friction to ensure that you're able to stick with your goals. You have to take a moment to think about it: do I really want to unlock this space right now and move this money out? Is it worth it? That's a little insight that we gained from talking to those customers that were not in our typical range of users.

Research as a creative challenge

[00:21:38] These are the three approaches I wanted to share with you. Stop short, which is really good for surfacing the expectations or the assumptions, the references that customers are going to bring to your product and your experience. Layer the cake, which is when you need to diagnose exactly where your problem is and solve it with laser focus. And go to the extremes, which is great for when you're trying to look at the possibilities and the opportunities, especially for a stable product that's already in existence.

[00:22:11] What I hope I've shown you is that research isn't just about executing standard methods or implementing standard approaches. It's really about looking at each research question as a creative challenge, and thinking about what's the right situation or experiment that I need to create, and what's the stimulus that I can put into this experiment in order to elicit the right response, the right feedback, from my customers.

Pro bono support during COVID-19

[00:22:41] Great. One more thing. I wanted to acknowledge that we're all in this really unusual situation. The fact that I'm speaking to 80 people over a screen from my home is unusual. COVID-19 has really brought out some of the worst and the best in all of us, and we're really looking to give back as best as we can.

[00:23:07] One of the ways I will be giving back, and some of my colleagues as well, is we're hoping to provide pro bono support and advice around user research, design and also product strategy to organizations who are working to create positive social impact. My ask to you is: if you work for an organization, or if you're part of an organization, that's working with positive social impact, please go to this link and see if there's a way that we can support you that would be helpful.

[00:23:39] There are a few different models that we've put up. It's all an experiment; we're open to a bit of failure, of course, and to learn and move forward based on what organizations actually need. And of course, if you know someone who might benefit from this, please share this link with them as well. I'm going to give a big thank you to UXDX for giving me this platform to share this initiative, born out of unusual circumstances. With that, thank you very much, and there are a couple of links to get in touch with me.