Creating a Continuous Learning Culture

Marc Majers, Ryan Leffel, Jennifer Cardello at UXDX USA. Video: https://www.youtube.com/watch?v=msI40cSyz6s

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions

[00:00:00] Jen: Thank you, Rory, and I'm so excited to see Ryan and Subrashi [?] and Marc again. Well, I didn't see you last time, we just talked, but here we are. We have some provocative questions that we want to talk about when it comes to continuous learning. As I mentioned previously, I'm voraciously reading Teresa's book, if you could see it, and trying to embed these practices in what we're doing at Fidelity. We created a continuous discovery and feedback program this past year in order to really embed the idea of continuous learning into a culture that is going through an agile transformation and had previously been very waterfall and very risk-averse, obviously, because we are Fidelity and we have a brand to uphold. But these experts are going to help us dig into those provocative questions and talk about what continuous learning is, and how we can unblock our cultures to make it work in the different types of organizations that we're in. So thank you for joining me.

What continuous learning means

[00:01:04] Jen: This is the question we talked about last time. I'd love to have each of you tell us what continuous learning is to you, in your org and for your practice. Subrashi, do you want to start?

[00:01:18] Subrashi: Yeah, sure. As you were showing the book, I was like, yeah, that's what I'm going to give to all of our employees, so that we can become those people. I won't say that we have understood yet what continuous learning is, or that we have become experts on continuous learning, but we are continuously trying to learn more about it and making sure that we are at least moving in that direction. As of now, we try to collect all the data around NPS, surveys, talking with our users, or using processes like jobs to be done, and again looking at what the users are actually doing in our product, to make sure that we at least have a base in what the users are doing. Then we can prioritize based on the problems they are facing on a daily basis using our product, so that we can go in the direction of solving those problems, or optimizing our solution, or making sure that the overall experience of our product for our users is getting better. But yeah, we are definitely not the experts yet in that domain.

[00:02:31] Jen: It's very meta, because you're continuously learning how to continuously learn. But I like the idea of bringing together and synthesizing multiple sources of data, different types of data, in order to inform your next steps. Which is really what you're looking to do: step through it, instead of coming up with a grand plan that you're going to execute on, launch something in 18 months, and then see if it works. It seems like you're doing a great job of pulling those data sources together to inform decisions. How about you, Marc? How do you define it?

[00:03:08] Marc: That's a really good question. I know the conference is team-focused, so I keep thinking about the team, and I think continuous learning is also getting people to see the aha moment. Because with UX, there are always teams that are changing, and there's always a culture where you're constantly trying to move forward and get something out into the world for someone to use your product. But during that process, there needs to be continual learning about the value of user experience.

[00:03:42] And I think the only way you can do that is a couple of ways. One is, throughout the process, and I talk about this in my talk, you involve developers, stakeholders, product owners in the actual research. Make sure you carve out time ahead of the sprint, ahead of the PI. Involve everybody, so they can actually watch the studies and participate. From that there'll be learning opportunities, because they're involved. They'll ask questions as well, and they're involved in the process. And I think when you're involved in the process, you have different aha moments, and that could also lead to the opportunity to hold some round tables and talk about more specific UX topics with your team and the group that you're working with.

[00:04:30] So there are a lot of doors that open with continuous learning. I think the data is definitely important, because you're collecting the information and it has to be valuable to pass forward, and there's a lot of process there. But I think it does boil down to the team. If you don't have a team that's fully engaged, then, well, we probably all have products that we work with where we wonder who worked on this, and I think it's because the team wasn't involved early on.

[00:05:00] Jen: Yeah, it's one of the tenets of good product, of building highly functioning teams: that they can all agree on what the right problem is that they're trying to solve, and on how the solutions that they're creating are actually resonating, and having people bear witness to that in person, to actually see it for themselves. Basically, the learning that you're creating is like continuous empathy. It's really getting close to humans to understand the response to the things that you're building, so that together you can make great decisions. Because what I see is teams that don't function well typically just don't agree. They don't agree on what problem they're solving. They don't agree on the solution that's been created. And unless they hear it from the horse's mouth, which should be the users and customers, it's hard for them to come to agreement. So that's amazing. You're actually building high-functioning teams through this practice.

[00:06:05] Jen: And what about you, Ryan? What's the deal at Priceline?

[00:06:09] Ryan: To me, continuous learning is really being in tune with what your customers need and, ultimately, what will make them more successful. Customer needs change all the time, and we don't always know what our customers want. If we did, we would just be perfect, right? We would always have it right. So I think it's really paying attention to what the data's saying, both qualitative and quantitative. I think there's also a big piece of this about trends: understanding what the trends are in your particular market, and how you could use those trends to help create something useful, and also talking to your customer. That's certainly been a theme. But hearing directly from your customer what they need and what they like and what they don't like, that's really important.

[00:06:58] I think any smart brand, one way or another, is continuously learning. I think the bigger question is how you adapt or adjust based on what those things are. At Priceline, I think we really thrive on experimentation. The more we test, the more we learn. These could be very simple, quick tests, or they could be large, complex initiatives, but you can still experiment into those bigger ideas. But at the end of the day, you really need to make decisions based on what your customers need, and make sure you're providing something that is valuable and meaningful.

Perfection, and nothing is ever done

[00:07:36] Jen: You mentioned a word that always becomes an issue, which is perfect, and perfection, which seems to sometimes get in the way. It doesn't interact well with the idea of continuous learning, and getting to perfect. How does Priceline deal with that? What is it about your culture where that seems to work well? It doesn't have to be perfect the first time is what I think you all have realized. You're experimenting your way to a better reality for your customers.

[00:08:10] Ryan: Yeah. What I was saying there is, if we knew what our customers wanted, then we would just be able to deliver it to them, and it would always be good and it would always be right. It's just not the case. We have a lot of different ways that we track and measure what we're testing, so that we can understand how something is performing. It's never going to be perfect. I don't think anything is ever going to be perfect, and that's because things constantly change. But it's our job just to make sure it's as good as possible. By getting continuous feedback and really tracking how the business is performing and understanding how customers feel, you get it as close to, I hear you on perfect, you get it as close to perfect, you get it good enough.

[00:08:55] But it doesn't stop there. You feel like something's good enough and keep going with it, because there's always a next step, there's a next move. I think a lot of the time, people who do a lot of testing have the mindset of, "This is good, it's working really well, it's done." It's like sticking it in a drawer, putting it in a box, moving on to the next thing. Whereas I think a better way to think of it is, "This is good, it's working really well, maybe we can step away from this for a little bit." But again, with customers, things are going to change. People's needs are going to change. Let's come back to that thing that we know is working really well and figure out how to make it even better.

[00:09:33] Jen: So there's a promise, essentially, that you're going to come back to it. It's not done. Nothing's done.

[00:09:41] Ryan: Yeah, and I don't think anything's done. I think projects continuously evolve, and what people are looking for changes, and that evolves. So it's not like you could just develop a feature, build something new, put in copy that works really well in that moment, because in another two weeks or a month, things could change, and there might be something else at that time.

From MVP to market ready product

[00:10:06] Marc: I was just going to add, Jen: I introduced in my talk this idea of a market ready product, that we have to get rid of MVP, the minimum viable product, this idea that we're going to give the minimum. I think what we're talking about is that we're experimenting because we want to actually put something in their hands that they are going to enjoy, that's going to be pleasant, where they're actually going to be able to complete the action they need to do. I understand that MVP is an old term; we've used that term for a long time. But if you think about it, who really wants the minimum of anything?

[00:10:49] So if we can keep pushing ourselves forward and thinking about what is something that would actually be ready for someone to use, not just the bare bones, but actually the features that will allow them to enjoy their day. Because all products have competitors. If I don't continually improve today, my competitor tomorrow is going to make that advancement, and so shooting for the minimum isn't really going to make you competitive. I think we're talking about the same thing. We're constantly trying to experiment and think about ways that we can improve, and the only way to do that is to step into the user's shoes and understand what their needs are. That takes work.

[00:11:35] Jen: Yeah, I like the idea of the MRP, and I've heard people previously use minimum lovable product. But I think MVP is a definition that has been misinterpreted. It's been changed to mean what teams wanted it to mean, or what companies wanted it to mean. I'd love to dig into that more. How do you know that it meets the criteria of an MRP? Is that standardized criteria? Is it dependent on the project that you're working on? And Subrashi too, is there a concept like that at LexisNexis that you use, and is there data that you use to prove that out, or predict market ready?

[00:12:19] Subrashi: We don't necessarily use that concept of MVP or MRP, but we definitely try to use data to make sure that there is a need for that customer problem. And again, I guess, as people were talking about in previous talks, there is a very important step of making sure that the business need meets the user need. Of course we are going to solve the user problem, but we'll also have to make sure that there is a business need for this, and of course that requires a lot of data to make sure that we are getting to a solution for both of those problems.

[00:13:03] Jen: And you mentioned you're using jobs to be done. Is that part of that, really uncovering where the prioritized user needs are, and the value that can be brought to market?

[00:13:13] Subrashi: Yeah, definitely. As I mentioned in my talk as well, being a company like LexisNexis, where we have lots of different products, there are always chances that we can improve something, or we can make enhancements, we can create new features. But to prioritize, to make sure which one we need to work on first, we use jobs to be done, and we also use other sources of data like NPS or surveys, and we talk to the users, and then try to come to a conclusion: this is the one we prioritize, this is the one we are going to go with, and everyone agrees.

[00:13:54] Jen: Right, it's magical. We have the data, so we all agree this is what we're going to do. I guess that's the other piece of it, the nuance to it: there has to be agreement in the organization. It's not always going to be very crystal clear.

[00:14:07] Subrashi: Exactly, and that's why I feel like combining all of these different sources of data helps a little bit. Because if you are only saying that our users are asking us to build this and that's why we are building it, there will be lots of people saying, "No, that doesn't make sense." Or if there's only surveys coming in saying that some of the users are saying that, if one source of data is telling us to do one thing, then that might not be the best solution to go with. But once you are combining different sources of data, you are getting rid of the biases that are associated with one source of data. So hopefully we are going to a solution which makes more sense, which is more viable, and which is going to solve the user problem.

Surfacing opportunities versus iterating on solutions

[00:14:53] Jen: Awesome. I wanted to ask about continuous learning for surfacing opportunities versus iterating on solutions. Are there certain methods that you use when you're trying to surface opportunities? It sounds like jobs is something that LexisNexis is using. Are other people using jobs to surface good opportunities worth going after, and are there certain methods that you would use for going after in-market improvements of things you're already offering? What's your toolkit, Ryan? Do you have a toolkit?

[00:15:27] Ryan: I think it'd be a lot of different things. There are always a lot of problems to solve. I think the bigger challenge is prioritizing and figuring out what problems you should solve now, because they're going to have the biggest impact. Different organizations solve this in a lot of different ways. It could be working with jobs, it could be countless various things. For example, a lot of companies have goals, and those goals become KPIs that you need to hit, and you break those KPIs apart, and it turns into different outcomes and different outputs that you have to create. So there are really a lot of different ways to go about it. But going back to what we were talking about earlier, I think it's a balancing act of really understanding what's needed and why.

[00:16:21] One good example that I've talked about in the past: think about a Swiss Army knife. You ask a customer early on, and everything sounds great; they just need all these things. But then the product is out there, and you realize all they really need is a blade and a saw. No one really cares about the lighter and the toothpick and all these other things that go with it.

[00:16:41] Jen: The wine bottle opener, though, we can agree.

[00:16:45] Ryan: Yeah, that's true, the wine bottle opener is important. But then maybe you're thinking about how do I just make that a better wine opener, because people aren't focusing on these other things. It's really easy to get caught up in all the complexity and all the features. And really, if you're paying attention to what people are doing and really hearing them, because again, people are going to say they want everything, it's really about asking the right questions at the right times and listening to your data, and that will actually help you be precise and really fine-tune what it is that you should be working on, identifying the right problem.

[00:17:23] Jen: So it's truly mixed methods. You're going to use the toolkit, what you need. What about you, Marc?

[00:17:28] Marc: Well, you bring up a great thing, and I always think of it this way: you've got two questions, you're kind of at a crossroads. It's: am I building the right thing, or is the thing I built working? So you've got this crossroads. Are you going to do generative research, where you're going to go out and do some observations and try to understand what's happening out there? In that, I found many opportunities, things that I had no idea about, because so many people had significant workarounds that even the administrator of the system had no idea people were doing these types of things. And you're like, that's an opportunity for a brand new product or feature.

[00:18:10] Then you've got the other side, where you've got a prototype and you're building it, you're going to do some usability testing, and you're really trying to fine-tune something. So I think you've got those two channels, and what I found in my career doing research is that there's value in both. It's going to depend on the organization and where they're at. You could be doing a very significant redesign of some software at one point, and that might be a really good time to do some field research.

[00:18:38] And now things are opening up, so you may be able to actually go out there and sit behind somebody and watch them do their work. Obviously you can still do that in Zoom, but you're going to be missing out on some of those things, like that cheat sheet that's hanging in their cube that has all the magical stuff that they're not going to show you. Those are the little artifacts that we UX nerds love. It's like, "Oh my gosh, I can't believe I just walked away with a gold mine." So I think with some of these techniques you can really come back with tremendous amounts of innovation and opportunity.

When it's not the right thing after all

[00:19:09] Jen: Do any of you ever have a situation where someone thinks that what they're asking you to do is help make something better, but when you go in to do the research and the investigation, it turns out, exactly what Marc was just talking about, it's not about building the thing right or improving it. What I mean is, it turns out it's just not the right thing, or it's not what people thought it was going to do, and it's not about tweaking it anymore. How do you back up a team and say, if we can't fine-tune this to get the outcomes that we're seeking, how do you turn the conversation into one about reinvention?

[00:19:58] Marc: One little tidbit, and often I'll throw it to the other speakers, but to me it's the same thing from the beginning. Do you have an artifact? Were you able to record the situation? Most companies won't let you record video when you go and do an observation. You can only record audio, or maybe you might be able to do a screenshot if you're lucky, depending on what vertical or sector they're in. But bringing back that little piece of evidence, you're like a detective. You're going back with something to say, "Review this. This may change your mind."

[00:20:35] Jen: Yeah, I'm thinking about the data that's in the system as well. I just think sometimes teams fall in love with their solutions, and it can be hard to hear that this isn't actually solving the problem that we thought it was, and we need to back up a little bit. I wonder if that happens at LexisNexis. Does that ever happen?

[00:20:57] Subrashi: Yeah, I guess it happens all the time. As I talked about in my presentation, we were trying to solve a very simple problem, an apparently simple problem, of printing or emailing in our product. But it involved a solution that needs to be compatible with the browser interaction as well. Because when you interact with a product in the browser, there is a component where you need to interact with the browser as well to get your documents downloaded to your system, or emailed from your system. So we were dealing with that, and it gets very, I guess, scary, that it's just a very simple solution, to be able to download, and if we are not able to provide that to our customers, then what does it mean for us?

[00:21:48] But I want to again agree with Marc, as he was saying: evidence and data are very important here. And that data can be from any source. It doesn't have to be only the usage data that you are getting from your customers. It can be survey feedback, NPS data, or just looking at your users and seeing what they are actually doing in the system.

[00:22:12] And then another thing is the art of pivoting. It's very easy to say, "Okay, let's forget about that solution and go back to the problem, or go back to the drawing board and figure out the next solution." But of course, if you have been working on that project for three months, six months, then how do you detach yourself from it and go back to the problem space? But I guess it's working together, and again, convincing people with data that no, this is probably not the right solution, and it will probably cost us more if we build a solution that's not correct for our users. So it's probably better to take a step back now, go back to the drawing board, and try to figure out a better solution.

Enablers and blockers

[00:22:52] Jen: Yeah, that's awesome. Okay, we're starting to get some questions, and I want to make sure we answer those questions that are coming in, and the awesome comments that are coming in through the chat. I would love to know about enablers and blockers. What enables us to work this way, to help the organization? Are there cultural signals, things that you look for in behaviors in the organization, types of leaders, types of technology? What are the enablers and the blockers that people should look for?

[00:23:32] Ryan: I think enablers really do come down to culture. I think passion, enthusiasm, curiosity are really important. I think having people know that it's okay to fail. Sometimes that's just really where you're going to learn. And I think if you have those things, your culture's probably headed in the right direction. In terms of blockers, and this is going to seem pretty broad, but from my experience blockers usually come down to time and money, or resources. That's really where I've seen blockers in the past: time, money and resources.

[00:24:18] Jen: It's always the Venn diagram. You can have it fast, you can have it cheap, or you can have it right. What about you, Marc? Enablers and blockers.

[00:24:30] Marc: I'd just add a couple in there. I think there are the standard ones, but a couple of others: you need to have a group of champions, folks at your company that support what you're doing, and that could come from leadership, and it also could come from some colleagues, some other groups. Look for the enthusiasts. Those will help you spread the word throughout those other groups when you're not there. And also, there's a whole lot of research techniques, whether it's secondhand research or anything out there, that other champions can do, so they can bring you some of the more heavy-hitter work that needs to get done. I think that's one.

[00:25:14] And then enablers, in the same vein as we were talking about before: make sure that you open up. You can possibly open up UX office hours, so you're available, so that if someone does have a question, they don't have to schedule a meeting with you; they can just come and bring something. There's a lot of validity to that. Also, those round tables are valuable, where you're just spreading knowledge. Maybe you learned something on a particular study that, because you're in a silo, another group may benefit from, so you could do a session on what you just learned, or maybe it's another high-level topic. So I think as long as you're open, that does help, and then finding others to join in will continually create this aura of user experience happiness.

[00:26:04] Jen: Speaking of round tables, do you have a scaffolding for that? Do you have that every month, where you're bringing insights to the organization that are surfacing, and anyone can come and listen?

[00:26:17] Marc: I would say quarterly if you can, because it does take time to put together something that is worthwhile attending. You do want someone to walk away with a lesson, or get them involved. So if you could at least do it quarterly, I think that would be a good schedule.

[00:26:37] Jen: Awesome. And Subrashi, what about you? Thinking of enablers and blockers.

[00:26:43] Subrashi: I completely agree with everything Ryan and Marc have said, but another thing I want to say, and it might sound controversial, is people who are not willing to fall into line. I especially see that at LexisNexis, because it's a legacy organization. We have been there forever, so there are people who have been working with us for 20, 30, 40 years. And sometimes, I don't want to say everybody, but sometimes if you want to talk to them about trying something new, there's pushback: "No, it's going well. Why do you want to try something new? Why do you want to go in a different way?"

[00:27:21] So people who are willing to, I guess, think outside the box a little bit and be willing to try something new are also very important to push the organization in a different direction, or a better direction. So those are probably the blockers. But culture-wise, it's very important to have people who are willing to try out new things, and, as Marc and Ryan mentioned, people who are ready to experiment, people who are ready to fail fast, so that they can try something new.

Unlearning old habits

[00:27:52] Jen: We have a question here that I think all three of you can answer, and it segues right into what you were saying. Is there a way for people to unlearn old habits that prevent them from believing in continuous learning and supporting it? Have any of you had to do that and seen it work?

[00:28:15] Subrashi: Yeah, I guess we have to do that every day, because as human beings we are always set in our ways of doing things. I was reading something about the idea of divergent thinking versus convergent thinking. I think that can be very beneficial while we are trying to solve a problem, because again, as human beings, we try to jump to the solution very quickly. But take a moment just to think: this is the problem space, this is what the customer problem is, and this is what I need to solve, and then go from there. Because we are always like, "Oh, I know what the customers want. We need to create that, we need to change that, we need to update that." But just maybe take a moment, and that's not only for yourself but for the entire team involved in working on that project or product or feature: let's just take a step back and try to think about the problem for a moment.

[00:29:10] Jen: Yeah, that seems like something that needs to be agreed upon. It's like a working agreement that there's an opportunity space, a problem space, and there's a solution space. It gives us the time and the dedication to actually unearth the root causes of the true unmet need. And then, because we're calling out that there's a solution space, it means that we're going to diverge and think about many ways to solve the problem. I think that's a big piece of the enabler: an organization that says, "Yes, I agree with that. We put that on paper, we put that on screens, and that's the way we work."

[00:29:50] Marc: I was just going to add, Jen, on that note: a methodology like the five whys, or even a usability test where you're talking with someone one-on-one instead of a big group, is actually using a specific methodology to think a different way. You will actually come out the other side with different ideas, because we all seem to grapple with the same concepts every day. If we can push ourselves by trying one of these methods, because it pushes you to diverge or converge, you can actually come out the other side, and that could actually change the way you think.

Exposing leaders to users

[00:30:29] Jen: That makes me think of something else. There's another question here. It says: in organizations that have a hierarchical structure, where the directors, VPs and C-level are multiple levels separated from the user, how do you get them to take time out of their busy schedules to observe the users? I've noticed that our president of personal investing, on her way to work every morning, would listen to calls with our customers. So she's already absorbing and empathizing and being open to bold experimentation for how to solve these problems that are very heartfelt. She made a dedication to doing that. But how do you expose your leaders to your end users? What are the techniques?

[00:31:18] Ryan: I would say now they are very much engaged, so it's not really something I have to live with. But I have been in situations in the past where it's not really the case, and I think there are a few things. When you talk to your customers, you're on a platform, you're recording it, having those conversations. I think when people actually hear what their customers are saying, it's kind of a game changer. It's probably one of the best ways to get an executive's attention: to say, "Check out this video. This is what one of your customers just said about their experience." It's powerful, and it really works. And when you're able to couple that with data and benchmarks, it becomes even more powerful. So it's not only "that's what this person said"; it's "that's what this person said, and these are the KPIs we should be striving for." When you start to pair these things together, it really does lead to an impactful message.

[00:32:12] Jen: Do you do that proactively, or is that only on demand? Are you proactively surfacing these feelings and the recordings and the data? This question's for all of you. We talked about Marc having the round table every quarter. What methods have you found for pushing things out? For instance, we have a Yammer channel at Fidelity, and I've really wanted to have an insights channel on Yammer, so that we're continuously putting stuff out there for everyone to see in the Fidelity community. Do you have channels like that that you've created in your companies?

[00:32:56] Marc: There have been channels like that. I think the challenge is that, as you said earlier, you're talking about individuals that are really busy. So if you can figure out a way... The video is good. I've also done, for example, the instant replay on a site. You've got companies like Tealeaf and ClickTale and Hotjar, there's a whole list of them you can go down.

[00:33:24] Jen: Contentsquare.

[00:33:25] Marc: Yeah. If you can replay sessions where there was some issue that you isolated, it's amazing. Not only usability tests, but being able to view and see, that's helpful. And then in those sessions, yes, I think you still need to figure out a way to summarize it, because the raw data is just like a fire hose. So if there's curation, summarize it.

[00:33:52] Ryan: Yeah, I actually agree, that's a great point. There are some great tools out there for scroll maps and user recordings, and I 100% agree those tell a great story, and they do have an impact, for sure.

[00:34:07] Jen: Yeah, and I think the storytelling capability, Subrashi, you're doing that with data, which is amazing. Our UX leaders are doing that as well, pulling in data from folks like yourself, and also from conversations with users. I think storytelling is one of the most critical skills for us to get the impact that we're looking for from the learnings that we're getting.

[00:34:27] Subrashi: As Ryan was saying, we also like to do that in our company: combining the what and the why. If one user is saying something, let's see if that's true for all the other users in our company for that particular feature or product as well. That tells a very compelling story, combining the why with the what.

[00:34:47] Jen: I love the what plus why. We have run out of time, and I think we could talk for days about this, but I wanted to thank Ryan, Subrashi and Marc. This is awesome. You guys are charting the course for helping people learn continuously. So thank you so much.