Key Strategies to Win Executive Backing on UX to Ignite Design-Led Business Growth

Jim Morris, Rima Campbell, Fahad Osmani, Ryan Leffel at UXDX USA. Video: https://youtu.be/ewTtclHdmb8

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions and a read of the room

[00:00:00] Jim: I'm Jim Morris, I'm the founder of the Product Discovery Group. I started my career in Silicon Valley startups and I've gravitated into leadership and product management, and now I coach leaders and teams in startups and large enterprises in product discovery and in digital transformation. Yesterday I led a workshop about building better prototypes and user testing. Who's there? There we go, yay. And I teach a class at UC Berkeley and I'm crazy enough to be writing a book about user testing, so look for that, so that I can help everybody become extraordinary at testing new ideas with users.

[00:00:42] Jim: With that, let's introduce our panel. I've got Rima Campbell here. She was the VP of UX research partner for UserZoom before the UserTesting merge, and now is the VP of experience research and strategy at UserTesting. Before this she was at Citi for 16 years, where she built the global user research center of excellence from the ground up. Her last role at Citi was SVP of user experience in FX[?]. So give a hand for Rima.

[00:01:13] Jim: Good, you're all in order of my sheet here. Ryan Leffel is the head of design at Priceline, has held multiple roles including one at Yahoo as well as at an agency, R/GA, as the interaction design director. What drives Ryan most is his learning from taking an entrepreneurial career pivot and running his own business, and it still shapes the way he works today. Making startups, your own business, is very hard. Ryan's also the UXDX New York ambassador, so we're here in his home ground. So applause for Ryan, thank you.

[00:01:48] Jim: And then we have Fahad Osmani, who's recently taken on the role as VP of product design at Capital One. Prior to that he was at Splunk as head of design, which as a former engineer I'm a big fan of log analysis. And he was over eight years at IBM where he held positions including director of design, data and AI applications. So a hand for Fahad.

[00:02:11] Jim: So let's get a sense of you, the audience. How many of you would self-describe as a designer? Awesome. How many of you would self-describe as a user researcher? Okay. How about a product manager? Okay, excellent. How about an engineer, any engineers in the audience? All right, those are my people, fantastic. What about managers or executives? Okay, great, welcome. Any other roles? I didn't want to leave you out, but we'll just be other. Design ops there we go, HR, fantastic, I love it.

[00:02:46] Jim: Now to switch tone here. How many folks here — this is sort of getting to the heart of the panel — are losing executive backing, or have had recent problems winning executive backing for design-led activities? Anybody here? Yeah, it's going to be a little bit of a tough time. So this panel is going to help you think about ways that you can elevate design and research and other design-led activities at your company, and green-light the resources you need to make impactful products. You can submit questions to the barcode, so we'll check those in a couple of minutes, so please check that out.

What is happening with research and design-led growth today

[00:03:25] Jim: Let's start off the panel. The first question is, what is happening with user research and design-led growth at companies today? So I'm going to have Rima kind of take that question and we'll go from there.

[00:03:35] Rima: Sure, I'll give it a try. I think right now every company is trying to become a software company, and what's happening, some companies don't know how to act like a software company. They got too big, too many processes. And to be honest, we all know that more than 50% of code is going to waste because it's not meeting user needs, it's not built based on user needs and market opportunities.

[00:04:09] Rima: We also know, based on a lot of analysis that McKinsey has done, that there is, statistically speaking, companies who know how to be a software company and they are the digital leaders in the industry, as we all know, without mentioning who they are — you know, Apple and Sony and others. But also they are the minority, they represent only 20% of those companies. The other 80% are the ones who are struggling and they're trying to act like a software company.

[00:04:49] Rima: So what's happening really now is that they're trying to restructure, they're trying to reorg, because all their products and projects are definitely building that code and it is going to waste. So they're trying to restructure, they're trying to reorg and figure out how to build better product, and with efficiency and with speed to market. And with that, naturally, UX research is getting cut. Budget for UX research is getting cut. Not all companies, but some companies. So that's what I'm seeing, and I believe the reason behind that is UX researchers, or UXers, are not becoming business partners to executives. They're not speaking the executive language, they're not tying their work to the business KPIs.

Changing the perception: pivot to outcomes and metrics

[00:05:45] Jim: Yeah, thanks for that perspective. As we think about this perception with executives, what is a way we can change that perception about research and design-led activities that you mentioned?

[00:06:00] Fahad: Well, I'll sort of dial back a little bit. In my past experience, mostly my time in design leadership has spanned about four different companies, so it's a select group. I'm talking anecdote rather than anything systemic over here. But I feel like we started with this point where design thinking, design-led growth, used to be a kind of "if you build it they will come" type of ethos at the company, and now what I'm hearing a lot more of is the sort of product-led growth, sales-led growth, user-led growth.

[00:06:32] Fahad: So pivoting to those outcomes and understanding what the metrics for success are, like how do you define the problem with numbers, how do you define the success state, again with numbers, getting involved in that conversation, shaping the problem statement, shaping the end success state, is a much, much more successful way, in my experience, to be in the conversation and shape design-led outcomes, as we used to call them before, as opposed to following the Field of Dreams approach, that if you just follow the design-led approach then good outcomes will happen.

[00:07:05] Fahad: So I'm finding that a lot more engagement in, as you said, the metrics and defining the states of the problem — what's the burning platform, what's the good thing you'll get at the end of it if you choose to engage in the solution — that's turning up a lot more success. And success doesn't necessarily mean adding to your budget or adding to your headcount, it just means producing that outcome in successful products.

[00:07:27] Jim: Yes, absolutely. Anybody want to add to that?

[00:07:30] Ryan: No, I think those are all really good points. One thing I would add is, I think there's also generally this idea of let's just educate and talk about it, and I think actually being able to show the value is really the best way to sell something through and make a difference. And I think nowadays it's really forgetting about what you know and thinking about how you could be more creative in your process to get things done quicker.

[00:07:57] Jim: For the audience, the majority of you said, hey, I'm a designer. How many of you have access to data and analytics at your fingertips? That's great. How many of you use it, like, once a month? Okay. How about once a week? All right, that's great. I think this is what we're getting at here, is this perception of attaching our activities to the success of the business, and how can we connect those dots.

Over-indexing on quantitative data

[00:08:17] Jim: But there is another side to this. Companies that are fully into optimization that really are maybe over-indexing on quantitative. So is there a story about maybe being too reliant on quantitative data, where quantitative is coming in and maybe stepping on all of the other research that we know about? Any stories in this area?

[00:08:52] Ryan: I think it's something that certainly exists at a lot of companies. I think in some ways it's easier to be dependent on quantitative, it's data and it's numbers. I think one of the problems that comes out of that is it could be easy for people — I'm speaking generally here — but it could be easy to look at numbers that come back from an experiment and make a hypothesis about what those numbers mean without actually really knowing what those numbers mean. I read something really interesting the other day and it said something along the lines of, you could be data driven but that doesn't mean you're data informed. And I thought that, to me, that really resonated, and I think in a lot of cases that could be very true.

[00:09:33] Jim: You just data drive right over the cliff.

[00:09:38] Ryan: Yeah, it's true, don't pay attention to the reality that's in front of you.

[00:09:42] Jim: As long as you're engaging with the data you must be headed somewhere good, right? Just keep looking down at your phone, you're not going to run into the street here.

[00:09:50] Fahad: Totally. A couple of things come to mind. One is, you can draw qualitative conclusions from quantitative data, so it's not about just doing a different type of research, it's actually about having someone who understands human behavior look at and engage with the data. And I think the other thing is that there's always a role for the perfect user quote, or a video of someone struggling to complete a task, that brings the point home. I personally used this in a number of projects in my past, where the sort of individual page or component level optimization data is telling us one thing, and then you actually see the user attempt to complete a task and they give up after 30 minutes. There's not a lot of arguing with the visible frustration on the screen.

[00:10:43] Fahad: So I think informing your audience with what's really going on with their users is sometimes more than the data, but can also be found in the data. It just requires appetite to engage with it. And I was really pleased to see the number of hands go up when you asked about how many people have access to the data. I bet you more people than the ones that raised their hands actually have access to the data. You might not be aware of it, you might not have, I don't know, made friends yet with your partners in product management or the data ops team or other parts of the organization, but the data is out there to engage with, and it's pretty rich in turning out what I think can be qualitative conclusions as well.

[00:11:21] Rima: I do agree, however I would argue sometimes, because executives love data and statistical data, they will rely on it 100% and they won't look at the qualitative, as you mentioned earlier, because they don't know what the qualitative is going to add to it. If it's working and it's significantly high, they will rely on it. The problem is they're never going to know why the user is behaving the way they're behaving.

[00:11:56] Rima: A/B testing is becoming a little bit more of, let's just quickly put two concepts and throw it out there and let it be tested. Problem is, 90% of the time we don't know why A behaved better than B. We don't know even if A and B are built based on users' intent, even. So we don't know why they're behaving that way. So qualitative is really crucial, to your point earlier.

[00:12:29] Jim: Yeah, I think combining the two is powerful. I think qualitative research maybe can sometimes be hard to be taken more seriously if you don't have that hard data, but then the overreliance on the hard data leaves you blind as to why.

[00:12:44] Ryan: Just real quick on that: in either case, whether it's quant or qual, the data that you get back is only going to be as good as the questions that you're asking.

[00:12:53] Jim: True.

[00:12:54] Ryan: And I think that that's really important. Understanding data, and you talk about how to get to the quantitative data, that's one thing, but the more important piece of that is knowing what questions you're asking of the quantitative data, right? Because once you're asking the right questions then you can figure out how to find it.

[00:13:10] Jim: Yeah, nothing's more true than asking questions of quantitative data and then really getting back more questions instead of answers. It's usually like three to six months, so those of you getting into data analysis, just know that the first couple of months are pretty depressing, and then you will get into the part where you're getting value. Just keep going.

Which metrics actually demonstrate the value of design

[00:13:27] Jim: So let's go to the audience. Got a great question here: in your experience, what's the best way to measure and demonstrate the value of design to non-design senior leaders? What are your most effective key metrics? Are they process metrics, the number of people, this and that? Is it metrics of the product, like revenue up, cost down? What are some key metrics you've used?

[00:13:50] Rima: I would say task success completion. Task success completion, that's the number one metric that will give you an indication that this design is working.

[00:14:04] Jim: Yeah, I know a great platform where you can measure that right now.

[00:14:06] Rima: We do. We do have many other metrics. I mean, we heard Richard also listing all the metrics, definitely. And we know executives love NPS score, right? So sometimes you have to kind of figure out how that task success is impacting the NPS score. We have QXscore, which is a combination of behavioral and attitudinal data as well, that also resonates very well because it incorporated the NPS score in it.

[00:14:35] Rima: And I think it's really important that you measure the performance of the design from a qualitative and quantitative as well, and the combination of the behavioral and attitudinal data, so that you have a better understanding of which tasks really need to be improved that tie to the bottom line, to the business KPI, like maybe your checkout process.

[00:15:00] Fahad: All right, I've always found more success in top line metrics than bottom line metrics. Maybe it's just a slight variation on what you were talking about, but what I really love to go for as my default, if it's applicable in whatever context I'm in, is time to value. Sometimes it is task completion, but for our teams it's also identifying which tasks are value bearing. Time to value for me to complete it, like something I'm doing, what's the value at the end of the task? So as opposed to just completing tasks and activities, actually engaging in the discussion of which are the valuable tasks that we need to optimize.

[00:15:36] Fahad: Because you can identify an endless set of activities, tasks, jobs to be done in any given system, and for a design team to actually engage with the conversation of which ones are the ones that are most valuable to the goals of the company is actually a really important part. Otherwise you're just firing blind on a number of things.

[00:15:56] Jim: Totally screws up your prioritization process, for sure.

[00:16:00] Fahad: Yeah, it just becomes such a target-rich environment that you feel like, well, I can always be optimizing any task. But if you can bring to the table, these three are the most valuable because they help, let's say, complete the registration process and registration is really important, as opposed to updating my profile picture — one of these is a lot more valuable to the business than the other.

[00:16:19] Jim: Yeah, I keep thinking login. Login is like the left behind sibling there, where we're all just stuck trying to remember how to log in, and those few sites that have made it fast, I'm just so thankful. So for those of you working on login, thank you.

Regaining trust after research fails to deliver

[00:16:36] Jim: Okay, how do you regain executive trust in UX research after a research initiative failed to deliver on its promise? And this is why I always tell my teams, buy yourself some time before that sprint start date, because you may have a research project that does not validate. So how do you recover from this, how do you explain that? I mean, in Lean Startup they talk about innovation accounting. Well, 15 years later nobody uses the word innovation accounting, so Eric Ries failed in explaining it. How would you do it?

[00:17:09] Ryan: I mean, I think any discipline is prone to failure, research is no different. I think failing fast and learning and figuring out what went wrong and pivoting and moving on to the next thing is the best way to get past it. I think if you look at it like, we failed —

[00:17:26] Jim: But as an executive, like, you're an executive and this happens amongst your teams and your fellow executives are thinking, well, Ryan's wasting all this time, he keeps failing. What would you say to them? Put you on the spot.

[00:17:40] Ryan: Yeah, I'd be okay with that. I mean, I think —

[00:17:43] Jim: But your buddies, like the head of sales or the other folks, I don't know, I feel like some non-design people aren't bought into this whole process.

[00:17:52] Ryan: Again, I think it's fail fast and understand what you could learn from the failure. Failing, I think a lot of times people see it as a bad thing, it's negative, we lost, we failed. At the end of the day I think innovation and the best ideas and the most successful products come because something failed along the way, and it probably failed multiple times. So I think as long as you recognize quickly that something failed and you're able to admit it and own up to it and look at it from the perspective of, what went wrong, what could we learn from this, and how do we need to adapt or pivot and keep on moving forward, I honestly think that's the best way to gain that trust. It's just not dwelling on it.

[00:18:34] Jim: And that's good. Well, we'll bring you into all of our rooms when we're explaining it. I love that.

[00:18:38] Fahad: I don't think it's a binary state of success or failure for a lot of these projects. I think you always learn something. You may not have learned or confirmed or validated whatever hypothesis you originally set out to, but there's always some interesting learnings out of these projects, and positioning those as value alongside, perhaps there's still an open question on this that we originally set out to solve, is important. Because otherwise, historically, those will get reframed as, this was wasted effort. And it wasn't wasted effort. It maybe failed to confirm or illuminate the hypothesis, but it probably gave you a lot of other interesting insights for alternative paths, or for what you were looking for.

[00:19:11] Jim: I also think, as long as you're failing in new and different ways each time, it's okay. The problem is if you fail the same way each time. It's actually probably one of the safest ways to fail.

[00:19:29] Fahad: Yeah, exactly, in the design phase. You didn't involve three engineers for three months, 10 engineers for 10 months. If a checkout button doesn't work, that sucks. If you lose a deal in sales, you might not get that one back. If you fail during research, you have time to quickly recover from it.

[00:19:45] Jim: Nice. See, research is not that expensive.

[00:19:48] Rima: Yes, you've got to do all these experimentations. To your point, it's a data point, it's an important data point. And to your point, Ryan, it's better to fail at the design process versus failing when you launch the product and then it's a disaster to recover from that afterwards. So have that stage gate moment where you're actually validating — and I don't want to say validating, that's a bad word. Testing, testing.

[00:20:19] Jim: Illuminating.

[00:20:20] Rima: Yeah, exactly, that's better.

When business KPIs conflict with user needs

[00:20:22] Jim: All right, we're going to go to a spicy question here. I love this because I've been encountering this particular problem as just a consumer. What do you do when the business KPIs, revenue, conflict with the user needs? I was checking out at a hospitality website that shall not be named, and the cost of the hotel itself was, I don't know, $500 for whatever nights, and the cleaning fee was $465, shown to me on the last page, and I'm like, oh no, I'm not going to hit yes on that. So adding ads to the product, adding extra fees, it makes revenue but it could be an anti-pattern. How do you as leaders in these organizations combat this desire to really squeeze?

[00:21:08] Fahad: I always think of the sort of influence vectors towards addressing user needs, user challenges. There's OKRs, business metrics, and then there's user pain points, and if you're really lucky there's a big overlap between those two, because theoretically your OKR should be about, well, the user is one path to success and therefore we need to take care of them. There's probably some unaddressed, non-overlapping portion where the user needs or user pain points are outside of the OKRs.

[00:21:36] Fahad: I try to look at those as a mining challenge. Could I weave those back into the OKRs? I need to do a more effective job engaging in the discussion when the OKRs are being formed, so that enough of those are being represented as business goals. And then I have to be intellectually honest about, some subset of them are not going to be tied to business goals, and therefore I have to question how much of that problem should I take on, versus just say there's some unaddressed user needs that are not going to be important business problems, and we have to put them aside for now. Maybe we'll come back to them next year.

[00:22:11] Jim: Let's do one final comment and then we'll let folks get to lunch, if anybody has one.

[00:22:14] Ryan: I agree. I would ask questions. I think if you're a designer or a researcher your job is to ask questions and try to understand why those are the KPIs you're shooting after, and maybe help executives figure out what problem they should be trying to solve. Maybe there's a problem of actually trying to address a problem that's slightly different than the problem you actually need to solve, and I think that's where a design and research team could also come in in a useful way.

[00:22:42] Rima: Yeah, if you have deposited that credibility with the executive you can have that conversation, for sure. 100%.

[00:22:51] Jim: I love depositing the credibility in order to withdraw when you need it.

[00:22:53] Rima: Exactly.

[00:22:54] Jim: I love that. All right, a big round of applause for our panelists. Thank you.