Enhancing the Impact of User Research in Enterprise Software

Fahad Osmani at UXDX EMEA. Video: https://youtu.be/VdIXP73iBcw

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

From software development to design leadership

[00:00:00] Thanks, Kevin. I'm the only person standing between you all and lunch. Stick with me here, I'm going to keep it interesting, we'll get you on your way, and we'll carry the conversation over lunch. I'm going to do a little bit of an intro just because it's relevant to the topic of my talk. My name is Fahad Osmani. I live in Austin, Texas, in the United States, and as they say in Texas, I wasn't born there but I got there as soon as I could. I work at Splunk, which is a machine data analytics and monitoring company. It's right in the center of enterprise software, which is the focus of the talk.

[00:00:32] I'm myself a little bit late to design leadership. My career started in software development. I have a degree in computer science. I got started really in the late 90s at the peak of the dot-com boom, and I moved quickly into technical consulting, which meant I did the same job as a software developer but I put on a suit, got on a plane and did it at the client site. By the way, if any of you are considering a career, or have had a career, in consulting or software services, here's what it looks like. You know the name of the flight crew of the flight that you take every week. You know the names of the hotel staff. You leave yourself weird notes in the bathroom mirror to remind yourself, this is the city that I'm in, here's the client that I'm servicing. It's all fun in your 20s and 30s. It wasn't for me after that, later on, so I shifted over into a different part of my career.

[00:01:29] But what it really taught me was that enterprise software is a different space. There are different dynamics that happen. Every company that I visited was making a different tool, trying to optimize software for a different type of productive use case, and I really got to spend a ton of time with those users. That was part of my job. I was co-creating the solution that I would build with them. I was sitting next to them for sometimes thousands of hours. I would spend months on these projects with them. I didn't have the vocabulary for it at that time. I wasn't in a design role, I was in a technical role, but what I was essentially doing was field research. So it gave me an idea about what that segment of our industry looks like and what those users want.

[00:02:11] Then I also shifted over into design leadership at IBM, where I took part in the recent revitalization of design in the 2010s. Now, all that to say, here I am after this short stint talking to you like I know what I'm doing over here. But it gave me a really good idea of enterprise software. It gave me a good idea of what it takes to talk to these users, because I was one of the influence vectors that could get you to them. So I want to share some insights about what some of the unique challenges of user research insights are, and also about leveraging user research in enterprise software.

The weird world of enterprise software

[00:02:49] Let's take a step back. What is the weird world of enterprise software? Isn't most software of the personal convenience type, software that you use to do e-commerce, communication, personal life conveniences? Where would an interface like this even stand for more than a day or so before someone said, hey, we've got to change this? It can happen in enterprise software. There is a whole wonderful world of tools that you yourself might not use. If you're part of a design team or a user research team they might not be personally relevant to you, but they're part of your support network. People that you work with might be using these tools. You might recognize the names of some of these companies but you may not be using the tools that they're using.

[00:03:31] Now, this is not necessarily a bad thing. The fact that these tools have such specialized use cases and such specialized users actually yields certain advantages in some areas. So for instance, something like over 50% of all credit card transactions still happen over mainframes, mainframes that are running COBOL. If you've ever worked in a branch of software called message queuing, most of the time this is a somewhat esoteric computer science topic, or it may be in the basement lab of your company called system software. But in essence all it's trying to do is get messages in a packet over reliably from one point to another. And if you find the correct use case it can actually enable mobile banking over non-smartphones, dumb phones, burner phones, and this can enable a second or third world country to really enable mobile banking for its citizens.

[00:04:26] So this is all stuff that is important to our society, it's important to our civilization, but it's not really meant for the individual user. That's one of the key insights that I took away about enterprise software: these organizations, the tools they build, the products they build, they are not optimized for an individual user. Rather they're optimized for sets of users or whole enterprises. The goal of the enterprise and the goal of the company often trumps the goal of the user.

The user is not the buyer

[00:04:59] So what are some of those unique challenges that we can look forward to? First of all, the user is not the buyer. This is one of the most key insights that I personally took away when I started to notice the contrast between adjacent industries that are dealing with business to business, or business to consumer, or consumer software: that the user and the buyer and the customer can in fact be very different personas. The person that signs the check to use your tool is often not even going to evaluate your tool. They're going to look at your pricing structure, they're going to look at your contract values, and they're probably a very important influencer in how your organization makes money.

[00:05:42] The customer is also probably not going to use the tool that you're building. They're going to look at a menu-like feature set that says, okay, you do this, your competitors do this. They're probably going to read some kind of analyst report that shows, okay, well, you are a visionary in this quadrant or you are a competitor in this quadrant. And also, the rest of my IT organization already uses tools from your company, so it's highly advantageous to me to use another tool from your company because the theory is they're all interoperable.

[00:06:11] By the time you get to the user, the actual person with their hands on the keyboard, their opinion about the usability of this, the volume is lowered. I'm not going to say it doesn't matter, because it does. They hold influence over those purchasing decisions. But the volume of the influence that they have over the economic transaction is much lower than it might be in a consumer setting. And in fact, that user probably didn't want the tool that ended up getting chosen. They wanted its cooler cousin, the one that has all of the free bells and whistles, the startup that's just putting out amazing new features. They probably don't want to deal with all of the extra burden of security, of reliability, of compliance, and they certainly don't want all the interoperability with the other tools that they hate using. So you're probably coming into a situation in which there's an existing resistance to using the tool that your company has sold them.

[00:07:09] And also, for most enterprise software the use cases that it's meant for are quite pervasive, so each individual user might only use 10% of the product experience that you're building for. I've actually worked with customers and users, both for my prior companies and also in my consulting days, in which someone pointed out, all I do is create this report and I use the alert button. That's bare minimum what the tool is supposed to do. There's tons of other paths through the tools, but that's all that that user needs. So for most organizational use cases they're going to use 10 or 20% of what you think you're building for, but their 10 to 20% is going to be different than some other organization's 10 to 20%. Good luck figuring all that out, what you should be optimizing for.

[00:07:57] Now, this may seem like semantics, but once you remove economic power, or once you lower the volume on economic power from the user, you really run into a lot of different dynamics. So you start to optimize for things like, is your pricing structure correct? That could be just as much a part of the user that you're targeting, because they're going to be influenced by the buyer, the person who's signing the check. Or, am I doing the right thing to grab interest from the analyst community? That might hold huge influence for your customer. So you start to care about things like the Gartner Magic Quadrant at that point.

Expert users are hard to get to

[00:08:36] Another challenge over here. When you're dealing with enterprise software, mostly the IT variant, you're dealing with expert users. Which means that when you're dealing with high expert users, super high information density and comfort with that, you are no longer a reasonable proxy for these users. You are likely not going to be in a job where you're going to be evaluating site outages, you're going to be evaluating fraud metrics. Yes, you're building for those users, but you yourself are not a reasonable proxy, nor is anyone that you casually know. You really have to go outside your network to find these individuals because of their scarcity.

[00:09:16] Also, a lot of these expert users are already compensated well enough. They're not really looking to fill out surveys as a way to supplement anything. So you've got this additional barrier to entry. All of the usual surveys and diary studies that you might do, those have an additional level of difficulty that you're going to get on to. And also, if you compound this with the first factor, the fact that they're probably using an unknown 10% of your software, something that you probably didn't know about but it's probably the most popular part with your users, you now have to get an additional sample size in order to make sure you are properly tuning your experience, in order to bring out the maximum level of value for the users you want to target.

[00:09:55] And then finally, because the expert users at any company might have reduced influence in the economic transaction, but they do have influence in the buyer's mind of, are my users actually adopting it, your sales team doesn't want you talking to these individuals, at least not without going through them. There's always a deal at stake, there's always some customer escalation, someone's always going to be asking, hey, do I really need to bring extra people from my organization into this conversation? The customer's at a really, really sensitive point. The last thing they want is someone from the design team or the user research team showing up saying, what kind of job do you do, what kind of tools do you use for it, does my tool actually fulfill those obligations? Well, we have an escalation going on to evaluate exactly that. We don't need anyone else coming in with existential questions at that point. So you've got this additional barrier to entry to get hold of these individuals.

Decisions first, data later

[00:10:48] Now, this is one of my favorite quotes. I'm going to leave the full version to your imagination, or for you to complete in your head or via Google, but the short version is that in the world of enterprise software, confident decisions come first and then sometimes data comes later to back those up, even if it's invited at all to the conversation at that point. And that's not all due to any bad faith decision-making, just to avoid cynicism. There's a lot of paths to economic success. Sometimes, as I said before, you have to focus on or prioritize for the analyst community. You have to prioritize for what your pricing structure does, is the buyer happy.

[00:11:35] And so this can make user research feel pretty challenging. A lot of times these additional dynamics that you have with optimizing for a lot of different things, things that can make you feel like the voice of the user is getting lowered in the conversation, they can actually end up creating extra runway for you, which can be advantageous at times. So for instance, if the economy is not doing so well, but because of certain optimizations you've made specifically for showing up well against your competitors and the analyst community, or making things easier for your buyers in terms of pricing structure, they can buy you additional time. Time that you can use in order to think about what's really important to your users. Time that you may not have if the only thing that was important was user sentiment. You might have a really short runway at that point, you might have to figure things out in a month or two. But sometimes these additional dynamics can give you some runway to figure out what's right for your user.

The technical marvel of the year

[00:12:32] But sometimes when you take that extra time you can run into a different sort of problem in enterprise software, which is that you can run into the technical marvel of the year, or of the decade. Sometimes tech itself can be a center of gravity. We have a saying in tech: this is going to be like a sandwich in search of a picnic. You've already got the technology, now let's figure out what use we want to make of it. Let's not start with a use case, let's start with the tech. We heard about this at the beginning of the conference today, where the tech itself can seem so glamorous and full of promise that finding the proper use case for it is a secondary thought.

[00:13:14] I'm sure all of us have heard, we need some AI in our roadmap. Well, what's the user value we're trying to create? What additional capabilities are we trying to unlock? I don't know. You can have a ton of user research for the audience that you're trying to target, what's really important to them. You can negotiate everything with your engineering teams, make sure that all of the top pain points for your users, or the top value that you want to deliver to them, is all possible within six months. It obeys the laws of physics, your engineers can deliver it, your product management team agrees this actually gets to the total addressable market, this gets to the serviceable addressable market. And at the end of it the answer might still be, we need some more AI in here. And don't get me wrong, these technologies have immense promise, but our nature makes us believe that proximity to the technology will yield intrinsic value, just through osmosis, like being near an influential person will make you influential just via proxy.

Amplifying a culture of user research

[00:14:13] So how do you try and solve for some of these challenges intrinsic to enterprise software? Well, I have some strategies that have worked for me in the past, but what I'm really curious about is learning more about how others have solved them too, which is why I hope our conversation continues after this. The first thing you can do on the first challenge, where the buyer is not the same one as the user, what's worked for me is really amplifying a culture of user research rather than just paying primary importance to the role of the user researcher. So, inviting in other voices into the conversation. If you're dealing with enterprise IT software, your developers probably know a whole lot about what's important to them. What tools do they use, what's their workflow, where are they already working, where do they want to be met in that space? Your product manager that's dealing with customers all day, they probably have a great idea of what pricing strategy works, what customer escalations they're mostly dealing with.

[00:15:09] Now I want to caution you over here to not over-rotate on this, because the flip side of inviting too many people into the conversation without role clarity, without actually saying, well, user research is actually a vocationally distinct and unique discipline, and while everyone can bring their own data we actually need an expert to synthesize the opinion, is that the plural of anecdotes becomes research. I talked to a customer last week, therefore we should do this in our roadmap. You've got to have some guidelines against that. You're going to have to thread the needle between inviting other voices in and also making sure that synthesis, insights and recommendations stay within people that are trained to look at those things, trained to look at validity, trained to look at real insights and jobs to be done rather than simple pain points that one person might have told you about.

Sellers and consultants are your best friends

[00:15:59] Now, the other thing to keep in mind: the sellers and the consultants on your team, they're your best friends. You should go out and meet with them. The reason why you're not getting invited into a lot of these conversations with your expert users is that your sales team or your professional services team doesn't know you yet as a partner. You've got to be able to build a relationship with different parts of your organization that are already connected to the customer. What can you offer them? So for instance, in my previous roles I've offered to put on design thinking workshops for our professional services team as they deal with their customer, to uncover more needs, help them get to better value in their use cases. I've offered to help sellers polish their demo, make sure that it really shows off the capabilities of the product. There's something of value on both sides of this.

[00:16:47] If you are good at this, your sellers, your professional services people, they'll invite you into conversations, they'll invite you into a site visit with your customers. You'll be in the meeting with them. If you're lucky there might be a user at that meeting. If you're really lucky you get to ask to meet their users while you're there on site. There's nothing better than just building that trust, following up with a customer, coming in as part of your go-to-market conversations. Now, remember they're not under any intrinsic obligation to help you with this, so you've got to stay 100% on message with them. The moment you start talking about, ah, the next version is a lot better, you should probably wait for that, that's when the invites will stop coming in. No one wants to hear that. They want to help you close whatever business they're wanting to close. So make sure you know about that customer situation, make sure you stay on the right message with them.

Playing back the voice of the user

[00:17:40] Now, while you're there you might, as I said, meet some users. You might want to write down some quotes from them about what the top pain points are, what are you using the software for. You can even try and record some of their interactions with the product that you're using. This might be a unique opportunity that you might not otherwise get: an actual client customer user rather than an anonymous user. What are you going to do with those quotes and with those recordings? You're going to play them back at the beginning of every meeting, at the beginning of every scrum, at the beginning of every critique. It's super painful but it's effective. Use the voice of the user, whether it's a recording or a quote or a seller story.

[00:18:22] And sometimes, if you're lucky, you can even use this dynamic to activate some pride of ownership in your team. I'll do a little anecdote over here. I declared a somewhat arbitrary goal for one of our product teams, where eight out of 10 users, typical users, new users, had to be able to finish a task in 10 minutes or less, and we started measuring that as a goal with each release. At the beginning it's super painful. You see all these users kind of stumbling around, not really knowing how to get to the end of the goal, and it really brought to life what it is that the user is trying to do and what value we're trying to impart to them.

[00:19:01] But over time, as you saw that number trend towards a lower and lower value, you could see it almost became like a team event. People wanted to see that line go below 10 minutes, and when it finally did, huge moment of celebration for the team. We all cheered, went out for a team event after that. But it really brought the team together, not just the design team but the product team and the engineering team, towards a shared goal, because we could see our users' frustrations and we could connect with it in a real way, in a way where the anecdotes overrode what the user was actually going through before our eyes.

Ask why about the shiny tech

[00:19:38] And sometimes, whenever you're faced with the question of the tech being its own center of gravity, one of the things I really want to encourage you all to do is ask why. This is going to sound a little bit Deus[?] at the end, but don't give in to despair about, yes, we've just got to put in this new technology because someone's already made this decision. Yeah, you can't always tilt against windmills to prevent the adoption of something that doesn't have a clear use case, but you can sure try to find an effective use case for it.

[00:20:12] One of the things I try to do is put most of the experimental technology that we're all being asked to work with in different buckets of value rather than just thinking about it as tech. So for instance, when we think about AI I think of it as solving primarily automation problems, things that were previously manual and difficult to get to, user insights, insights from your data. When you think about different AI technologies like speech to text, text to speech, natural language understanding, natural language processing, and more lately natural language generation, these are all things that you can find different use cases for your users for. When I think of blockchain or hyperledger I usually think of a trust issue. You've got different users in a marketplace, you want to connect them directly, when the barrier to entry for a lot of users of going through a third party is too high. It can solve that barrier to entry problem, it can solve that trust problem. When I think of quantum computing, for instance, I think of brute force tech, algorithmic brute force. These are things you can use to solve security issues, you can also use it to solve for clinical trials, building financial models.

[00:21:20] So at the end of the day, what I really want to leave you with is that there are some unique challenges in enterprise software. These challenges come with their own advantages. They also come with an opportunity to grow some unique skills about how to better build a network within your organization, get to know more people. And getting hold of the users and figuring out what to do with their insights is a multivaried problem. You're going to have to think about not just the user's insights but the buyer, the consumer, at the end of all of this. So I hope this has been informative. I'm eager to hear from some of you that have already encountered some of these problems out there, and for those of you that haven't, that might consider a career in enterprise software, hopefully this has given you some headlights into it. I'll stop here and open it up for questions. Thank you.

Q&A

[00:22:11] Host: I have plenty of questions on my phone as well, so let's go with this one from Alessandro. Can you mention some example or story of how you were able to be involved in the sales cycle, which technique you leveraged to get in touch with the customer, users, buyers, and how you leveraged it?

[00:22:25] Fahad: Awesome. Okay, so this is a great question because I've been on both sides of this. I have been the person who's going out to meet with customers as part of the professional services organization, sometimes helping sales, and I've had designers, when I've not been in the design team, reach out to me for, hey, can you bring me along for a call, to ride along or a side car. And usually my response is there's just too many people in the room, unless that person says, okay, here's something I can do in order to move the conversation along.

[00:22:53] One of my favorite stories is that I recently started working with the sales and professional services team at my current organization, and what we've started to do is offer a facilitated design thinking workshop, or in some cases, as I mentioned, help with product demos. I've actually found the sellers really want to cooperate. There's something in it for them. They like it. There's something of mutual benefit. And I find that just building that relationship, building that trust, asking them why is this customer at an escalation point, really helps with the conversation. So that's worked out for me.

[00:23:29] Host: Okay, fantastic. Both of us have worked at enterprise companies. What is something that you think is unique to trying to get stakeholders to listen or be involved directly at a scale of that size?

[00:23:38] Fahad: When you say stakeholders, you mean inside the company that you're at, or at your customers?

[00:23:42] Host: Like reaching the sales team, reaching these teams.

[00:23:45] Fahad: That's a great point. This is another thing that could probably be a whole separate talk. But when you just reach out to sellers, you can expect zero to no response, because unless you're actually offering something, their head is, they're in a quota plan, they've got to figure out what will help them get to that quota, what will help them make their customer successful. So getting stakeholders to listen is more about not only helping them transactionally in this tactical conversation, but helping them stay attuned to, hey, I noticed that you're having a lot of these conversations with customers, they all have this feedback, I am your inside person in the product team, I will make sure your voice gets heard, I will make sure the roadmap is influenced.

[00:24:32] You can't make any promises because you actually don't control the roadmap, that's for product, but you can say that you will represent the voice of the user and the voice of the seller inside that room. Quite often when we make decisions as a product team we don't have every single person in the company, from sales, professional services and support, in the room. They're often informed of the roadmap, but they would sure like to influence the roadmap earlier. You can be that person, their person on the inside of the product team. That's all been helpful for me.

[00:24:57] Host: Totally agree. Let's go with this question, the one with the most upvotes. Do you think that some of these techniques can be applied for products and tools built for internal users as well, and if so, give an example maybe.

[00:25:09] Fahad: Yeah, I definitely think so. There's a lot of, we used to use the term dogfooding before but now we call it drinking your own champagne. It just gets more and more sophisticated over time. But one of the things that's really, I guess, an unfair advantage in enterprise software is that your developers are probably your first user. So you're always going to have access to a set of people that can try an internal version of this first. And you can always evaluate, what are you using some of these internal tools for, what kind of use cases are you trying to tackle that I could actually bring back into the product roadmap. We've actually found that for a lot of our cloud ops people, our own SREs, they're using essentially homegrown tools that we use as inspiration to bring into our own roadmap.

[00:25:56] Host: Okay, wonderful, nice. Let's go to the next one. Do you have any science fiction book recommendations that somehow overlap with UX and/or research topics of your talk?

[00:26:03] Fahad: Oh interesting. Oh, I want that next question though, Concur, everyone. You know, none that I can think are super super relevant to UX, I'm just fascinated by the topic. If you love post-scarcity economy, I really like Iain Banks's Culture series. If you like to see how we might bridge from current day to that post-scarcity economy, then I've really enjoyed Neal Asher's Polity books. Those are super good.

[00:26:31] Host: Nice. I was going to throw in Ender's Game, because it feels like an A/B test.

[00:26:35] Fahad: That's true, yeah, that's a great one. It gives away the ending, but yeah.

[00:26:39] Host: All right, let's go with one from Alex. What's your favorite enterprise software?

[00:26:43] Fahad: Is it, it's Concur, right? We all use Concur and I think it greatly illustrates the story of the tool that you end up using. It's probably not the tool you would have picked but you still have to use it, and I believe the team at SAP probably puts in a lot of effort into making sure that their users are actually getting to the end value they need. But as far as my favorite enterprise software tool, it's probably still Jira. I get a lot out of it, it provides just a bird's eye view of everything, where your roadmap is. Of course it depends on who's using it. A lot of companies you work at, your roadmap lives in PowerPoint, not in Jira.

[00:27:27] Host: Or Notion, or exactly, yeah, or Figma. All right, let's take another question. Can you mention some example or story of how you were able to be involved? We already did this one, right?

[00:27:39] Fahad: Did that. I talk so fast. All right. Oh, Citrix, same thing, we can skip this one.

[00:27:46] Host: Let's go with, I have questions as well, or we can just go with the third one in the list. Or I can just ask you a question. We're just going to go with it. All right. So one of the things that you and I both have in common with enterprise, my last team was 105. What is it like trying to get the entire team to use a process like this?

[00:28:11] Fahad: Oh yeah, that's a good one. One of the things that's unique to enterprise is that sometimes you're not all working on the same product. Sometimes you're actually part of a different business unit within the same company. Those dynamics are true for the last couple of places I've been. So it's actually not super important to me, at least in the context that I've been in, to get everyone to follow the same process. It's more important for them to follow a set of strategies or techniques that work within their context.

[00:28:40] When you've got these different divisions and BUs going on, you've actually got a different culture in each sales team, you've got a different culture in each different professional services team. You've got to figure out what dynamic works for them. The key common goal is: get to know your field team, understand that the buyer is different from the user, and resist the gravitational pull of shiny tech until you've figured out the use case. You might have very different ways of getting there depending on what context you're in though.

[00:29:07] Host: Fantastic. Let's give one last final round of applause for Fahad.