How To Overcome Prioritising Challenges

Vincent Feeney, John Cutler at UXDX APAC. Video: https://youtu.be/-tZOp70f5ys

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions

[00:00:00] John: Great, hello everyone. That was an amazing talk. I'm not sure always how to pronounce your first name. It's "boujeens" [?], right? Is that close?

[00:00:10] Speaker 1: No, it's not, John, that's okay. You artists. Silent J, silent J. But five points for trying.

[00:00:16] John: Five points for trying, five points for trying. Okay, hello Vinnie, how's it going?

[00:00:21] Vinnie: Hey John, very well, thank you. Thanks for hosting us today.

[00:00:25] John: Yeah, I'm excited to moderate this. The general summary for today is how do we overcome prioritization challenges. What we're going to try to dig into is specific examples of how, in your organization, you approach prioritization. Obviously prioritization is not just one capability. It's also a combination of research, it's a combination of communication, as we just saw, it's a combination of strategies. There's many things involved in that. We're going to rotate through the panelists and get a few prioritization frameworks or approaches as we go through. Very briefly, Vinnie, would you mind introducing yourself?

[00:01:11] Vinnie: Yeah, absolutely. My name is Vinnie. I'm currently a product design manager at Zendesk. I'm working on the Zendesk platform. I've been there for about 18 months or so in that role, working on one of our platform teams.

[00:01:25] John: Fellow Zendesk person. I also worked at Zendesk a while ago, so there you go.

[00:01:31] Vinnie: Yeah, I didn't know that. Maybe you've got some insight into how it was a few years ago. It'll be interesting to see if it's changed.

[00:01:37] John: Yeah, we should go through that.

Where ideas come from

[00:01:42] John: Okay, so we've had some intros before, you're able to introduce yourself now, Vinnie, that's good, as we continue off that conversation. First of all, I think a lot of times people equate prioritization to prioritizing ideas. Now, we could talk about whether that's the right way to go about it, whether we should prioritize leverage points or whether we should prioritize objectives, but certainly ideas are a big part of it. There's a bunch of people who have ideas. One question would be, Vinnie, maybe you could start: where do ideas come from in your organization, where do they spring from, and what do you do with them when they come up? Maybe you could talk through where the ideas come from.

[00:02:26] Vinnie: Yeah, that's a great question. I definitely think there's a lot of challenges with prioritizing things. I don't think a shortage of ideas is ever the problem. We have plenty of different feedback rolling in through different channels. If I think of things like our customers, they're telling us things every day, whether that's to the design team in a user interview, whether it's to our sales team, whether it's to our advocacy team who are doing product support. And product managers have no shortage of ideas, engineers no shortage of ideas. We tend to see ideas come from everywhere, whether it's internally or externally.

[00:03:02] I think you're right, though, the challenge is then, well, what do you do with all that information coming in? How do you actually make some sense of that? I think that's where it comes to formalizing some of those channels. For example, having a voice of the customer program in place, so that with all the stuff that we're hearing from our customer success teams, rather than bringing a hundred bits of different feedback to the product team, maybe you have somebody summarize that and bullet what are the key points, or what are the key things coming through that we might be able to address. And that's also then on us to actually have some kind of loop in place where we can take in that feedback and respond to it as well. I would say the ideas just come from everywhere in the organization, and that could be inside, but then outside with our customers and partners as well.

[00:03:50] John: Nice. In some ways I would like to think that ideas are data in their own way. They're encapsulated insights from people. Often people express it as an idea, but there's something behind it. Flipping over to Microsoft, maybe a big different situation than Zendesk, but where do ideas come from? How do you process them, how do you extract insights maybe from them? Where do they come from?

[00:04:22] Speaker 1: Yeah, in an incredibly big organization that has many other organizations in it. For anyone who's interested, my version of how I see Microsoft is: there is the product group, there's Microsoft Research, MSR, and then there's what is like the customer-facing aspects of Microsoft, which is the world in which I work and occupy. Similar to Zendesk, the product group would have a lot of user feedback forums, a very mature design and research approach with potential people who are going to be part of product experiences. For Microsoft Research, we're talking about people for whom it's their day-in, day-out existence to have ideas.

[00:05:11] When it comes to customer-facing work, ultimately the customer has come to us with the motivation, to use some of the language from the presentation, with the motivation to achieve something specific for them. In doing that, we see different qualities from that customer. Are they really mature in terms of the idea that they have come to us with, or are they looking for some kind of leadership, or less leadership but a sounding board, or guidance on how to go about exploring it further so that it gets to the point of maturity that it needs to be at?

[00:05:54] I would say that in that situation, most definitely the inception of the idea has come from the customer. What has influenced them in that idea is a myriad of things. There is the entire milieu that we all live in, them being exposed to advertisements or to ideas within their own industry. And then it's a matter of, at a project level, how we engage with them to work that project, that idea, through to whatever stage it needs to be matured to.

[00:06:23] John: Right. I love that description, that it's not as simple as "I've got an idea, we can build that." There's probably some folks listening where in their particular organization it is the battle of ideas, who gets the first best pitch or not. But as you can see, in an organization like Microsoft or Zendesk, ideas exist in this whole other context of teams that are working with it. That's interesting.

Prioritization frameworks

[00:06:51] John: As we're thinking about that, I could go either way about frameworks, in the sense that a lot of people believe that when it comes to prioritization, they'll Google prioritization frameworks, and up will pop one of these five frameworks, and then suddenly you'll figure it out and be able to prioritize all these ideas. I'm skeptical, having been on the ground on teams, that the frameworks provide that level of certainty. But I do think that they can provide a level of rigor to your thinking and, more importantly, probably get the right conversations to happen. I was wondering, Vinnie, at Zendesk at the moment, do you use any specific frameworks for prioritization? If so, do you modify them in any way? And how do you make sure that the framework works for you, instead of you working for the framework?

[00:07:44] Vinnie: Yeah, I think it's a really good question. Certainly there's a few frameworks that we use at Zendesk. I can also recall the last company I worked for, a large accounting software company, and we used the cost of delay framework. I don't know if anyone's familiar with that framework; we used to use it a lot. What I saw, especially initially, was people would present things and they would present their cost of delay: "We need to do feature A over feature B, because look at this cost of delay." Then when you get into the weeds, people are just gaming the system to make their pet project win. That's something I think about with those frameworks as well. It's great to put numbers on things, and people love assigning numerical values to things, but it doesn't always make sense, and it's pretty easy to fudge some of those numbers.

[00:08:32] Our approach today, so that's something I've been exposed to. From a higher level in the company, there's certainly things like OKRs that we use. They very much come from the top down and they're set at that organization level. In other companies I've worked in, those OKRs cascade the whole way down into individual teams. I would say for us it's not to the same effect at Zendesk. We have these broader OKRs that we're trying to hit as a company, but within our product teams we've got a bit more flexibility in terms of how we're actually going to hit those metrics. In terms of frameworks within different teams, you will see slight variations of frameworks used.

[00:09:11] It's not an exact science. We've got product ops teams who are very much trying to standardize that across the organization so that there's some level of consistency. But we're nowhere near even the size of Microsoft, and in an organization with 5,000 employees and hundreds of product teams, it's really hard to get everybody using the exact same framework. And I don't even necessarily know that you're going to get the right outcome from everybody using the exact same framework. It's certainly something that will change team to team a little bit.

[00:09:46] John: Yeah, I love that point too about mono-process. We think that consistency is good, but it really could be that the best outcome is the best outcome for customers, and if a team is using something that works for them in that particular context, there's an argument for doing that particular thing. At Microsoft, are there any canonical frameworks that everyone has to understand like the back of their hand, or is there a ton of variation between groups when it comes to prioritization?

[00:10:23] Speaker 1: Within the area that I'm in, most definitely there is a difference. There isn't one set method. I echo much of what Vinnie has said, actually. There's not a great deal of difference to what he said within Microsoft. We're also rolling out objectives and key results at an organizational level. That has not filtered down to an individual level of "here's what you must do in order to do it." I would hope that wouldn't happen, actually, because people need autonomy, and you most definitely want to invite serendipity. You want to invite someone to creatively interpret something, because without that you will never achieve that other elusive thing of innovation, or at least doing things differently and better. What a framework at the point in time that it was created, or what a set of objectives, was aiming to achieve will inevitably be out of date at the end of, choose any time in the future.

[00:11:24] John: Yeah, I think about the conversations that happen with these frameworks when people just copy, even "Oh, we're going to do OKRs now." If you think about the context within which OKRs emerged at Google or any other organization... I talk to so many teams, and many of these things are used in a very unique way depending on the organization. There is that element too, localizing these methods for your particular context. We do have a question about gaming numbers. We'll get to the numbers in a second.

At what level do you prioritize?

[00:12:03] John: I'm wondering, maybe this is one way to help people think about prioritization: at what level do you prioritize? One could argue prioritization is happening up and down the stack. At some point someone is saying, there's that slide with the three to four pillars on it. You have to have that, and that is a form of prioritization in itself. Hopefully you don't have 17 of them, but ideally there's something there. Then you could go from that to where are teams going to spend the next 12 to 18 months to focus, a broad swath of focus, all the way down to the backlog or things like that. Vinnie, maybe you wanted to take a quick stab at that. If you work down the stack, from the pillars down to the things, when does it happen, how often does it change? I think it would help folks out.

[00:13:00] Vinnie: Yeah, absolutely. It's been interesting to be working in different organizations with different approaches to this. Again, the last organization I worked for really went through these big three-year plans. Everything was a three-year plan. Now, the three-year plan didn't mean we'd decided every roadmap item for the next three years. It was just some really big, high-level targets, maybe it was seizing the enterprise or hitting a particular revenue goal. But it was very consistent through the three years. You'd have slight variations, but it would always be through three years. There was internal branding and all that kind of stuff on the plan.

[00:13:37] At Zendesk, I think 12 months is probably the time frame for really properly reassessing everything and checking in: is this the right direction? Every 12 months there'll be a big bang, almost, and often that's lined up with product marketing as well, where we'll actually have events and talk about releases. In the product teams, though, we're very focused quarter to quarter. A lot of our rituals are focused on what's happening in the next quarter, how did the last quarter go. We'll have mid-quarterly check-ins where we're checking in mid-quarter, but we really talk about quarters a lot. When we're talking about prioritizing work and where different projects fit in, it's very much on that quarterly basis.

[00:14:25] To go back to maybe more the heart of the question, though, we will have these annual plans, and there'll be some broader OKRs for the company. Then typically, for the product team, there'll be some key focuses and asks of the product teams: this year, here are the five things that we want the product team to be focused on. Different teams will get assigned to those different areas. Within the teams, you've got a really clear goal that's been set from the higher-ups in the business, but it's really up to you then to actually start prioritizing, gathering the bits of work, prioritizing roadmaps, looking at what are we going to do in Q1 versus Q2, and, as you said, starting to talk about backlogs and managing the work at that kind of level.

[00:15:13] John: Okay, awesome. I was wondering, for you, especially with the research background, maybe we could even take a twist there too, in the sense that sometimes research is informing prioritization, but then you have prioritization happening at multiple horizons at once. As you saw how Vinnie did, if you work down the stack of the resolutions that you're thinking at, and maybe how research might be informing those particular levels, how do you think about it in your particular group at Microsoft? Is it the five-year plan all the way down to what's going on in the day-to-day, or how does it work?

[00:15:53] Speaker 1: A slightly different context, because we're working with customers on their products or their solutions or their experiences. What I typically see is an average of a one-year product plan for customers who have products in market that we're then helping them with, or we're doing a piece of work on. What's difficult in that situation is if they have a desire, while this product has a one-year commitment, to do some parallel work around an exploration phase of something that could be a game changer for that product. We need to be very mindful of the time, of the costs associated with that product that is already committed to a one-year time period. Those product managers have already settled on the people that they'll have in their team, the variety of skill sets required, the disciplines, et cetera. But at the same time they want to simultaneously run this parallel piece of work, so there's always a little bit of a tango around managing that.

[00:16:53] Where there's, let's say, an innovation group, or a group within an organization that has been carved out in order to achieve a particular new thing within that organization, so it might not actually be a product yet, in that instance we're working day-to-day. The thing that we're always aiming for is, of course, to ensure that research starts ahead of any engineering work. Even though there's some things to talk about in that, because there are fundamentals that can be put in place, so there are moments in time where you could run two tracks in parallel. You just need to be very aware to catch that moment in time, to ensure that people aren't continually continuing on those tracks in parallel. There are these moments of connection planned in, by design, throughout, rather than that default position. If you start out one way, it's very hard for people to question and get off that track. Being intentional about designing in those meeting points is a requirement in that kind of a situation.

[00:18:04] John: Yeah, I love that description too, because if you're new to product or anything, you imagine you're just force-ranking a backlog and that's the degree of prioritization. But it is really a lot more dynamic than that. You're prioritizing at different levels, and you're using research to diverge and then converge on where you're going. In your example, too, the team's established, so someone has placed a bet. They've prioritized some kind of opportunity that exists at that point, otherwise the people wouldn't necessarily be there. It's very interesting to hear about how it's happening at these different resolutions.

Closing the feedback loop

[00:18:35] John: I'm curious about the input of how things have been working back into prioritization. One thing we notice a lot at Amplitude is that people get extremely focused on using data to think about the opportunity space, and then think about the various problems that exist, and then think about how to design their solutions or their particular bet. They spend a lot of time focusing on that, and then they ship it, and then they never go back and look at it again. They're not closing the loop of prioritization back into that. That can be qualitative as well, it can be quantitative; there's many ways to keep that loop going. I was wondering, Vinnie, when you're thinking about how you prioritize at Zendesk, is it product ops, and I guess UXR and other groups, who are trying to cast the net for the qualitative feedback? But how are you inputting back into the prioritization based on how stuff is doing?

[00:19:42] Vinnie: Yeah, it's a really good question. Certainly in every team that I've worked in, it's always been a real struggle to give as much time as you really want to go back over some of that stuff again. As you said, there's this bias towards shipping the new shiny thing, and then once it's shipped, we talk about v2, v2, v2, and then v2 just never happens. It's certainly always been a challenge, and I certainly don't think we've solved it. Whenever we're releasing things, we want to be really clear about what our success metrics look like, but also what is the path to getting customer feedback.

[00:20:24] For example, right now our team's working on something which is going to be a beta release, and before we actually put that to general availability, there's a few gates that it's going to have to pass through. Therefore, when it actually becomes generally available, it's going to have satisfied a certain bar that we're really comfortable with. We're able to get that feedback from people before it's actually generally available to the population. Again, that's not perfect in itself, because what happens if it takes a really long time to get there? Maybe we just can't hit those metrics, maybe we still move on to something else. It's certainly a really tough one to be really robust on.

[00:21:00] Another thing I think is worth mentioning, something I think about a lot: we often talk MVP, MLP, the phrase of the day. I often think, what if we never came back to iterate on this thing? What if we released this into our product and we got moved on to something else? Is this something we're happy to live with for another five years? Because in reality, that's what's going to happen a lot of the time. That's not a framework, there's no number on that, but it can help in some of those conversations about whether we should do that extra little bit of work. It's like, well, look, if we put a few more weeks into this, maybe we're comfortable if we get pivoted onto something else.

[00:21:42] John: I love that too, because prioritization is happening all the time. I read recently that an engineer or a designer makes a cost-benefit decision every 15 minutes. We think of prioritization as deciding at the high level what to do, but you're prioritizing where to spend your time. You're prioritizing, like, "Oh my god, we were a week late. Should we spend another week or two on it?" It really is happening all the time. I love that you extended it to that, and that you're carving out that space for the feedback loops.

[00:22:13] I was wondering, for you, what role does the feedback loop play in prioritization decisions, either on where to go next or maybe as you're working with these customers? We deal with a lot of consulting firms at Amplitude who use Amplitude for their customers, and they always describe an example where they do these excellent startup activities with the executives of the company. It feels very organic, very participatory design, there's a vision. And then it gets to shipping the thing, and they're already a couple of months in, and they have the stickies all over the wall, and suddenly they never go back to that. They get into such delivery mode. I'm curious, maybe extending that: you've got a lot of research going into it, you have a lot of assumptions, a lot of hypotheses about what's going to work out, and you want this customer of yours to be successful at what they're doing too. How do you even encourage them to think about getting this out there early enough to be able to have enough cycles before that year is up that you have with them? It's tough sometimes.

[00:23:20] Speaker 1: Really tough, and it goes to that third area of the soliloquy that I had half an hour before this, which was around motivation design, or persuasive design. It's a reality of the world that we live in that, on one hand, there's so much marketing that espouses how quick it is for a piece of technology to be deployed. You have a group of executives who hear that and have been trained in this mode of thinking fast, quick and cheap. The Silicon Valley mantra of "move fast and break things" is terrible. We have to bury these notions. They are so problematic, and particularly, I can't underscore the point enough of how problematic it is to take such a notion and apply it to the world of artificial intelligence.

[00:24:15] In terms of that ongoing negotiation, that's a fact, and that's part of the reality. You need to start out being clear, and you have a responsibility to be clear, about what that research program is going to look like, what it's going to do and find, and compel people to see that there's a need for it. I love the fact that more watchdogs are demanding that standards are going to be implemented in this space. And then the work proper happens, and everybody does get into delivery mode, and you really need a very mature team to understand how they each need to work for each other.

[00:25:00] It also is not helpful, again going back to the idea of these separate tracks, if you have an engineering team or a data science team that isn't appreciative of working with other disciplines. Then what they're going to be saying to a customer will be setting the expectation that you only need to think myopically. It's important for everyone to be there to support each other, and knowing how that needs to come out in the words being expressed in the space of a meeting when perhaps your discipline isn't being represented in that meeting.

[00:25:33] I strayed a little bit, I recognize that.

[00:25:35] John: No, but I think that this is part of it too, because you get one part of the team... I even remember being at Zendesk myself, or wherever: one part of the team was lined up to do rigorous follow-up on this thing and to keep it moving, and meanwhile, in the product manager's lens, it's a bait and switch. I always make the joke that the product manager at night goes and flips the feature flag switches and gets it out to everyone. "I'm sorry, last night I was getting a little tired and I decided to release it to all the customers."

The biggest prioritization traps

[00:26:09] John: Cool, we have a couple of minutes left. I was hoping, Vinnie, when you've observed teams, what's the biggest prioritization trap in your mind? What's the one that even you've caught yourself doing, but you observe over and over and over again? Where do people often go wrong, so you can leave them some actionable advice about trying not to go wrong there?

[00:26:36] Vinnie: Yeah, it comes to mind really quickly, which means that it's probably something I see a lot. I think there's a couple of parts to it. One is a bias to features over outcomes. Especially when you're an expert, it's easy to jump and think that you know best, and assume that you are your customer and you're thinking the right things for them. I see priorities, or things on a list to prioritize, that are actually not outcomes, but just feature A, feature B, feature C.

[00:27:09] As part of that, people that have been around a long time, it's almost like they have a vision for what the product's going to be, and they just want to execute on what their vision is. The priority is, "Here's 10 things to execute my vision. How can we stack rank my vision?" as opposed to being something that's much more flexible and outcomes-focused. I love the framing, and you've mentioned it a few times, of opportunities, like an opportunity backlog. That language, I think, is much better than just a feature list or whatever else we might call it. That bias towards features, and maybe people just executing visions as opposed to properly looking for feedback.

[00:27:53] John: Yeah, that's great. I would think that's exactly like mine, where it's this quest for ultimate certainty, instead of saying, "That's a really powerful opportunity. We might try a dozen times and miss the mark on that, or it might take us 12 to 18 months, but that's what we want." There's the difference between prioritizing at that level and playing this inane feature Tetris. If you're trying to decide between five or ten days to take on something, you're in the wrong game. That's a hard game to play. It says okay, we can wrap up any time. Quickly, what's your big prioritization gotcha?

[00:28:40] Speaker 1: I think just if people were more explicit about something that's intuition-driven versus data-driven, or insights-driven. And by data I don't mean numbers necessarily. Obviously I'm very much a supporter of qualitative data when it comes to getting an understanding of the needs. That was quick, John.

[00:29:00] John: No, I think that, well, your request, be explicit about it, that's what I think too. If it is an intuition-focused thing, I think some people wrap that in this faux rigor. Intuition is okay, but let's just be honest about it.

[00:29:20] Speaker 1: Right, exactly.

[00:29:22] John: This was great. I was really happy to moderate this with you all, hearing stories from Vinnie and "Behanus" [?]. This is a wonderful thing. Prioritize some opportunities next week and get out of the feature Tetris game, and you'll be in good shape. There you go.