Framework for Balancing Short-Term Wins with Long-Term Product Goals

Michelle Parsons at UXDX USA. Video: https://youtu.be/R7lTwrb_yYU

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Strategy is nothing without execution

[00:00:10] Hello. How's everyone doing this morning? Pretty good. Okay, awesome. I did think that I was speaking like an hour later, so I got here and I was like, oh my god, I have not really practiced too much. But I am Michelle Parsons and right now I am the co-founder and CPO of a company called Lex. We are a social discovery platform for the LGBTQ plus community. I came on and founded Lex with two of my co-founders a few years ago, after having spent two years as a chief product officer at Hinge, where I helped scale Hinge into one of the fastest growing dating apps in the world, helped grow the company from 75 people to over 300 people and helped it reach over $300 million in revenue.

[00:00:56] As mentioned, I've worked at places like Netflix and Spotify and Kayak, and have really spent my career in the consumer tech space really trying to understand how to build products and brands that people love and people stay with. So today I'm talking about de-risking big bets to drive impact.

[00:01:18] Just a quick show of hands. How many of you all have had to come up with a product strategy and then create a roadmap and then get a team together to execute against it? Hi, this guy. All right, we're at a product conference, I hope a lot of you all have had to do this. Great strategy is nothing without great execution. I oftentimes will hear PMs, emerging PMs or mid-career PMs, saying I want to do strategy, and I often couch that in: you can't do much with a great strategy if you do not understand how to effectively go and execute it to drive outcomes, as Rory was talking about. So today I want to talk about how to de-risk the products and features that you deliver, to deliver more impact faster and more often.

[00:02:11] Really quickly, let's orient on what product strategy is. I won't spend too much time here, but essentially product strategy is what you get to when you really understand the company mission, the vision, the business goals, the outcomes that you're trying to drive, and the vision that your team ends up creating. Good strategy is informed by data, an understanding of who your users are, and defines very clear outcomes. What are the metrics that you're trying to drive?

[00:02:41] So in this whole process, you're going to develop your strategy. You're going to use data and insights to inform that. You're going to socialize that and get buy-in with your cross-functional teams. And then you're going to go off and create a plan to deliver against it. Here's where I think all the details really matter and where things can either go really right or really wrong. Did you pick the right outcome, or are you just focused on outputs? Did you actually have an understanding and an alignment across those cross-functional teams to go in the same direction?

The balanced portfolio: quick hits, small bets, big bets

[00:03:19] Now let's fast forward. Now that we have our strategy, I want to talk to you guys about a very specific framework that I started to develop when I was at Netflix. Again, I'm sure many of you all have been in situations where you get to a team and resources are constrained. Is that true or not true? Yeah. That was kind of what I showed up to at Netflix. Resources were constrained and I led our kids and family team there, and it was a team that was kind of put in a corner where everyone was like, we have to have a kids product, I guess.

[00:03:53] This was really before every streaming service out there existed. So no Disney Plus, no Peacock, no nothing. It was just kind of still the early days of streaming, just Netflix and maybe Hulu. And we knew that Disney Plus was coming, so they said, well, we had to hire a kids lead, I guess. And I got there and I was like, okay, well, where's my team? They're like, well, you need to tell us what your strategy is and then you have to advocate for resources. And I was like, okay, cool, we make this plan.

[00:04:23] All right. So in order to anchor myself and my team, I started thinking through how can I actually do more with less. This balanced portfolio approach can really be used to help you understand the buckets of work that you potentially can do. So I broke it down into three areas: quick hits, small bets, big bets. When you think about what you're trying to do here, I won't read this word for word, but your quick hits are meant to unlock your learnings. How can I really understand what I need to do to actually get to the outcomes that the big bets are going to hopefully help me pay off and realize these 20, 30% gains? And then these small bets, this optimizations and enhancements area: how do I build on the learnings that I already have and harvest those wins, so that at the end of the year I don't feel like I've just been tinkering and tailoring, and I haven't spent too much time just taking these big home run swings praying that one of them goes over the grandstands?

[00:05:33] When I think about how I actually approach this, I create this 80-20 rule where you're placing enough small bets to harvest the gains from features that you already know are good. These features are things that you've seen gains in before. You can continue to enhance them. They're probably the things that you said, actually let's ship the V1.5 of that and we'll come back to shipping the rest of that. This is where you need to start to improve that user experience and actually make your design team believe you when you say we're going to get back to that, that you actually go back and pay off those unfinished bets.

[00:06:12] And then your big bets are really things that are company changing. These are the 10%, 20% gains in impact that most people want to see and that most PMs and most teams actually want to spend their time working on. This is innovation. This is actually step changes to impact for whatever your feature is, your product, your vertical or your organization. But these are extremely risky, but they help unlock exponential business and user growth.

[00:06:49] But these are the areas where there is oftentimes high risk for failure. You're doing a ton of trade-off costs here. There's a ton of unknowns because the hypotheses aren't really fully vetted. And it takes oftentimes a long time. So when you get into these reviews with your team, or when you start talking about these things with your team, what you're probably hearing is: are we sure? What data do we have to validate this? What insights do we have? What else could we be doing? That seems kind of risky. Should we actually place our bets over here? This is what I was being told from my leads at Netflix. And I was like, but you know what, if we're actually able to deliver this impact, what if we're actually able to be the market leader? And how do I as an individual build that credibility?

The Netflix kids case study: vision and strategy

[00:07:46] So my framework of the big bets, quick hits and small bets: I'm going to put this into action, a case study from Netflix. I want to spend most of my time here. We have some extra time, 15 minutes left, we can do some questions. During my time at Netflix, again, I said it was just Netflix, there wasn't really a kids streaming product out there, but we knew Disney Plus was coming. What we knew that we needed to do was beat Disney, because Disney had beloved IP. They had Moana, they had Cinderella, they had Beauty and the Beast. And Netflix kind of had a bunch of brand new stuff. StoryBots, maybe. We were trying to build an emerging catalog of best-in-class kids content without the marketing budgets that Disney had. So we had to win on product and we had to win on user experience.

[00:08:35] So I sat with my team, we did a big sticky note exercise. This is what's really important, you cannot skip this step: coming up with what is your vision? What's your team's vision? What's your product vision? And we came up with this: make Netflix the number one trusted service that empowers kids to effortlessly engage with their favorite characters and delightfully discover new ones. Every single word up here is intentional and purposeful. It anchors kids, because we wanted to ensure that we are focused on kids as the primary customer and the primary user.

[00:09:12] But trust was critical, because it's not just a kid who uses your product, it's also their parent. And so we knew we had to win the heart share and mind share and trust share of parents in order to ensure that our kids could come to Netflix. And then it was really critical, favorites were something that were very unique to kids. And as a business goal and business need, Netflix needed kids to discover their net new shows, not just focus on the stuff they already knew and loved.

[00:09:43] So now we had this vision. We are off to the races. We created our strategy. We looked in our data. We really tried to understand who these kids were, what they needed, and then what the business needed. I'm a really big huge fan of metaphors and imagery and visuals, and so we created this really cute little castle metaphor to help articulate this to our executives and to our leads. But our three-part strategy was very clear. We were to build a foundation of trust built with parents. And then everything else was going to be about empowering kids to engage with their favorites and the familiar effortlessly, and then build bridges across to help them discover and dive deep and immerse in that content.

[00:10:26] And a big shift that we wanted to make, that we needed to prove out because it was just an idea and a hypothesis, was that kids behaved very differently than users. Even though that's what the data and research was telling us, we still had to prove this hypothesis out, that kids came to the product saying, I want Pikachu, not I want to watch something. So with that in mind, we had our three-part strategy: connect kids with their favorites, help them deeply engage, and guide them to discovery. Flywheel, rinse and repeat.

Ideas, and the bet that was too big to start with

[00:10:56] Now, this sounds super amazing. I thought it sounded brilliant. I presented this and I was like, let's go, give me the resources. And they're like, how do we know? And I'm like, that's very true. Okay, so back to our portfolio approach to balancing our big bets. We said, okay, step one, we need to connect with favorites. Right now, Netflix is not doing that. All Netflix is doing is pushing discovery content on kids in the exact same interface that the adults have.

[00:11:26] Well, kids are very different users. They have different needs. They had different dexterities. They had different abilities to read, comprehend. They can't read huge synopses. What are these called? Yeah, synopses. They don't really understand tone and context of box art. They get characters. They understand sounds. They understand the familiar, the favorites. That's why you think about Halloween or unboxing, these things are fun for them. So we basically said, okay, we want to change the way that Netflix engages with their favorites by having those things be presented right in the forefront, without friction and with delight.

[00:12:11] So we got to brainstorming with the team and we had a million amazing ideas. I'm going to show you a couple of them. Kids universe: we hypothesized that the UI will allow kids to explore the world of their favorite characters, leading to increased engagement and time spent watching all the content of these particular kids. We're going to make some music, we're going to an eclipse, all these amazing things. This is the world of true[?].

[00:12:32] The other version that we had here was our explicit input. We really wanted to help kids tell us who they loved. One of the things that was really complicated for us is we had no idea who someone liked or did not like. So we explored all these ideas, each of them with our own unique hypotheses. We sat the team, we brainstormed, we presented these ideas back to our executive team. And which of the ones do you think that we wanted to start with? So we had the implicit inputs, we had favorites on homepage, and we had character universe. Just shout it out.

[00:13:10] All right, cool, I love it. So yes, it was two. And why two? Well, two for us was the first experience a user landed on back on their Netflix. So this was really the opening. This was kind of the entry point for all of the other things that ultimately we wanted to do. And we really wanted to test this particular experience to see what would happen if, instead of having a bunch of discovery content on the screen, we instead presented users, or kids, with their favorite content depicted with the character art themselves, with brand new trailers behind that actually were built and made for kids, with a direct-to-play model instead of direct to details page and then another play and another play.

[00:13:56] But we needed a lot of things. We needed a new kids algorithm, we needed a new favorites algorithm, we didn't have this. We needed brand new character art, we didn't have this. We needed to entirely overhaul the UI page, we didn't have this. And we needed this brand new click-to-play experience. So when you think about it, we took a step back. We're like, actually this is a pretty big change that we need to make with a lot of different teams involved. And that's exactly what happened, we got told that by our team. We said, okay, it's going to take a long time, we actually still don't really understand if this is going to work or not, and so the investment is really high compared to what the potential for impact or outcomes might be.

[00:14:33] So we said, okay, all right, how do we identify what we might be able to test really, really quickly? What's our core hypothesis? Again, a hypothesis that kids want to interact with favorites and that will increase watch time. How do we test that somewhere else much more rapidly to create value for our stakeholders and trust, and then allow for us to advocate for resources and investments?

The quick hit: the search canvas test

[00:14:53] All right, so back to the drawing board with my team. We said, okay, another no. What do we do? So we looked at the data and we found that 57% of the hours in kids profiles came from rewatch. Great, we are validating that kids like to watch their favorites over and over again, amazing. We said, where do these watch hours come from then? And what we found was honestly super surprising. We found that 37% of the hours were coming from search, meaning that a kid was coming into the product, navigating to search, typing out on their keypad or on their remote the favorite title, and then pressing play.

[00:15:32] Now, a couple of things came to our mind. One, kids literally cannot use a remote effectively. Kids do not know how to spell. So what is happening here? What was happening is that literally we're causing so much friction for them to get to a watch hour. And also on the other side of that, the parent is probably also frustrated, where they're calling out like, I want character name, typing in character name, being like, that is not a name of the show, what is the name of the show?

[00:16:02] So we're looking at all this information and we're saying, okay, there's something clearly here. These other two rows are also important, the continue watching row and the watch it again row. The watch it again row was filled with content that you had previously seen before. The row floated up and down on the page and sometimes wasn't even there.

[00:16:21] So we said, okay, I'm going to skip this really quickly. How do we then test? What do we currently have in our toolkit that we can really quickly test our hypothesis and get some data? So, search, the search canvas. We knew that kids were already going here. Popular searches prepopulated the search canvas. So this is really meant for adults. You went to search to search for, let's say, Grey's Anatomy or something, and you might see the net new popular titles, and also the button or the cursor was on A.

[00:16:53] So we said, ah, we also have a watch it again algorithm. We saw that kids are also using that particular row to watch content over and over again. Interesting. What if we replace the popular titles, that algorithm, with our watch it again algorithm? And then what if, rather than having the button on the A, we move it to the first title? And what we saw was an increase in 15% of watch time for kids, which actually had an impact on all of Netflix. Now you're taking a subset of users and one profile and having an impact across the entirety of the organization.

[00:17:38] And we were like, wow, this is amazing, awesome. Imagine if we just didn't make them go through all these hoops and put it right on the homepage, then what? And imagine if we actually had an algorithm that was tuned and built for kids favorites rather than this random watch it again algorithm. We got everyone super excited, and we were able to actually go on and produce this bigger bet, which took us a couple of months to build. We had to train a brand new algorithm, get all the character art. This impacted metrics even more significantly.

Unboxing: the next quick hit on discovery

[00:18:09] And then we were able to hop on actually to the next bet, which was discover. What we found actually was that playing your favorites over and over again does what? It doesn't make you play discovery content. And so that was a known risk, but again it was a business goal for the company for us to ensure that we could actually get discovery content into the faces of our new kids users. So rather than deeply engaging, we're like, we think we can do that one, we'll come back to that one later. Let's go test another hypothesis on discovery, another quick hit.

[00:18:43] And so what we ended up doing was we created this thing called unboxing. If any of you are parents, you might have heard of like Ryan's World or something. I think he unboxes a bunch of toys on YouTube and he makes millions and millions of dollars. We were like, oh, that's an interesting idea. What if we take that same concept and apply it to discovery content? Why? We had some data and some insights that showed us that the reason why kids don't want to watch a net new thing is because they need some time to become familiar with it. If you think of the Disney concept, they have sequels and prequels and all the -quels, and they have these moments of breadcrumbing throughout their universe where you get familiar and comfortable with those net new characters before you actually start to watch them for the very first time.

[00:19:28] So we wanted to try that as well. This is your favorites row. We inserted a discovery title in position four, three, whatever it was, and kind of randomized it. And as you got to it, it opened up as a surprise. And now, because it was anchored between your favorites, it gave it a little bit of validation. Hey, this is really cool, you should watch Over the Moon, your two other favorite characters love this thing. This thing also was able to get us to another net new milestone. And that was that, basically.

Takeaways

[00:20:02] Anyway, here is the framework for balancing your portfolio. Takeaways for me, again, are that quick hits are not meant to be just fiddling. Develop a clear and informed strategy. You saw all the work that we put up front in developing our kids three-prong strategy. Our vision was really critical. Identifying who our users were, identifying the outcomes that we wanted to achieve. Dive into your research and data. Get in there, get messy, try to make sense of it. If it uncovers a question, ask more questions. Understand what you actually have available to you.

[00:20:38] What UIs, what algorithms, what things can you, it might not be perfect, what things can you hobble together to try and validate your insight? You need to be able to test a hypothesis quickly, which means articulating a hypothesis, and then leverage those learnings to create buy-in and then de-risk those future bets.

Q&A

[00:21:05] How do you make sure your PMs and other leadership come back to those smaller bets, capacity, when the resources are low and priorities and resources immediately shift after release? Yeah, I think for me it's really critical and important on this front, because I think this also helps to build trust with your team. I've often experienced situations where, especially with my design partners, they'll be like, I don't know, I really don't want to cut this out, I don't have the confidence we're going to come back to it. And I think there's a couple of things. One is you have to build trust with your team, and you have to pay off trust with action.

[00:21:40] I do believe that when you're prioritizing a roadmap, if you can break it up into, hey, 20% of our effort's going to be on these really big home run swings and the other 80% is going to be on optimizations and learning tests, I do think that you can incentivize those learning tests and optimizations because ultimately you should have a greater degree of confidence that they're actually going to produce outcomes. That's how I've gotten in the habit, both personally when I was an IC back in the day and with my teams as well, of encouraging that access to go do your enhancements.

[00:22:20] And again, learning tests or quick hits are not meant to be just fiddling around so you can pump out a bunch of features. They're really meant to couch back up to the core hypotheses that you have in those really big big bet tests. So in the case for favorites, our hypothesis was that kids want to interact with their favorites and that will lead to an increase in time spent on service and watch time and ultimately more moments of truth won. And so in order for us to figure that out, which we're going to get to just in a bit, I'll show you the quick hit test that we ultimately ended up running that gave us signal, some of which we did not even actually end up productizing. They were just a method for us to very, very quickly get an understanding and a learning to our ultimate desired outcome, where we ended up spending a lot more time.

[00:23:16] Next question. How do we balance high engagement with ethically respecting kids' addiction to their favorite shows? Yeah, I think this is something that we had child psychologists on our team, in our UX research team as well. I really think that it's important for us to ensure that, again, I'm never a believer in outcomes and business outcomes ahead of users. I believed that in my time at Hinge, I believed that in my time on many products that I work on. And I think it's our responsibility as people who work in tech to be able to advocate for that voice of the customer and for that user, because there's so many things if you think about in our own products, us as a customer, us as a data source, on things like social media where that dopamine hit is now just part of who we are and it's really hard to escape that.

[00:24:12] I think the same thing with kids is, you really have to be cognizant that these are people who are learning and developing and even more impressionable. So I think for us it was really about finding that right balance of, hey, how do we get you still in the right watch modes that's going to get you feeling confident and empowered, and also into that parent use case, which is also really critical. And so we weren't just thinking about kids, how do we get them to watch more and more and more? Because again, at some point there is an inverse correlation there. The kids are spending so much time watching, the parent's going to be like absolutely not, or pulling this thing away. So I think that's kind of how we spent a lot of our time there.

[00:24:54] Did he do better than Disney? I mean, I think from our user experience, for sure. Disney content I think is just one of those things, and I think that was the ultimate long-term strategy for Netflix, is how do we actually create this longer term relationship with the customer. I think every single one of us saw a Disney movie growing up, and so we have that deep connectivity to a Disney film. We can probably transport right back to when we were a kid watching Cinderella or Beauty and the Beast or Snow White, or whatever it was, and tap into those core memories. And I think the same way, for a brand to be long lasting, and I think this for brands all the time, if you can tap back into that core memory you can really keep that user throughout the entirety of the time that the product is around.

[00:25:48] Where does tech debt fit into these percentages? Yeah, I think tech debt is almost in that small bets area, because it really is an anchor point through which everything else gets done. I spent a lot of time with my technical partners just really trying to understand what can we do now to get us that learning, to get us where we need to go, and does it make sense for us to actually build in, you know, we have to rewrite the database, we have to, whatever the case may be. What do we need to do now versus later in order for us not to be doing a lot of double or triple work? I think that's a really critical question for you to really sit with your technical partners to understand.

[00:26:33] Because sometimes we can get into the habit of saying, let's just move fast, we'll come back and fix it later, move fast break things, it's fine, we'll fix it later. And then you don't necessarily get to that part of fixing things later. And I think this is where again this tension between some of the partnerships with PMs can sometimes emerge or surface, because again there is competing interests. As a PM, you're trying to hit your business goals and your OKRs. And as a designer, you're trying to keep in mind the user experience. As a technical partner, to try to keep in mind the stability and the infra. All right, thank you all very much. Appreciate it.