Plan for Failure to Ensure Success

Nathan Henry at UXDX Community: EMEA Series: Product Strategy (Part 2). Video: https://youtu.be/BRW0KI6ieRY

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Planning for negative outcomes

[00:00:00] I appreciate having a chance to speak. My name is Nathan Henry, pronouns he/him, and I'm the head of product management at Postlight. It's a digital strategy, design and engineering firm in New York City, and I'm so excited to speak now about planning and success. As we know, every good product is made up of a series of decisions that rely on the best-known evidence at the time. But when we product managers rally with our designers and our engineering partners to make decisions on behalf of what's best for users, we bias ourselves towards positive outcomes. That's the right default orientation, but it's only solving half of the problem. There is the blank space of opportunity of planning for negative outcomes, which means that we can avoid mistakes that are predictable failures and that we aren't safeguarded against.

[00:00:46] As I was preparing for today, I found a quote that I just had to share, because it summarized this better than I could say it. It's a quote from Taylor Swift, and it says, "Just because you made a good plan doesn't mean that's what's going to happen." Honestly, I think that's the core of what we want to do here. We want to plan for everything that we can, but we don't know what the outcomes are going to be. What I do want to say to start this presentation is that we should be hopeful and we should ship great products, but we should also know that our best laid plans hardly ever pan out the way that we want.

[00:01:23] So then what? How do we move forward? How do we get to our new definition of success? What will our clients think? How will our customers respond? Those are all great questions, and I'm hoping that we can shed some light on this topic today and give some mindset shifts.

[00:01:39] First and foremost, I just want to say that planning to fail isn't negative or defeatist. It's just realism. The first tip for today is a mindset shift. A strong PM knows all too well that no product is ever built without mistakes, or bugs, or stakeholder issues, or risk. That's just what we're going to expect, so why not lean into that a little bit? While planning we often ask "how might we," but what about also asking, "And if that doesn't work, then what? If that fails, how will we?" Push back also on bold statements, anyone that says, "Oh, this is easy," or "That's just an edge case." True, we don't want to solve for every known problem, but focusing on what could go wrong with our solutioning actually points out risks and holes in the problem that we're attempting to solve.

Lessons from a 911 call center

[00:02:30] Before we dive in a little bit more, I should give some background on myself. I have a different background. PMs all come in different sizes and shapes, of course. For me, the influential time of my 20s and 30s was spent working in a 911 emergency call center, so my world view is on disaster planning and crisis response. Admittedly, that's a bias that I bring. Life or death was often in the decisions and risk mitigation planning that I had to make, and that was oftentimes spur of the moment.

[00:03:03] If I think about this as a product: my product was help, or safety. My users were usually ill, or in distress, or victims of a just-committed crime. My features, the tools I had available, were police, firefighters, medics, and then over-the-phone tips and tactics, such as administering CPR, or hostage negotiation. That's something I've actually used before. I became a master of asking questions and eliciting information, and I used this information to make split-second, life-changing decisions. Most of the time I shipped success, but sometimes I didn't. Sometimes a bad outcome happened. So that's my world view, and I think the important part of the story is that bad stuff will happen. But fortunately for us, it's just software, and it's a bug, it's not death.

[00:03:52] How does crisis management scale to software? Let me talk to you about why I think this is important, through a learning that I took in my life. Back in my 911 days, we had what we would call frequent flyers. These are folks that would call repeatedly. There's one person in particular I'm thinking of. She never had an emergency, but she often called: someone was smoking cigarettes too close to her home, or the neighbor's vacuum cleaner was too loud. Things that were clearly not emergencies, not super important. Over the course of time, her user behavior was defined through a series of interviews with predictable outcomes, very much Mason [?] in terms of what the typical problem would be.

[00:04:35] But one day she called and she was having a medical emergency. I had bias, and this is the thing that I learned on this day. When I saw her name and address pop up on the computer-aided dispatch display, I immediately settled into my routine normalcy of calm questions. In my head I was thinking, "Well, I'll start the report, I'll clear the ticket, no big deal." But I learned quickly that her behavior had changed and her needs had evolved. I had to respond immediately to this new requirement, this new user story, if you will. In the end everything was fine. She was okay, and that happy outcome was transformative for me, because I wasn't ready for it. I wasn't prepared for this new requirement. My mindset was set on routine and expected outcomes and behaviors, not thinking through what might happen when I take that call, and what could be the outcome of what that user needs. So I really took this as an opportunity to shake it off and reset.

Outcomes over outputs

[00:05:31] Applying that lens of crisis management to software, a few things that I'd like to talk through today. Really focus on outcomes over outputs. We spend a lot of time thinking about what success looks like, but I also want to think about what failure looks like. How would we plan for that? What shape might that take? We know the outcomes that we want, and we know the outcomes that we don't want, but we might get them anyway, so we should start to plan and talk about what those actually look like. Apply a lens of the unexpected.

[00:06:00] A framework to think about this with your users: outputs are what you end up with after a series of inputs. Your inputs can be decisions, effort, features. But outcomes are what you're given and what you're left with at the end of the day. Outcomes are what you desire, and they're the thing that you can most influence. You can influence them by influencing your users. You can change patterns, you can elicit motivations, you can begin to change goals and actions.

[00:06:26] The other part here is that as you do start to change outputs, your user behaviors likely will change. Think about when you're first designing a product. As you start discovery research, behaviors are completely undefined, but as you learn more you begin to predict behaviors and start solving for them. Your product decisions are set out on how to support a goal that your user is aiming to achieve, but those users evolve and their needs will change. You should set clear goal lines as to what you will solve for, because obviously you want to make sure that your scope is controlled, but also have some variability within those features to create experiences that will scale as your users' needs evolve.

[00:07:07] The last thing here is the evolution of your customer needs. As we all know, any company's main goal usually is revenue, and that's usually driven by adoption and continued engagement. But as your software is being designed and built, has the landscape changed? Have your users evolved? Think globally about factors that could create a recipe for failure.

Case study: the Bird Migration Explorer

[00:07:27] Taking this through, I'd like to talk through a case study of something that I've recently shipped. I worked with the National Audubon Society, and just a couple of months ago we shipped a gorgeous web app, if I may say so, called the Bird Migration Explorer. It was a 20-month program. It was multi-phase. It's one of the most technically complicated products, with the largest data set, that I've ever been part of, so planning was paramount to our success here. The end of the story first: we did launch on time, and the launch was boring, which is exactly what you want. But how did we get there? We planned. We had to plan a lot, and there were a lot of considerations.

[00:08:01] The key hallmark of this app is the aggregation of data. There's bird tracking data, bird banding data, abundance data, currents data, GIS data; I could keep going on and on. The headline here is that these interactive, animated maps paint the full annual migration for 458 bird species, and to do so we have more than 4.2 million point-to-point migratory bird connections across the Western Hemisphere. That's just one piece of data that we're showing, and there's a whole bunch of subsets here.

[00:08:35] One of the tricky factors in our planning was that the Audubon Society doesn't actually outright own all the data. Actually, no one entity does. So we had to find partnerships and agreements, and we designed and built our app around having access to all of this data that we needed. We planned for success. The designs included these programs, and the architecture and the schema supported having all this data available. But what if we didn't get the data? What if we missed one piece? What if we didn't get any of it? How could I evolve my product vision to support the failure of obtaining this key data, integral to our digital narrative?

[00:09:10] We had clear plans for how we could evolve the story for any missing data, or all of the data, and still a plan to support users if our agreements weren't successful. This was done through planning, expecting different scenarios. We had contingencies, we had safeguards, and we had plans. Ultimately we did get access to all the data, so the failure plans weren't needed.

[00:09:32] Additionally, behavior changed in our users. When we started this, we first identified two primary users: birders, and conservationists, so scientists. Both are well versed in migratory birds. They understand taxonomy and guild and other attributes. But what if our mapping data and visualizations just weren't enough to keep those users interested? What if we failed to create a compelling reason to engage users to return to the website? This lens of thinking about "not good enough," or not offering a deep enough experience, created some new features, and it also downscoped others, both of which added to the overall experience.

[00:10:08] We found ways to engage new users, and this is a personal story. As we were thinking of bird nerds and scientific conservationists, we wanted to also find use cases to get new folks into the app. So my target user became my mom. The tooling that we eventually came out with supports my mom watching birds in her backyard in rural Illinois, as well as these bird nerds and these scientific conservationists. Long story short, by thinking of how we may fail our users, we found ways to support their needs even more, and we actually even created new target segments for engagement.

A framework for mitigating challenges

[00:10:52] One thing that I really love about UXDX is evolving from project thinking into product thinking, and that aligns really well with how I approach product and how I lead teams. Without being negative, or seeking to be an anti-hero, I want to ask some different questions, and I want to put a framework in place for how we might think about mitigating challenges. A lot of times we focus on: what does success look like? What is our MVP? What's the definition of done? What does a successful launch look like? Those are all great questions, absolutely wonderful questions. But at the same time, thinking more about success, we want to mitigate challenges along the way. We have to validate all of our decisions for recovery against our product requirements, to ensure that we have a bulletproof user story and solid acceptance criteria. So more than just a backlog or a story point.

[00:11:42] In order to do that, we want our features to be really well rounded and fully vetted. So I actually like to think about other questions, such as: what does failure look like? What could go wrong? You can plan for that. What is our MFP? I just thought this was kind of fun: our most failed product. What is the worst thing we could put out there, and how do we not do that? Plan to not do that. When is done good enough? Scope control is also extremely important. We don't want to continue to solve all the problems for all the users. We want to have some controls, but we also need to balance the line between being good enough and being done. And then, what stops us from launching? What are roadblocks that we might anticipate, and how do we start to get ahead of those?

[00:12:25] If you don't plan for outcomes that include all possible outcomes, you're basically telling your users, "Hey, you're on your own, kid." You don't want to look back at your clients with a failed launch, thinking about what could have and should have been. You want to think through what works. You want to plan for failure.

Tips for planning for failure

[00:12:42] As we continue to move through planning for failure, you're probably thinking to yourself, "You need to calm down. This is software, this isn't 911. There's no hostage situation happening here." And you're right, not everything is that bad. But these tips can help you ship a safer, better, faster, more inclusive and scalable product. This is just a bullet list of some things I came up with. Again, we started with a mindset shift, but I also want to make sure that we're clear here that this is not just being negative, this is not being defeatist.

[00:13:12] So don't dwell in despair. When I think of client engagements, they're really exciting, and in that honeymoon phase during the project kickoff, the world is possible, there's eternal hope, and you're going to do all the things that you want to do. Enjoy that, be in that moment, celebrate that time, because eventually an invisible string and an unknown constraint will show up. It's how you handle that and plan for that that will undermine [?] your path to success.

[00:13:36] When something does happen that is unexpected, it's okay. You've planned for something bad to happen; you know what's going to happen. So enjoy the discomfort. I know that seems like an awkward place to stay, but discomfort is a wake-up call that things aren't going well, and that you need to find a way out of the woods. Sure, you could run to the nearest daylight, but you could be running further into harm's way. Instead, stay in that moment and really analyze why there is discomfort. What happened that led the team to this place? What went awry? What could go differently next time? How did my actions contribute to us being in this place? When you have all that data, you'll be able to lead your team to the best safety, which might not be the closest safety.

[00:14:20] The third item here is be reasonable. You can't plan for everything. I can't imagine anyone would have predicted COVID changing our world, or catastrophes, or natural disasters. You don't have to be a doomsday person here. Scope your planning to what you might expect, and not a global emergency. There is some normalcy here.

[00:14:39] Rely on, but don't die on, data. Data is historical, and why data is great is the lesson in what it suggests the future might be, but it doesn't guarantee the future. It doesn't construct how you'll get there. Data also sometimes has gaps and biases, and users aren't data, they're humans, and humans are very complex. Humans will always foil best laid plans.

[00:15:05] Iterate often. See things in a different way, ask users different questions, hear feedback and show new paths. User testing is the best real-world sample for your product that you can ever ask for, and believe what they tell you. If they're having an issue with something, it's probably not working for them, and you might want to go back and figure it out.

[00:15:24] Think of obstacles, not just opportunities. If we want to action a particular piece, what might get in our way? What could prevent the user from completing this action? It's easy to understand "click this button, launch the screen," but think it through the other way. How would the user not find that button? What if the user doesn't want to engage with that call to action? Those types of things help to find these patterns.

[00:15:49] And finally, turn the "how might we" into the "what if." Not all planning is disaster or life or death, but the mindset shift is similar, and it'll pay rewards to your team and your product. There's a quote that I like to think of here. Being a tennis fan: as you enter the halls of Wimbledon, there's a quote from Kipling that says, "If you can meet with Triumph and Disaster and treat those two impostors just the same." I believe that in those moments you will have earned your reputation as someone who delivers client and customer confidence in their product. Hopefully these tips will be helpful. I'd love to answer any questions that you might have.