Building Product & Engineering Roadmap

Guillaume Maron at UXDX Community: France. Video: https://youtu.be/eu96OFZB-70

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Prioritization is like sport: if it doesn't hurt you aren't ambitious enough

[00:00:03] I'd like to take some time today to share with you what we call the prioritization and pitch process. Basically, how do we choose what we work on, how do we start the teams, and how do we do prioritization? That's always something difficult, I guess, in all companies. Just quick words on myself: I'm Guillaume and I'm one of the co-founders of Dashlane. We've been doing this a while, and we're a password manager, so we help our users manage passwords.

[00:00:46] As an opening note I would like to share something, which is that I find prioritization to be a bit like sport, in the sense that if it doesn't hurt, if it doesn't hit, it usually means you are not ambitious enough. I have found prioritization to be very painful at that time, and I hope that this is because we are very ambitious.

[00:01:10] Why is that? In the recruitment process we always focus on assessing if a candidate is motivated by the product, is motivated by the vision, cares about the product, has tried the product and so on. And so at the end you end up with so many people who care about the product. There is something I'd like you to have in mind through the presentation, which is that if prioritization is frustrating, is painful, I find we deal with them because it's because everyone cares in the company. So that's something to keep in mind, and this presentation does not aim at explaining how you guys shall do things. It's more to present to you how we do things in the hope it can help you experiencing some processes.

Where Dashlane is, and how the organization evolved

[00:02:03] A bit of context. We created the company 10 years ago out of school with [?], who was already a successful entrepreneur at that time. Ten years later, a bit more than 10 years now, we have like 14 million B2C customers and 10k enterprise using the product. Our scale is basically we have about 20 product and engineering teams working at any given time.

[00:02:36] A bit of context in terms of where we are in terms of maturity, so you understand the process we have today. It's important to understand a bit the path we followed to get here. What we try to do is to evolve the organization iteratively as we need to scale, because from two people to 270 people it's not the same. We started in the garage mode, of course, as everyone. We actually did not start in the garage but in the basement of an engineering school.

[00:03:13] We moved five years later to form what we thought was a proper structure[?], what it is when you join with proper training and so on. We tried to build a by-the-books portfolio process to track everything we do from a roadmap perspective. We set up OKRs, of course, to track objectives. Then we evolved at some point from a platform focused organization to a business focused organization. Before, the Android developers were working together, the iOS together, and now we work in teams. We continue to evolve that into mission teams. The difference between a mission and a business team is that a mission team has a clear goal that is time-bound, in a sense, versus the business team which was more business.

[00:04:07] Recently we revamped what we call the dual track[?] process, which is how do we invest both in the tech stack and in the business itself, in the product itself. And this year, as everyone else I guess, we are learning how to work from home. Obviously that was not the plan, but this is what we are learning today and so far it's working pretty well.

Discovery: where the ideas come from

[00:04:37] Let's enter into the specifics. The question is really, how do we choose wisely what we do? How do we maximize the chances of success? How do we discover ideas? How can we best serve our customers? How do we choose between the different ideas, and so on? First thing first, we have to set up a strategy and to enter a framework to follow how we progress towards that strategy. So we are using a framework, OKRs.

[00:06:11] And metrics. Now that we have defined what is a strategy and we have a framework to follow that strategy, we need to find out, okay, what do we do? That's discovery and ideation. There are a lot of sources. I will go quickly through each of those sources. It's very important, because you identify what you want to do from them.

[00:06:59] What we have is data analytics first. The data team is helping us in several ways. They are doing analyses from how the users are using the product, behaviors and so on. They are also doing deep dive analysis on why our premium users are churning: is it because they've stopped using the product, is it because they had a payment issue? And they are doing recommendations on the things we could do to improve the business, and then of course supporting other teams into the discovery phase.

[00:07:33] Of course, customer reports, abuse reports, getting insight from my users, getting every designer close to the users by each mission team to be part of the process. We have a product research team. Their goal is to understand the user. So I would say there is the data team and there is the product research team: one is very quantitative, the other one is more qualitative about understanding the user, doing things such as market research, market benchmark, surveys and so on.

[00:08:14] And one tool they use is Productboard[?] to get insight from both customers and employees. It's a tool where customers and employees can submit ideas and provide insight.

The pitch process: turning ideas into qualified missions

[00:08:33] All of that brings us a lot of ideas, and now maybe we can prioritize. But actually what we found out is that at this stage we cannot really prioritize, because if we do a parallel with the marketing world, what we have is unqualified leads, or unqualified ideas. We just have a bunch of ideas but we do not have enough information to do prioritization. We don't know how much effort it will take, what we can get out of it and so on.

[00:09:01] So what we've set up recently, like six months ago, is the pitch process, which is how do we translate ideas into concrete initiatives, how do we qualify ideas to have qualified mission teams. What is a pitch process? It's very similar to a startup pitching its ideas to investors, but here it's a team who pitches an idea to the exec team for resources. So we have to formalize that plan, to give the context, to explain what we want to achieve, to request some funding, people, budget, time.

[00:09:42] We do that whenever it makes sense. There is not a specific time to pitch, and every time a mission is pitched it is either directly rejected or, most of the time, it is added into a backlog of qualified, already-pitched missions.

[00:10:01] How do we do that? First we have the discovery phase I described earlier. Then the product manager, for business driven mission teams, or maybe staff engineers for technical mission teams, takes the lead on pitching a mission. He would trigger the pitch, he would iterate with stakeholders, and he will do a first pre-pitch with the CPO and the CTO. He will get more feedback, adjust his pitch, and eventually go and pitch the mission to the exec team. And after that, of course, will come the prioritization process, but I will talk about the prioritization process itself a bit later.

[00:10:52] Sometimes we need to fast track things because of legal or business requirements, or blocking technical issues of course. But we still try to pitch any mission, even things that we have to do no matter what, because it helps creating the fair framework for all the business needs.

What goes into a pitch

[00:11:14] What is the outline for the pitch? The outline is, of course, what business problem do we want to address, what opportunity do we want to seize. Then, what OKRs we'll use to make this problem or opportunity more concrete, and how is it tied to company OKRs. Then we would ask the data team and the product research team to add some background on the problem, leverage data, leverage insight we have from Productboard[?] for instance, from user surveys. We link with strategy and look at the risks and how we can mitigate those risks.

[00:12:02] And then we will try to assess, to quantify, the impact. How is that different from the OKRs? The OKRs are not always like moving active users, but it's still important for the exec team to understand what will be the impact in terms of new active users or other major metrics. And that is usually done by the analytics team, because it's very hard to predict exactly a business impact.

[00:12:33] Of course, as this is a kind of funding exercise, we want to list the skills to achieve the mission, the staffing, to give an estimate of when we think we will achieve this objective, of course, and then to give a confidence indicator, so the exec team can balance between high-risk high-reward missions and low-risk low-reward missions. Of course we'd all like high-reward low-risk missions, but that's usually not the case.

A worked example of a pitch

[00:13:03] So there are a lot of things in the pitch, of course. What I've done is I've tried to build a very, very simplified version of one which I would like to walk you through today. Of course, usually pitches are much more dense. It's usually between 15 to 25 slides with a lot of data, a lot of thinking. Here is a very, very simplified version.

[00:13:32] Let's imagine that the data team has done analysis and has found that user segment X is churning on mobile much higher than other user segments, and I want to create a mission team to address that issue. And I also know that those users are not performing the top retention actions in their first week, such as adding a password on mobile with Dashlane.

[00:14:00] So my goal is to build the onboarding experience for mobile users to get started and to not churn in the first week. My mission is pretty clear: get mobile users set up and using Dashlane. I have OKRs, decreasing churn on mobile, increasing key actions within seven days, and I'm linking that to the company OKRs, because it will be much easier to measure the first week retention impact than to measure [?] value directly. And so when I set the success of my mission based on my OKRs, I will explain to the exec team how it is related to the company OKRs and to the company strategy.

[00:14:55] Of course, when I pitch my idea to investors I cannot just ask them to give me money and time by telling them I will solve this issue and that's it. I usually have to explain a bit how I intend to do that. And so we are iterating on what we call a framework[?] which goes from very high level down: there is a vision, some pillars[?], some strategies, some tactics and some KPIs to measure the success of the tactics, and then of course the capability. This allows for discussion at different stages with different stakeholders in something that makes sense together, and each process may need to adjust as we go, to adjust either at a very high level or at a lower level, like we don't believe that this tactic is the right one. We like the general idea but we disagree with the specific tactics, so we'd like you to work on that.

[00:16:01] Of course, risk and mitigation. One example is maybe user segment X just has a lower intent, which is why they churn. So mitigation can be to have the product research team go and talk with users from those segments X to prove that, no, actually they have the same intent, it must be something else. Well, maybe it would be hard because you don't know why this user is churning now, so I will have to experiment. So what we can do is user testing, to try to avoid implementing everything before getting some insights, and of course a contingency plan[?].

[00:16:43] So here I have minimized the size of quantifying impact, but we have to take some assumptions, we have to involve the data team, and as I said it's not an easy exercise. Impacts can be about revenue, but also brand image, technical debt mitigation. That's why prioritization is so difficult.

The quarterly review and the prioritization exercise

[00:17:03] Moving on: staffing, the people we need for the team. And now is the time to do an investment decision. Well, actually not quite yet, because the pitch process is for new mission teams, but at any point in time we have plenty of missions that are running. So what we do is that we have a quarterly review. It's once a quarter, it's basically the same thing as a pitch process but for mission teams that are running currently. So we will review the wins and the setbacks, we will review if we want to continue, if we need more staffing, if we need less staffing, which sometimes happens, more rarely.

[00:17:51] We have done that, so we are in a good place to actually do finally the prioritization. We have the pitch processes which described the new missions, we have the review which allowed us to get a refresher on what we are working on currently, where we have some wins, where we have some setbacks, and so we can decide what do we do for the next quarter.

[00:18:15] So it's a quarterly exercise that we do. Last time the way it worked is that we did four different scenarios with different assumptions, with different strategies. We did staffing for each scenario, we noted the impacts: if we do that, this team cannot start. Then the exec team discussed and then came up with a plan[?] which is in between some of them, with the CIO, and when we are there we can start working.

Communicating the decisions, and tracking progress

[00:18:52] So, communication, because that's a very important point. As I said, I have found prioritization to be very frustrating, because some people will get their mission approved, some people will get their mission stopped, some people will get their mission with reduced resources, and you have 100 percent of the company who is really into the product and into Dashlane's vision, and who cares, basically, who cares.

[00:19:21] So it's very important to communicate. I think it's very important to communicate a lot. What we try to do is to be very transparent, to explain the whole process, to explain all the rationale of the different decisions, and to communicate first with the teams impacted, especially the teams who will stop, have their staff reduced, before we do a company-wide communication. And then it's done, time to start and stop the mission teams, so the [?] convenes.

[00:19:52] And finally, how we track and share progress. Here we could discuss about that for a long time, much more than one minute, but let's go quickly. Actually what we do is nothing very original, but it's very important. We have a sprint dashboard[?], which is a Confluence page with a summary of each mission's sprints, so anyone can go and look where we are and ask questions. We have OKR check-ins to track OKRs, very important. We have internal demos, that's super important to be able to know the work done by the teams. And we have, obviously, celebrating success and also failure.

[00:20:39] So in terms of closing notes, thanks a lot for the time you've given me. It's hard to do a presentation remotely, so I hope you liked it as much as possible. If there is one thing I'd like you to take away from this, beyond of course trying Dashlane and using that six month premium code of course, there is one thing, it would be what I said earlier, which is that frustration in the prioritization process should not be only viewed as a negative thing. Of course it's good to iterate and to improve the process, but if you have a team with a hundred people who care about the product, you will have a team who will get frustrated during the prioritization process, and it's important, I think, to be transparent about that, because verbalizing that is always a good first step to make the process easier. Thank you very much, and if there is any question feel free to ask anything.