Definition Of Done

Paul Adams at UXDX Europe. Video: https://youtu.be/X4gzW80iezo

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Roads are done, software never is

[00:00:00] Good morning everybody. I'm going to talk about the definition of done, and I'm going to start with a really simple picture of a road that I think everyone can agree is not done. When it comes to things like roads, the state of finished or unfinished is very clear. So here's another picture of a road which is clearly done, and this road will stay like this for decades to come.

[00:00:25] I made this highly scientific chart for today, which shows the pace of innovation on one axis and then how roads developed over time on the other. You can see that there were huge increases in innovation in the Roman Empire era, nothing much happened over the course of 100 years, and in the 19th and 20th century you started to see technologies developed that gave you roads that look like that.

[00:00:49] The challenge for all of us, or the opportunity for us, is that roads and software are completely different. With roads the pace of innovation is very low, at least in our current society, whereas with software it's very high. Software changes rapidly, day after day. And with roads there is no direct competition. There isn't someone over some other hill trying to build a different type of road, a better type of road. With that road there's no direct competition. There's indirect competition, like rail and flight and so on, but it's basically a solved problem and it's easy to see when it's done.

[00:01:26] Whereas with software there's tons and tons of direct competition, and even if you find yourself in a space where there's very little competition, your success in that space will soon mean that there will be many, many a startup chasing your success. So in software nothing is ever done. Nothing's ever done. I think that's just a true reality that everyone has to embrace.

[00:01:46] The real question then, and the thing to think about, is that in a state where nothing's ever done, if you're slow at shipping product changes, if you're slow at that, you're effectively moving backwards. You're not even standing still, because everything around you is moving at such a rapid pace you'll soon find yourself behind.

[00:02:07] To illustrate this point and really bring it home to people, I think it's really, really important I've introduced this character. This is the Grim Reaper. I was not familiar. This is death. This is death for any project or company or product. The pace of innovation is so fast you need to obsess about moving quickly.

[00:02:32] So the real question then isn't when something is done, because nothing is ever done. It's really: when do you stop working on one thing and start working on something else, and then more importantly, when do you go back and start working on that first thing again? Because in software you will always, always have to go back and work on that first thing again.

Inputs, outputs and outcomes: the system we all work in

[00:02:52] I'm going to talk about three things today to try and illustrate this, and I'm going to build out a system for you. The first thing I'll talk about is inputs. In other words, there are inputs in deciding what you need to build, and you must build something. Then there is outputs: you must ship the thing that you've built. And then there's outcomes, so you need to know what's the impact. The good news for everybody is that there is a system in which inputs, outputs and outcomes all connect, and it's the same for all of us, and what I want to show you is dead simple.

[00:03:22] This is the system in which we all build product. I'm going to assume that everyone here has a product and engineering team. You may call them something else, maybe it's developers versus software engineers, but basically it's the same for all of us. We all work in these teams. Maybe big, maybe small, but inside every company we have these product engineering teams, and they do projects, which is pretty straightforward. I would hope those projects change the product. That is the point. That is the point of doing these projects: they change the product. And when you change the product, you again would hope that it will change the behavior of your customers. It might sound dead simple, but it's the same for all of us. You do projects to change the product, and that changes the behavior of customers.

[00:04:17] This really simple system needs to be extremely deliberate. When you think about the changes that you see in customers — and by the way, you can use users, consumers, I don't mind the terms — when you think about the changes in behavior in customers, you assume some basic things to be true. One is that you have existing customers, and for those people I'd imagine the change you're trying to bring about is that they will use the product more deeply, or they'll broaden their usage of the product so they use more of the product and get more invested in it, or they will go deep in their usage of the things they already do today.

[00:04:50] I've got a little dollar sign here to illustrate that I hope you've a good monetization strategy in the company, where people changing their behavior and doing those things results in money for the company. That's what pays your salaries, so this is an important system.

[00:05:09] On the other side you have new customers. Maybe people who need to sign up, or maybe if you're an e-commerce company or something like that it's people who are using the thing for the first time, they're buying, they're making their first purchase. Whatever it is, you have new customers. Again, I'm not really going to talk about monetization today. Instead I'm going to assume that the company has got good practices in place where you're able to monetize usage well. It's a simple system that we all live and breathe whether we like it or not.

[00:05:37] The good news is that this is simple and the same for all of us. The bad news is that every single competitor has exactly the same system. Exactly the same system, and they're trying to do exactly the same things. If you don't obsess about this system and obsess about the ways in which it works, they will. They will. That's far more important than worrying about whether the project is done or not.

[00:06:02] All right, so let's get rid of that guy. I'm going to make this a little bit simpler. I'm going to take away those two axes around monetization and usage, and I'm going to put a dollar sign here to say okay, I'm going to assume that that works pretty well. Not to say that you have to monetize every single change in customer behavior, but over the long course, so that you run a good business.

[00:06:25] So that's basically simple. Let's bring in some other stuff here. Here's inputs, and for the most part inputs tell you what projects to do. For most people, inputs look like roadmaps, so you take these inputs, you build a roadmap, and that roadmap will tell you what projects to do. Then outputs: this is shipping. In almost all cases you take these inputs, you do projects, and then you ship something, and then that changes the product. Then the third one here is outcomes, which is the impact of those things. So you take these inputs, do projects, change the product, ship that to customers, and then by design you'd hope — otherwise there's no point — that customer behavior will change. That's the outcome. That's the impact that those projects will have. Hopefully so far this is straightforward.

Why the community pushes outcomes over output

[00:07:17] The reason I'm talking about this is because there is a lot of energy in our community around outcomes over output. If you go back here and see this chart, you have inputs, outputs and outcomes, and for the longest time we didn't think about talking about outcomes, but now it's got huge energy. So you have books like this from Josh Seiden. The reason I'm showing this book, by the way, is because it's excellent, and I'm going to argue for outputs in a minute. This is an excellent book and it takes 40 minutes to read. I highly recommend you get it. But this book is making the case for outcomes over output.

[00:07:48] Another book that somebody may be familiar with is Marty Cagan's book, Inspired. Again, it's a classic book about product management and building products. This is a photograph that I took of the inside of the book. I've even marked it with red pen, and it says it's all about outcome rather than output. So Marty is also making the case for outcomes over output. Again, a brilliant book, and the reason I bring it up and use it as an example is because I think everyone should read it.

[00:08:16] So we've got a lot of energy in the community around outcomes over output. The question for me is why. Why does it require so much energy? Why are people like Josh and Marty, who could be doing a lot of other things, focusing their energy on trying to persuade people to think about outcomes instead of output? The reason, I think, is pretty straightforward. If you go back over the course of the last few years, this was the obsession: it was just ship, ship, ship. And our friend the Grim Reaper is back, because we were told it's ship or die. If you do not ship, your company will not succeed, your product will not succeed. And so it was all about shipping, and it was all about that middle dimension of output.

[00:09:02] If you look at this system again, it is true that we have become obsessed with output. That is absolutely true. But I'm going to make the case that we need to think about all three equally. So inputs, outputs and outcomes, all three matter equally, and it's the system that matters the most. If you look at and think about these things, it's actually the system, and thinking in systems, and thinking about every constituent of the system, that is way more important than focusing on any one of them. As at the Grim Reaper moving slides — I'm not sure what's happened — but that's way more important than focusing on just outcomes or just outputs.

[00:09:43] All three of these matter equally, and this is the system. Inputs, outputs and outcomes. I'm going to go through each one in turn. I'm going to focus most on inputs, so we'll start with inputs, go pretty heavy there, and talk a little bit about outputs and a little bit about outcomes.

Inputs: the five things that decide what projects to do

[00:10:02] So, inputs. In this system, inputs are the things that you bring in in order to decide what your roadmap is and decide what projects to do. As a great system designer once said, in equals out. That's the first thing to understand about this diagram and these systems. There are actually two types of this. There is one, how you decide what projects to do, and two, how you frame projects. Both of these things can be done excellently and done terribly.

[00:10:42] Firstly, how you decide what projects to do. Remember that we have these inputs which decide what projects we do. I'm going to show you how we work at Intercom and show you the things that we do and the tools that we have to make sure that these inputs are good inputs and not bad inputs. You can take these and adopt them, but there's very, very similar types of tools and systems available in your company and you can design this thing yourself. If you take anything from this, it's that I want you to think about the design of this overall system.

[00:11:12] Here's the first thing. Here's the things that we use as inputs, and you can already see that in this basic system we now have this subsystem, this new little mini system over here, that's telling us what goes into the inputs, and that's going to tell us what projects to do.

[00:11:27] For example, we have our mission and vision. We have a mission for the company, we have a vision for the future. I'm not going to show you today, but it's written up, it's like a three-page doc. We spend a lot of time on it and it's very clear. The second thing is that we've got a business strategy. As we look ahead over the next one to three years, we have a strategy in place that tells us how we're going to achieve our vision. All these five things are deeply connected and more or less in this order. So our strategy tells us how we're going to achieve that vision.

[00:11:59] The third thing is business goals. We do have quarterly goals, we do have annual goals. Some of them are revenue oriented, some of them are not, but those goals are pulled out of that strategy, and then that strategy is there to support that mission.

[00:12:15] The fourth thing is that we have prospective customers. We have people who do not use Intercom today and we want them to, assuming that it solves a problem that they have. So we have prospective customers. Our sales team talk to our prospective customers every day of the week. There are literally dozens and dozens of conversations happening between our sales team and prospective customers, and there are reasons why people decide to buy Intercom and there are reasons why they will decide not to buy Intercom. So a huge input is us understanding that.

[00:12:45] Then on the flip side we have existing customers. Our customer support team has thousands and thousands of conversations with existing customers every day. There's amazing, huge insight, and so we've a process — and I'll show you in a sec — to gather all that insight from those thousands of conversations, and aggregate them and make sense of them and make sure that they're a really high-quality input.

[00:13:05] So we take all these five things and we have a little system where they come together, and they're the inputs that tell us what projects to do. Here's basically how it works. The leadership team — I'm on the leadership team, there's ten of us, we run each of the functions like product and engineering and marketing and sales, and then the leadership of the company — we set the vision and we set the strategy. That's what we do. You'll see in a second that even though I run a product team, I do not get involved in any projects or telling people that they have to do project X or that they should do project Y. That's not how it works at all, and that's how we think it should work.

[00:13:50] So, leadership: vision and strategy. We then have a finance team or business operations team, a bunch of other people with the leadership team, setting business goals. We then have our research team, our product research team, and our customer support team, and they collaborate really deeply to create this thing that we call the customer voice report. I'm going to show you in a second, but it's basically those thousands of conversations aggregated and made sense of.

[00:14:14] Then last we have our sales team and our marketing team, and our sales team and marketing team collaborate to create this thing called problems to be solved. This thing, problems to be solved, took us many iterations to get right. In the early days when our sales team first started — Intercom is maybe seven or eight years old and our sales team is maybe three or four years old, and so along the journey we brought in a sales team and started to understand how to work together — in the early days what the sales team would bring us in the product team was feature requests. They would say, hey, customer X needs feature Y, if they get it then we can close a deal, because Intercom was not fit for purpose. We did not think that was a good way to work.

[00:14:59] What we said to the sales team was, we do not need to understand the feature request specifically, although that's useful. We need to understand why those customers are asking for those features. What's behind it? What's the problem that they have? What's the business problem that they're trying to solve, using Intercom as a potential solution? So we evolved this thing together called problems to be solved. That's very specific language. It's kind of Jobs to be Done in a way, but it's: what are the biggest problems that our prospective customers have that we can solve, and if we can solve them they'll adopt Intercom and grow our business?

[00:15:34] So, pretty simple. We have all these inputs and then they all come together to form a doc, basically, these days. It's evolved over time, but it's basically a document that says, hey product engineering teams, here is all the inputs that you need. Here's our vision, our strategy, and we've got our business goals, semi-annual and quarterly. We have our customer voice report, this is what all our existing customers are saying, and we have problems to be solved. That goes to the product engineering teams and then they set the roadmaps.

[00:16:04] So each product and engineering team — we've about 15 product engineering teams and they're quite small, they're all less than 10 people — they set the roadmaps. Again, the leadership team does not set the roadmaps. We are almost two steps removed from it. Obviously there's checks and balances in place, we do look at roadmaps and have opinions here and there, but it's feedback, it's not mandate. So that's basically how it works.

The customer voice report and problems to be solved

[00:16:34] Let me show you two of these. I think you can imagine what a vision or strategy might look like. I think you can imagine what business goals look like. Here's what the customer voice report looks like and here's how it works.

[00:16:43] This here is a conversation in Intercom. Our customer support team use Intercom to talk to our customers. You can use any tool you like. I'm sure every single person here uses some piece of software in their company to talk to customers. There's many out there, and this process works for any of them.

[00:17:04] Here's how it works. Here's a conversation. This is someone giving us feedback about our feature called Campaigns. Our Campaigns feature, we're actually redesigning it, it's kind of hard to understand, and so this person is basically saying, I don't know how to use this thing, I sent screenshots and I'm lost, please help. The customer support team member in that instance can help with all that, but what they do is they tag every single conversation. They tag every single conversation with the team — what team is this, is it the campaigns team — and the type of problem it is. So, hey, this is a usability issue; hey, this is a feature request; hey, this is some other type of thing. We have a whole set of categories and they just tag the conversation, and they tag every single conversation.

[00:17:47] So every week we're getting thousands of conversations tagged by the customer support team. What that means is we can do things like this. We can pull this — it probably doesn't make much sense to most people — but we can pull up a pie chart, for example, here, an aggregation of data, and say, hey, this is for a team called Team Inbox who work on our inbox. Hey Team Inbox, here's all the stuff we're hearing from our existing customers. A lot of the stuff the team hear a lot of right now is like more power, more sophistication, better reporting, better workflows. But it's all in here.

[00:18:20] You can click into this thing and get down to this level. In this example here you can see in the third column it says theme. The theme is advanced editing, and then what does this customer want to do with it, and you can link to the conversation. In this case they want an updated emoji set, and another customer wants to insert specific GIFs, and another one wants to add custom GIFs, another wants an updated emoji set. People are very passionate about emoji and GIFs. You can see the level of granularity this has. So the team, in this case the inbox team, can look at these themes or these specific requests. This thing is really powerful.

[00:18:58] Then it means we can do things like this. We can write reports that say, hey, the biggest — the research team put these reports together and say, hey, the biggest themes are X and Y. In this case we pulled all the recent conversations from the first 14 days of someone being new in the product. In the first 14 days, what do people say, where did they get stuck? We can put a report like this together, which obviously helps us to decide what to do. This is a really big input.

[00:19:27] So that's our existing customers, and the problems to be solved process looks very, very similar. What the team there do with the sales team is more or less exactly the same thing, only it's in Salesforce. You can see there's a column here that says cases. That's a Salesforce case, it's a record of this conversation. They do exactly the same thing. Every sales conversation that they log in Salesforce, they tag it, they annotate it — again this is a process that we've agreed with them and worked and collaborated on — they tag it and then they can create exactly the same type of reports.

[00:20:00] When you go back to the inputs you start to see that we have all these rich inputs. We've got existing customer requests, we have prospective customer requests, we have guidance from the leadership team, we have business goals, and we have a little roadmap process then to decide how to take all those inputs, put them on a roadmap, and then turn them into projects. Then the projects change the product and change the behavior of customers.

[00:20:28] Obviously then there's a feedback loop here, where as things change in the product and customer behavior changes, those things, if they're important enough, will bubble all the way back up into the customer voice report and we'll hear about them all over again. So already you're starting to see that if you're done, you'll kind of know you're done, because you don't hear about the stuff anymore, you solved the problem. But if you're not done, you'll start to see this thing come back. And if I've learned anything over the last years at Intercom, it's that everything eventually comes back. Every company's moving forward, things are getting better all the time, everything eventually comes back.

Frame projects as customer problems, not business problems

[00:21:08] That was the first one, how you decide what projects to do. Here's the second, how you frame projects. Again, back to our system of inputs, outputs and outcomes. The biggest mistake I've seen people make and product teams make is that the projects they do are framed as a business problem. They're framed as a business problem, not a customer problem. I'm going to make the case that that's not good practice, that's not the right way to frame the thing, if you want to see success — success being behavior changing in customers.

[00:21:39] Instead, these projects must be customer problems. They must be customer problems, because those customer problems result in changes in the product that customers can value, and those things end up in business results. So rather than starting with business problems you should start with customer problems and business results. Of course this thing then loops back on itself. Is a business result, good or bad, an input? Absolutely, absolutely it is. This is in many ways the conclusion of the talk, we'll get to that in a little bit.

[00:22:13] So projects must be framed as a customer problem, not a business problem, so that customers see value in the product change and then change their behavior.

[00:22:24] I'm going to give you one simple example of this. We had a project last year where we did it all wrong. The project was to increase the number of seats used by Intercom customers. One way we charge is by seat, so if someone has a seat we charge them an X amount of dollars. If they add a teammate, like in a support team, add another person, it's another seat, it's more dollars. So we had this project to increase the number of seats used by Intercom customers, which is a business problem.

[00:22:47] This project went nowhere. Went nowhere. It didn't even get close to getting shipped, because we quickly realized in writing up the problem that it was all wrong. Business problems. The problem was, why would a customer want to add seats? We want them to add seats because we want them to pay more money, of course, like any business, but why would they want to add seats? It turns out we didn't have any good rationale or reasoning at all, and it didn't reflect in the problems to be solved or the customer voice report. So we just didn't do the project. It just went nowhere. It went round in circles.

[00:23:22] You can see that the business results must come from the inputs. They must come from the inputs, across through the projects into the customer problems, and then down into the business results.

[00:23:32] So, two key takeaways here about inputs. One is build a strong inputs system and process and let that guide roadmaps. And two, frame projects as customer problems that come from those inputs.

Outputs: shipping is your company's heartbeat

[00:23:45] That's inputs. Next is outputs. I'm going to do outputs really simply. I'm going to just make the case for one thing. Outputs, if you remember, in this little system outputs in at least my explanation is shipping, shipping product. Again, it should be shipping things that change the product. It could be in the background, maybe you're making the product more secure, more stable, but the product should be changing. That's the whole point.

[00:24:10] Outputs are getting a bit of a hard time at the minute. We have Josh's excellent book about outcomes, and to fully understand this thing you should definitely read it, like I said. And Marty's book, Inspired, is making the case for outcomes over outputs, and again you should just read that to get the full picture.

[00:24:25] It is true, absolutely true, that way too many people building software are obsessed with counting things that are outputs, like story points, or whatever points system you use to count things like that. We do count outputs, and people are obsessed with counting outputs versus understanding customer problems. That's real and it's true and I don't question that for a second. However, it's the system, it's the system that matters, not going all in on outcomes — hey, it's outcomes time.

[00:24:56] So outputs matter too. We deeply believe this. We've written about this for years. This is a post from many years ago by Darragh, our head of engineering, that says shipping is your company's heartbeat. Shipping is literally the lifeblood. Back to the Grim Reaper and the competition and the fierceness of competition in any technology company these days: shipping is your company's heartbeat.

[00:25:21] I'm going to show you our product changes page, because I'm going to show you how much we obsess about this. This is the most recent thing that we've shipped. Again, we're a Series D company, we're not that big, we don't have a huge product and engineering team. This is October 3rd, which I think was last Wednesday, where we shipped a product change, a meaningful product change, and we told customers about it. Then the one before that — I'm going to realize I can't read the dates anymore — is September 30th. So October 3rd, September 30th, we added some improvements to product tours. Then on September 24th, six days before that, we had reports improvements, improvements for our reports product. What else? September 18th, six days before that, viewing conversations for custom bots, another product change that customers really care about. And then we have September 13th, September 12th.

[00:26:11] So in the last three weeks we've shipped six really meaningful, valuable product changes from those inputs. The reason I'm showing you this isn't to big ourselves up and say this is awesome. This is necessary. This is necessary. I'd like to bring the Reaper here — no, that's a pity, I should have had him here. This is necessary, because there's other companies doing this, and you have got to maintain your obsession with output and your obsession with speed.

[00:26:39] Oh, that's to be done again, hang on. All right, sorry, these are all bunching up. These are — all right, so we're back. Last year — never do that, never go back to animations — last year we shipped over 150 product changes. That's three a week. And actually it turns out there is a Grim Reaper there, but I don't know, in this light it's dead. All right, so this is critical: don't lose sight of outputs. Don't lose sight of outputs.

The feature factory, and why factories are good at some things

[00:27:14] Another thing that's happening is you're seeing more and more talk of the feature factory. John Cutler wrote this and gave a talk, and I generally agree with most of the sentiment in the idea of a feature factory. You kind of know what that is: your product and software team is like a big factory shipping out all these features and no one's really paying that much attention to the features. This thing is gaining traction, so here is "Confessions of a former feature factory PM" and "Moving from a feature factory to a value creation organization".

[00:27:46] I was talking about feature factories with Des, our co-founder, last week, and one of the points that Des made to me was, if you want to ship consistent, high-quality product, which is what we're trying to do here, it turns out factories are awesome for that. Factories are really good at that. If you want to ship great product consistently over and over and over again, and if you want to build this conveyor belt of — assuming you have amazing inputs — you want to build this machine that just outputs all this great product day after day, week after week, it turns out factories are great and there's a lot we can learn about factories. So I stopped hating on the factories.

[00:28:26] All right, so reminder for this: get your inputs right, obsess about the pace at which you ship these things, obsess about it, and shipping is your heartbeat. The minute you start thinking about it as your heartbeat it just changes your mental state. We want our hearts to keep beating.

Outcomes, and designing the whole system

[00:28:45] So lastly, outcomes. Again, a reminder, here's this real simple system of how we all build product. Outcomes are the impact, and this to me is actually quite straightforward. If you want details, again, you should read Josh's book about this. I'm going to keep this really simple, so I'm going to remind us of a few things.

[00:29:03] Here's our outcomes and here is the customer problems, changing the product, and then business results. So outcomes are business results. We also saw that the inputs, which are the roadmaps, which are the projects we do, should be framed as customer problems, that results in change in the product, and that results in business results. So this thing about customer problems is a big deal. Not business problems. The business results come from the customer problems.

[00:29:34] I said that projects must be framed as a customer problem, not a business problem, so that customers can see value in the product change and then change their behavior. So what? Therefore, what? Customer problems must have clear proposed business results. That's how the system works. If you don't have clear proposed business results for this customer problem, question the project. Why are you doing it? If it's not going to help the company that you work for and the thing they're trying to further, then what's the point? So knowing this system and how it all connects together is actually the power here.

[00:30:15] The last thing I'll remind you of is we saw this mini system, this mini subsystem up here about inputs and mission and vision and strategy and so on. That's feeding the inputs, which is feeding the projects that you do. And we saw this loop that goes back, which is that the behavior change in customers goes back in. Maybe you got it right and it's like, hey, that thing is off the list, people are happy. Did we close those deals that were open, or those existing customers who were saying they needed a product change [?] — they're happy now. Or we didn't solve the problem, we didn't solve it. At Intercom — something I'm not going to talk about today — we aggressively scope projects down to the very minimum, and so maybe we didn't solve the problem first time and it's like, go again, try it again.

[00:31:04] In fact, this is how this whole thing works. This is how this whole system works. In some ways it's dead simple. You have inputs, projects, customer problems, then business results, and then the business results and the changes in customer behavior in the middle loop back and become inputs again. This thing happens over and over and over and over again, like tens and dozens and hundreds of times over the course of your company, over the course of a year, potentially. Every single project has this loop.

[00:31:37] So what I will advocate for is not obsessing about outcomes or outputs, but obsessing about this system and designing the system. This is actually how you can get great things done. So, you're never done. We should embrace that idea, and instead — we've got some wonky formatting, imagine it looked really great. Oh no, time here doesn't — I'm sorry, never mind, still a line in there, all right, ignore. So instead, focus your energy on designing the full system. Full system: inputs, outputs and outcomes. And that's it for me. Thanks a lot.