How To Run A Better Engineering Team

Will Demaine at UXDX Community: London. Video: https://youtu.be/l1LeQ2ZFFX8

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction

[00:00:00] I'll talk a little bit about going wrong the right way. [?] This is specifically around building products, and the companies that were just mentioned, so I should say Expensify, Uber, and, just before I left work, the CEO did say to me, if you don't hire anymore from this event, you're not doing another one. So here's the sales pitch. You heard a little bit over there, but we're an insured peer-to-peer rental marketplace where you can list any items or borrow any items. We are hiring lots of people: engineers, designers, [?] operations, engineering managers, data scientists. If you're any of those things, or something else, and you're interested, come talk to us. That's enough about that for now.

Building a great team

[00:00:55] This talk is about building great products, and I think that it's quite important to first of all talk about building a great team, because without a great team you might as well forget the product entirely. I don't want to spend too long talking about this, but I think it's important to know that they're there. The first thing that I found from these companies is that you need to empower your employees to make decisions. This means giving everyone data, giving everyone the data to make the right decisions for the products. Give people access to the data. Don't lock down data because you're scared that people are going to misuse it. Anyone from people working in customer support and operations all the way through to engineering and any other functions throughout the business: give them access to the data, let them see it, and let them be smart about it, to make some interesting discoveries and decisions. I think it's very important. You find that a lot of good product insights come out of the data.

[00:01:52] The second thing is fostering responsibility. If you're giving everyone access to everything, you need to make sure they understand how to use it responsibly, and this means everything from setting expectations to having a blameless culture for when things go wrong. If you can't let people go wrong, then you're never going to move forward; people are scared to do things. Let builders build. This is stolen directly from Uber's cultural values, or the old ones. It's basically saying, if someone has a good idea, let them do it. Get out of the way as much as you can and let them do their thing.

[00:02:24] The last one is be cross-functional. I've seen small companies transition into bigger ones and not have a cross-functional setup in their teams. They have a mobile team and a web team and a back-end team and a front-end team and a QA team and all these other teams, and that works. But you get to a certain point where you'll find that you can be much more focused on a product if you have cross-functional teams. That means you divide the teams up along the lines of the business. For Fat Llama [?], that might be something like a lenders team and a borrowers team, and you give each of them their own mission and their own metrics, and they have to work on their own products and their goals. You put on each one of them a designer, a front-end engineer, a back-end engineer, mobile, whatever the team needs, support people, sales people, operations. You put them all together and you say, that's your mission, and then they can go ahead and build that product in a much better way than having to wait for another team to deliver on something because they were blocked.

Understanding the trade-offs

[00:03:27] I think it's important, if you want to build a great product, to understand the trade-offs when we're building a product. This is the classic phrase: good, cheap, fast, pick two. You can't have it all, and you have to pick. You can either do good and fast, or cheap and fast, whatever. I think ignorance of these trade-offs will hurt your product in the long run.

[00:03:49] Some examples of common fallacies that you see when talking about or building products. "Yeah, we'll come back and tackle the tech debt later." Okay, it's never going to happen, so be aware of it when people say that. You can't just gloss over it. "Can't we just add a toggle, a button, a filter, whatever?" As soon as you hear "can't we just," warning signs should be going off. "Can't we just" is just a horrible phrase for products. Everything has an implication and you need to be aware of that. You have to think about the trade-offs you're making. Just adding a button might seem simple, but it might add a whole bunch of complexity down the line.

[00:04:27] "We'll just put more resources on it" is a classic one. You've heard about The Mythical Man-Month: if you have one person working for one week on something and you have two people do it, it's not going to take half a week. It's going to take a little bit longer. "We'll build that in v2." Sure you will. Just like you'll get rid of the tech debt. It never happens. "Cheaper will just turn into a really great [?]..." Nice sentiment, cool, but you'll end up in probably a worse place if that's all you do.

[00:04:54] You have to understand the trade-offs, and when you do understand the trade-offs, you realize that there are no platitudes. You can't always get it right, and you will go wrong. I don't know if that's a revelation to anyone, as it certainly was to me at the start of my career, but you will go wrong in building products and making the decisions, and that's okay. You just have to try and go wrong the right way. I spent a bit of time thinking about what that actually means.

Going wrong the wrong way

[00:05:19] I thought it's best to start off by thinking about what going wrong the wrong way is. Building things on a whim that you have to support forever: this is a disaster. It's so easy to do something that's hard to undo. An example of this might be, let's say, "We want to get some other businesses to use our stuff [?], let's build an API for them to use." You build this API, and you think this is cool, and all of a sudden you have people that depend on that. They've made stuff around their business that depends on your thing working. You have to support that forevermore, or upset a lot of people.

[00:06:01] Another one is, "This is cool, we should just build an app." Once you build an app, you can't take that back off people's phones. You can't make them uninstall the app, which means that either you have to make that work forever, or when they open it, it's going to be broken and you're going to look silly. So don't make light decisions on things that you're going to have to support forever.

[00:06:21] Similarly, locking yourself into architectural decisions. This is a little bit more on the engineering side, but if you build all of your infrastructure around a single provider or service like AWS or something, and use all of those things, be aware that you're locking yourself into that, pretty much, and you're probably never going to get out of it, or it's going to cost you a lot of time and money.

[00:06:40] Building things that are hard to remove. You're thinking about the product and you think, "We could add this feature," and it ends up impacting the billing, support, the user experience, or across the mobile and the website and all these other things. Sometimes it's inevitable, but be very aware that if you have a feature that cuts across all these different concerns, removing it is going to be very, very difficult.

[00:07:08] And building things that are hard to build on top of. I think this is one that people don't often see a lot, but if you are trying to build a prototype for something, you might say, "It's just a prototype, just bang some code together, push it out, see what happens, it's going to be fine." Then you go, "Cool, it worked, now what?" You go back and write tests for all the code that you just wrote? No one ever does that. So building stuff to be forward-facing, to be maintainable, so it's not going to be hard to build on top of, I think is key.

Going wrong the right way

[00:07:41] Going wrong the right way. Being experiment driven. I think this is stuff that people talk about all the time. If you're a data-driven company, it's very difficult to actually do in reality, but it is a very good way to build a product if you can. I think it's really important to have a hypothesis or have some goal metrics. If you just build a product and you don't say what success is for that product, how do you know if it's working? You have to define what this thing should do and how it should be used for it to be a success. You might think that it's fine not to do that, because if a few people use it, maybe it's good enough. But the key to building a really tight product is to be able to kill stuff as well.

[00:08:35] Build enough to test that hypothesis, or to test those metrics. This goes into the debate around whether you should build an MVP or an MLP. That's minimum viable product versus minimum lovable product. The thinking around that is, you might build a minimum viable product. This is the least that you need for this thing to work. Then you set your metrics and the goal, say, "We think this is going to improve conversion by 50%." You build this product as fast as you can, an MVP, which is testing to see if it works, and then no one uses it and you think, "Well, that didn't work, kill it." The problem might be that maybe it doesn't work because you didn't actually spend enough time making the experience good, but the product idea was great. If you build a minimum lovable product, the idea is it's still a small scope, but it's enough to test your hypothesis correctly. Sometimes that's going to take a little bit more time, but it's probably going to be worth it.

[00:09:40] The third thing, I think, is to not get attached to products. This is something I do myself all the time. If you're an engineer or a designer, anyone that's working on building a product, you get excited. You put all of your effort into building something, you spend weeks working on it, you launch it to the public, and no one uses it. You think, "What if we just tweak this thing? What if we just do this?" Then you end up in the cycle where you just keep going and going and going, trying to get it to work, because you love that feature, you spent so much time on it, when in reality probably you just want to bin it. That's a very difficult thing to do. It's very hard to let go of something you just spent the last month building, but if you want to make a product really, really tight, I think that's important.

Plan the whole feature, then cut back

[00:10:26] I want to tell you how we can do this right, and this is really looking at my experiences of those three companies over the last however many years, and figuring out where did I go wrong, where did we go wrong as teams, and seeing if we can think about how to avoid that.

[00:10:46] The first one is plan. It sounds stupid, but it's true. Some people think time spent planning is wasted. This could be anything from, "We have a bunch of engineers, why aren't they coding something? That's their job. They're engineers, they should code." For engineers, if you spend 100% of your time coding, you're going to do stuff wrong, pretty drastically. Just take some time to plan. There's a bit of a law of diminishing returns, in the sense that planning forever is not going to get you much more than planning a little bit, but do it. It might just be a couple of hours on a hundred-hour project, but it's not time wasted.

[00:11:25] What I see a lot is people figuring out, "We want to do an MVP, we want to move really quickly, like agile," and you just think, how do we build the MVP? Really, what I think you should do, to make sure that you're building things the right way, is to plan the whole feature. Imagine what the best implementation of this feature could possibly be, plan out how that would work, and then cut back the scope so you come to that MVP or the MLP. I think it is a bit of a subtle distinction, but I see it happen all the time. I think it's very, very important to do, and that's how you get that lovable product. If you do an MVP, you might end up with buttons all over the place just because it works, but paring back the scope of a feature to make the core of it great is really key.

[00:12:19] Then decide on success criteria. It sounds stupid again, but decide on the success criteria before you launch, because again it's so easy not to do this. You just say, "Launch it, we'll see what happens." But then how do you know if it's successful? That's on planning.

The Uber for Business story

[00:12:38] An example of where I think maybe we got this wrong in my career: I spent a few years working on the Uber for Business team, and at the start of that project it was just the CEO, whoever, coming to us and saying, "Right, you five, get in a room. We want to build something for businesses. Go figure it out." So we sat in a room deciding what we were going to do. We realized we had no sales team, no support, no operations, none of this stuff. It was basically five engineers and a designer. Well, actually, we had a designer, yeah.

[00:13:15] We had to figure out what we were going to do. We don't have any of these other things to support us, so we have to build something that's very easy for people to onboard. We need to build something where it's all self-service and they don't have to contact support or sales. We ended up building a product that works very, very well for small companies. If you're a small company, you can onboard easily. We thought big companies were going to be a sales cycle of a year to sign them. They want contracts, they want all these bells and whistles and features, and we don't deal with any of that. We just want to build this thing that's easy to onboard.

[00:13:51] We built that, and it worked, and then the big companies came to us anyway and said, "Well, now we want to use this." We had salespeople by then, and they're like, "We want to sign the big companies, because that's where all the money is," and we're like, "That's cool." Then we got to the point where we had built the product centered around small teams, and now we needed to fit the big teams into it. At the start, we didn't build a feature that let people group their employees into different buckets and give them different rules or experiences in the product. We definitely thought about that when we started. We thought, maybe we should build groups, people are going to want that when they're bigger. But we're not focusing on big companies now, forget that, we'll tackle that later, it's going to be fine.

[00:14:35] We didn't plan out the whole feature, how the whole thing should work, and we built the system so it didn't support groups. Two years later we still hadn't built groups, because it was so difficult to put back into the system, because everything was centered around one group of employees. It took us so long to get around to building this, and that was a really big key feature in unlocking massive enterprise companies. If you want to sign Goldman Sachs onto your business, you need groups, because they don't want to manage fifty thousand employees in one organization.

[00:15:10] That's a time when just a little bit more foresight into what we were eventually going to become could have helped. It probably would have taken an extra week to build the initial version to support groups, but that's a hell of a lot better than a couple of months of work of a whole team to get groups into it in the future. Maybe, I don't know. Life is one experiment; you can't go back and see what would have happened. Maybe the product would have died. But my gut's telling me we probably should have built groups from the start.

Experiment quickly and kill ruthlessly

[00:15:40] Okay, going backwards. Experiment quickly and kill ruthlessly. As I said, it's so difficult to do, especially because you get attached to stuff, but it really does let you have a much more focused product. If you can just kill features that don't work instead of leaving them hanging around, you're going to be in a much better state. People sometimes think of negative experiments as being a failure, an experiment with a negative outcome. If you're doing an experiment and you say, "Let's see if this improves the conversion," and you try it and it doesn't, that's not a failure. You've just learned that that doesn't work, which is great.

[00:16:17] The people who take this to an absolute extreme are Google X. If you haven't seen the presentation, there's a really good one, I think it's a TED talk, by Astro Teller, the guy who runs the X division at Google and Alphabet. It's a very interesting presentation. In it he basically says that they try as hard as they can to kill a project. They try, "How many ways can we kill this? How can it not work?" They do all of this, and if it still works at the end of it, they know it's probably a good idea. If they do manage to kill it, then they celebrate. They have a party, everyone gets a bonus or promotion or whatever, and they all celebrate that they've killed an idea. That's so backwards to the normal way people work, but it's brilliant in enforcing the fact that a negative outcome for experiments is not a failure. You've learned something. You've saved yourself a bunch of time building a product you don't need because it doesn't work.

People care, but not as much as you think

[00:17:13] Then, people care, but not as much as you think. This is just a little bit of my opinion. I've just been speaking to people who are doing user research, and maybe they think differently, but I found that people really care about your product, but not as much as you do. Every day you're building your website or doing research or talking to customers for your specific product, and you live and breathe that product. Your users might come along once a month, and they're not going to notice that when you shift the button five pixels to the right something's different. That might be the biggest thing that you're caring about right now, and they're like, "Whatever, I just want to buy this thing."

[00:17:53] So people care, and you can go backwards, but don't be afraid to stick to your vision. If you have a vision for how something should work, don't be upset if you get those users who do care and say, "You moved the button five pixels to the right, and now it looks bad." You're like, "Well, this is how we want it to be."

[00:18:11] Again, from my own experience at Expensify, this happened. If any of you are familiar with Expensify, it's an online expense reporting system, and in it we had so many ways to do the same thing. Just the example of adding an expense to a report: you could either go onto your expenses page and click a bunch of them and say, "Add this to report," or you go into the report and import expenses to it, or you go into an individual expense and select the report you wanted to add it to. You could do this from all these different ways. What that meant was every user did something their own way, and that was the way that they did it.

[00:18:57] When we had the focus, the vision that we just want people to be in a report and import expenses, we took away the other buttons, and people went absolutely bonkers. "You took away this button, now I can't add the expense." "You can, you just go to the report." "I don't want to do that, I want to do it the other way." The mistake we made there was we went back on ourselves. We said, "We're really sorry, we don't want to upset you, we'll put the button back." And I was like, how can we ever build a really tight product if every time somebody gets upset we don't stick to our guns, we don't stick to our vision for the product, and we go back on ourselves? It's not that we were wrong there.

Have a North Star and watch for feature creep

[00:19:46] Have a North Star. This is sort of tied into what I was just saying: know where you want your product to be. Have a roadmap, have an idea of what it is that you're trying to achieve. Very important to do. Then watch out for feature creep, and especially quick fixes. Feature creep, if you're not familiar with the term, just means you start to build one thing, and all of a sudden you need this thing to make that one work, then you need one more thing, and you end up with just this giant mess of features that don't really work together, because you just added them and iterated. It's probably very difficult to avoid, because you want to move fast, you want to try lots of different things, you want to add things and see how they work. But if you don't kill things, you end up in this situation where you have all these features, you have nine different ways to add an expense to a report or whatever it is, and you end up in a mess.

[00:20:33] Be wary of quick fixes. An example at Fat Llama: we have this live search feed, so you can see what people are searching for and you can fill that demand, or you can learn about the marketplace. We think, okay, now we've just launched New York, and the live search is showing items in London for people in New York. That doesn't make any sense; no one's going to rent a camera across the Atlantic. So, "What if we just add a little toggle, a switch between the UK and the US?" And you think, hold on a minute. Okay, we could go with that, but if we do that, how is that going to scale out to all the other countries? Do you need to be able to filter this thing? All of a sudden the product that's so simple, "look what people are searching for," becomes this crazy mess where you want to filter this thing and search by location and toggle these different things on and off. You've just built this big, massive thing that defeats the whole purpose of what you wanted to build in the first place, the North Star: look what people are searching for.

Measure the right thing

[00:21:38] Then make sure you're measuring the right thing as well. Again, it may be a little subtle, but I think it's very important to measure the right thing. An example that I was just thinking of on the way over here, a little bit out there, I feel, is supermarkets. If you go to a supermarket, it started off with someone at the till at the checkout, going through items as fast as they can. You might think, okay, how do we iterate on this? They can do 27 items per minute through the checkout or whatever. Okay, that's a good metric: how many items per minute can they get through the checkout? You say, how can we optimize that?

[00:22:23] Right, now let's build in an automatic barcode scanner, and now you can get through 50 in a minute, and they don't have to type in the product or the price. Then you add scales, and you get to 60 a minute. Then someone comes along and says, "We can do way better than that. We can build self-checkouts, and now instead of having only four in there, we can put 12 in there," and we can get through a lot more items scanned per minute overall, with people doing it themselves. I think that's a great improvement.

[00:22:51] Then you get Amazon coming along and saying, "Hold on a minute. Why are we measuring items per minute through the checkout? Why aren't we measuring how long people take to leave the shop?" And they just said, "Let's get rid of checkouts. Let's build the technology so you can just walk out of the shop and it automatically checks you out." That's the Amazon Go technology. So if you're measuring the wrong thing, if you're too tightly focused on measuring very specific things like items per minute, or whatever it might be in your products, you might miss the bigger picture [?], and you might not think, what are we actually trying to measure here? I think that's one that's very, very common. And I think that is pretty much all I have for you, but I'm happy to take questions.