Five Ps: Lead Teams & Land Products

Paolo Malabuyo at UXDX Europe. Video: https://youtu.be/x3VSHFLzoSs

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Looking for a better way to build products

[00:00:04] Thank you, good morning. I know it's the first morning after a late night out. I had my coffee, so hopefully you did as well. My name is Paolo Malabuyo, and I lead the UX team for Google's YouTube video ads. I'm here to share with you this framework that I've been trying to put together for the past couple of decades. I'm still trying to make sense of it, but hopefully you get something out of it.

[00:00:33] I've been doing this for about 23 years, for a number of different companies and brands, some of which you may be familiar with, and I've gone through a lot of different product design and development processes. If you go off and google "product development process," you might see a picture like this: waterfall, agile, lean, design thinking, et cetera. We're all trying to figure out different ways of doing what we're doing better. Often it ends up looking more like this: there's some central idea that starts off, we get fueled by coffee, and it always ends up in an argument. So I've got my coffee; I'm ready for a good argument.

[00:01:27] Your plan rarely ever matches up with what you actually end up doing, but we all try to plan anyway. With all the time I've had working at different companies on different products, I've been asking myself: there must be a better way to do this. I think there might be. I hope there is. And I'm learning a lot from what I've done and what I've seen other people do.

[00:02:00] A quick caveat, one of my favorite quotes: good judgment comes from experience, and experience comes from bad judgment. So hopefully you're all the beneficiaries of my bad judgment. I'm going to share with you the five Ps. By the way, the reason why it's the five Ps is because my memory is horrible and I need a mnemonic to help me remember anything. My path hasn't been a linear progression. There have been all these twists and turns, and I've picked up a few things along the way, and hopefully you can see where I've come from to tell you where I'm going.

The five Ps

[00:02:46] First, principles: the fundamental values and beliefs that guide behaviors and decisions. People: the team that embodies those principles, and the users or customers they serve. Priorities: what's important for the business and the people. Process: just enough structure to enable the pursuit of those priorities. For some of you who love process, you might be surprised it doesn't come first. And product: the outcome, when evaluated against the principles. Is it good? They are in order. They're not just randomly sprinkled in this order. I actually believe it goes from principles all the way to product. So let me talk to you a little bit about my experiences that led to these.

[00:03:37] Let's start with principles. I worked on Xbox 360 from 2004 to 2007. I was the lead interaction designer and information architect for the console, and I was the UX manager for the team until I left in 2007. We had these design principles when we were designing this product: open, clear, consistent, athletic, and mirai, which is "future" in Japanese. The not-so-well-hidden sixth value is Occam. You can see I like these mnemonics. It refers to Occam's razor, the basic idea that the simplest explanation is usually the correct one. All of us who worked on Xbox 360 around this time kept referring back to these principles. They actually helped guide the decisions we were making throughout the design and development of that game console.

[00:04:39] Mercedes-Benz, the venerable German luxury automotive brand. I led the advanced UX design studio in Silicon Valley from 2013 to 2015, and our guiding principle was the design philosophy that the head of design set out for us: this idea of sensual purity, being the pure expression of modern luxury. If you think about design as a practice that spans the spectrum of art and science, this hews pretty strongly towards the art.

[00:05:20] Moving on to people, the team and the users. Leadership matters. At Xbox, Mercedes-Benz and Netflix, I had the opportunity to work with some very larger-than-life leaders, and for better or for worse, leadership sets the tone: the behaviors, the decisions that they make, how they do it. Everyone else will conform and follow their lead. At Netflix, where I was from 2015 to 2017 (sorry, there are a lot of points along the path), and you can find these on the web, these are the values that Netflix looks for in the people who work there. They're very explicit about it. They say, this is what we expect people at Netflix to be like.

[00:06:22] Now, looking at the users or customers side, I learned early on how valuable it was to be able to tell the difference between what people do and what people say. At Xbox 360 we used all these different methodologies to be able to tell the difference. We depended a lot on usability testing and heuristic evaluation as the primary methodologies, but we used a number of different ones throughout. In 2011 I joined Zynga, where I led a team from 2011 to 2013, and I got exposed to this missing dimension: understanding the difference between qualitative and quantitative data, to inform strategy and to inform decision making.

[00:07:20] At Mercedes-Benz, a number of different methodologies. You'll notice there's no A/B testing. It's kind of difficult to say that dashboard design A caused fewer accidents than dashboard design B, so we didn't do that. We did a lot of testing in simulators to get us as close to that as we could, but we did do some quantitative work in the form of surveys. And then at Netflix, we had each quadrant well represented. I learned that you have to use the right methodology to get the right kind of insight.

Priorities and process at Xbox, Mercedes-Benz and Netflix

[00:08:02] Priorities: what's important for the business and the people. For Xbox 360, it was getting it ready to ship, getting this thing on the shelf and making sure it was there for holiday 2005. It needed to be able to play games, it needed to be able to connect to the internet, and it needed to have updatable software, because that's what enabled all the updates later on. So we didn't have time to putter around with anything that got in the way of hitting those three priorities.

[00:08:37] At Mercedes, for this particular project, what we called the F 015 Luxury in Motion autonomous concept car, we were going to unveil it at CES in 2015. We needed to hit that date. It needed to be on stage, and it needed to run, because that was the idea of having that project: to show it to the rest of the world on the world's biggest tech stage. For Netflix, it's relatively straightforward. The priority was to do what we need to do in the product to drive subscriptions up, because we thought that's really the best proxy for user happiness. Are they signing up, and are they staying as members? Everything we did was in the service of one of those two things.

[00:09:33] Next, process, way down the line. Just enough structure. In some places it may be best to be much more process oriented. When I worked at Mercedes-Benz, I learned that when safety is a big issue, process really does matter. But if you're in software, in a lot of the high-tech work that we're doing today, I don't think that's where you need to start.

[00:10:05] For Xbox, it was pretty straightforward. It should look quite familiar: define, iteratively design, prototype it, evaluate, and ship when you're done. As with many software development processes, it all starts on a whiteboard. I don't know what we would do if we didn't have whiteboards. You go from whiteboarding sets of high-level architecture to some detailed wireframing, then prototypes to actually test it. Yes, we actually prototyped the whole thing in Flash, just so we could figure out if it was working and if it was usable. And then you ship once you pass verification.

[00:10:43] For Mercedes, car design and development is a very traditional process. It's been going on for a long time, and there's a lot of institutional muscle memory. Often it starts with a very beautiful concept drawing of what this thing might be, and even today there are still a lot of very traditional art techniques used. You would have full-scale clay models being worked on by craftsmen and artisans in a workshop. Then you iteratively, and sometimes that's too hard a word, refine the details to the nth degree in the studio.

[00:11:36] This may end up in a real concept car. These are amazing pieces of industrial art, from the outside to the inside, and they're pretty fully realized. This was the concept for the S-Class Coupé, stunning inside and out. If it passes muster, then you go through and actually productize it. If any of you have ever had the chance to visit a car factory, or if you haven't, do so. It'll give you a new sense of appreciation for what it takes to actually build one of these and have them rolling off a factory floor and shipping around the world. And then, if you're lucky, there's a car at the end. For a lot of them, it's a five to seven year process.

[00:12:31] Now imagine that's the institutional muscle memory you have, and you're trying to develop software as well. You've got one side that knows how to ship atoms and the other side that's figuring out how to ship bits, and it's a very different approach. This transformation is a difficult one for a lot of traditional manufacturing companies.

[00:12:56] And then there's Netflix. One of the ways Netflix does what it does, which it's very famous for and very strict about, is how we improve the product by validating it in the wild. What I'm talking about is A/B testing. A few years ago we wanted to update and improve the Netflix.com website. We started with a couple of hypotheses. One was that richer assets would make the experience more interesting, and the other was that a flatter experience would too, and that both of them would lead to increased streaming, meaning people would watch more, and retention, meaning people continue to be members month over month.

[00:13:47] We designed and iteratively designed different versions, and then we built fully functional, fully realized versions of these experiences that we ran out in the wild. A lot of people didn't realize they were actually inside the tests. It's double-blind, the scientific method applied in product development, to see whether streaming, retention and those types of things actually go up. After going through millions upon millions of people and seeing that kind of data, we'd find the winner, and then we'd go and productize that. The Netflix.com experience you might see today is actually very similar to this, and it's constantly being improved upon using the same exact A/B testing methodology.

Product: is it good?

[00:14:41] Lastly, the product. Is it good? For those of you who aren't gamers or didn't know, this is the Xbox 360. It's a gaming console that sits next to your TV for you to play games. When we look back, we're able to say we did some amazing things. It was the first console that shipped with wireless controllers out of the box. Amazing that that wasn't the case before, but that's what it was. High definition and internet connectivity straight out of the box. A system interface that was accessible from within the game with a dedicated button on the controller. If you play with the Sony console or the Nintendo console today, that seems like, "Oh, well, that's the way it's done." It wasn't until we did it with Xbox 360.

[00:15:34] And we created the idea of the universal profile and achievements that span game to game, yet another thing that you take for granted now in the gaming world, but we actually invented it back then. It's great when you create a product that shows up on the cover of Time magazine. Unfortunately, they decided the red lights just looked good, when in fact that's an error code, FYI. And they made Bill Gates look like a Borg. But what really matters ultimately is sales, and this proved that Microsoft is not only willing but able to compete at this level with the established players.

[00:16:30] Mercedes-Benz: this is the Luxury in Motion F 015, the most unwieldy name for a project. This is not a render. This is actually being driven about an hour outside of Las Vegas, where we closed off a piece of highway and were blocking traffic for a while. We designed the experience inside and out. These are large touchscreens on the saloon doors, and they were actually working. You can see the guy was pressing a little hard on them, because we needed to make it work, but it's mostly working.

[00:17:12] Among the things we were most interested in exploring was: in a world of self-driving cars, how does a car communicate to the people outside of it, not just inside? How do you make sure you don't get hit by a car at a four-way stop? Now, you look at the driver. What if there's no driver? Here we decided: what if the car's lights would project a crosswalk and tell you to cross? This is filmed, the way it works in action. You can see it actually had an actual laser pointer to draw the crosswalk in front of you. That wasn't CG.

[00:18:02] Why would we do a concept car like this? Well, for press. What's a concept car for? It's a signal to the world that you have some interesting ideas, in this case the idea of the autonomous car: that Mercedes-Benz, a company that's been around since the 19th century, is very much as much a leader in the 21st, thinking about autonomous driving, and getting the kind of press that gets the world to stand up and take notice.

[00:18:32] With Netflix, here's something not so new now, but new when we finally unleashed it. We wanted to reduce the amount of browsing that people would do in the interface, and we introduced the idea of these auto-playing previews in the UI. Not just because we like previews; it's because we wanted to see if this would actually reduce the amount of time and effort people would take looking for something to watch. So using that same A/B testing methodology I mentioned earlier, that's exactly what we did. And if you haven't watched Stranger Things, you need to, by the way. I know it's Friday night, but go off and watch Stranger Things in seven days. An amazing show. Ultimately you can see how that's reflected in the subscription numbers. I think it's kind of working so far, and Netflix continues to do amazing things in the digital space.

Applying the five Ps at Google: principles and people

[00:19:41] So, the five Ps: principles, people, priorities, process and product, in that order. Now let me tell you a little bit about how I'm actually using this in my day job today. I work at Google in Silicon Valley, more specifically in the YouTube video ads business. How many people here have used YouTube? I should have asked the other question: how many of you have not watched YouTube? If you haven't, you should try it out.

[00:20:15] I'll start with principles. For Google, it starts here. Internally we refer to these as the three respects: respect the user, respect the opportunity, respect each other. All sounds great, but one of the challenges with principles is that it's relatively easy to talk about them at the aspirational level. Part of our job as leaders is to translate them so that people can actually operationalize them. One of the ways I do this for my team: these are the principles I've set out for my team for 2018. I unveiled these to my team back in December last year: design for trust; clarity, simplicity and insight; and innovate at scale. Of course there's a lot more detail under each of these that I work through with each of my leads: how do we want these to manifest specifically in the products we're developing?

[00:21:16] Another principle I try to explain to everyone: perfect is the enemy of good, by everyone's favorite, Voltaire. I think a lot of people, myself included, in UX sometimes have trouble letting go of the perfect: the perfect process, the perfect idea, the perfect design. There's no such thing as perfect. It's all about making the right compromises, and this is a principle that I'm trying to make sure I'm able to live on a daily basis and get my team to do as well.

[00:21:57] There are many disciplines involved in product design and development today. Often people talk about the three primary disciplines as engineering, product management and UX. For those of you who do this, hopefully this sounds familiar. A common metaphor that I hear used is that these are the three legs of the stool. Unfortunately, in a lot of places this is a very rickety stool; not all the legs are the same length. I prefer the metaphor of a big tent. What I mean by that is not only that it's a tent where everyone gets together. No, I'm not talking about the warm and fuzzy feelings about that.

[00:22:37] To raise a big tent, you need a nice tall tent pole, and how do you do that? A tent pole is held up by three or more very strong anchors. You can refer to each of those ropes as engineering, PM and UX. What do they have? They're deeply anchored in their disciplines, and they're pulling very hard, equally hard, on everyone else. I think you need healthy tension. We in UX need to be pulling as much as we are being pulled, and we need to have the confidence to do that, and we do that by being very strongly anchored in our disciplines.

[00:23:17] Let's talk about the people. There's a term that is out in the world, and even internally, called "Googley," as a quality of what it means to be a Googler. The three ways this manifests for me are these. One, we take the moral responsibility of what we do very seriously. We may not always agree on everything, but we care about the moral responsibility we have to the world and to each other. The second is that whatever it is we're doing, we'd rather make sure that we do it together. So we go out of our way to argue and to talk, so that we can all agree that this is the right thing to do, and then we go execute. And the third thing is that it's okay to have some fun while you're working. Life's too short to take it way too seriously.

[00:24:22] These three leaders happen to comprise my management chain. Like I mentioned earlier, leadership matters. They set the tone for me, and I set the tone for my team. Here's an example of us taking an afternoon off to go have fun. We had a team summit earlier this year. We have an office in Zurich, and we all made some chocolate at the Lindt factory, where I think I developed diabetes because I had way too much sugar.

[00:24:58] I also have this set of qualities that I look for in people on my team: curiosity, imagination, communication, empathy, rhetoric, open-mindedness and synthesis. Because my memory, again, is not very good, the only mnemonic I could find was CICEROS. It's not directly related to the philosopher, but it's a way I can remember each of these qualities.

[00:25:19] This is my team's purpose: to enable a healthy YouTube ecosystem where viewers watch great free content; creators can express themselves, create a community and monetize their content; and advertisers can effectively reach the viewers interested in their products and services. So we always keep in mind what it is that we're doing for viewers, creators and advertisers. You saw this before: we've got a very flexible way of looking at all the different methodologies, to both inform and validate the work that we're doing. One of the ways this manifests is personas. These are, in some ways, the way we represent one of those sets of users, our advertisers.

Priorities, process and product at YouTube ads

[00:26:05] Priorities. I can't go too deeply into the specifics, but we use OKRs, which stands for objectives and key results. The objective is what it is that you want to do, and key results are quantifiable measures of success. It's not good enough to say we want to make the product better. How specifically are you going to do that? What are you doing, and how are you going to measure that you actually did it?

[00:26:30] In terms of how we prioritize our team's work: operational work is something you need to do to keep the business going, whether it's a few hundred dollars that you need or the few billion dollars you need. These are the day-to-day things that tend to keep most teams busy. But we try to carve out time specifically for foundational, experimental work that might fail, where you may not see the value for a year or more. Because I feel like if you don't do that, you're not guaranteeing your future.

[00:27:06] For process: my team does this end-to-end experience in the video advertising space. We serve the needs of all the different kinds of users I talked about, and I've structured the team both along the products and along the different phases of the experience. In this way, they can do whatever process they want. We don't say you have to run this way. It's up to them to figure out the best way they can work to achieve the goals they set out for themselves. We're also spread out around the world, so it's hard for us to dictate one way to do it. One way we make sense of it is we created this dashboard, which is anonymized here, where we see who's working on what, who's working on too much, and what the status of all these different projects is. But we don't dictate how they go about doing it.

[00:28:04] Lastly, the product. There are lots of products my team works on. I'll just share one, a product called Director Mix, and there's no better way to explain what it is than by showing you a video.

[00:28:15] "You'd pay more attention to this ad if I wrote it for you. But who are you? Is this you? How can I write an ad for all the yous out there? Enter YouTube Director Mix. With Director Mix, you can create one generic version of your ad and set elements to be swappable, things like text, images, sound, video and even image mapping, giving you the ability to completely customize your creative to reach specific audiences. Say you're advertising this and you want to gear it towards someone who is about to get married, or reach a new parent, or swap things out for people who are into this, or maybe that.

[00:29:04] "Even after launch, you can create new versions of your ad. Say the product's packaging changes, it's time for a price promotion, or you want to localize a version for Brazil, Germany or any other market. You can go back into the project and easily update it, giving you the ability to assemble the perfect spot for any viewer. So one ad for this could look like this. Personalized video doesn't have to be daunting. YouTube Director Mix: just right for you."

Director Mix in action

[00:29:37] So how does this manifest for real? An early version of this tech was used by Campbell's Soup in Australia. "Excited for the new season of Orange Is the New Black? You'll be served this. If you always need 'Single Ladies,' you'll see this. Trying to make sense of world politics? Can't let go of 'Let It Go'? Looking for fail videos? Searching for Carpool Karaoke? Catching up on UFC highlights? Need cooking advice? New Star Wars trailer? We even created daily pre-rolls related to the world's trending topics, like Brexit, the Atlantic [?] soccer team and Pokémon Go. Imagine creating each of those manually. It's almost impossible."

[00:30:30] They created 1,700 different variations of this ad that served up a different one depending on what you were actually looking for on YouTube. The results were kind of good: a 24% lift in ad recall and a 55% increase in sales.

[00:30:45] "Leading up to back-to-school, Kellogg's turned to Director Mix, a new video builder from Google, to introduce the concept of the Right On Wrapper [?]. Director Mix enabled the creation of more than 110 custom six-second ads, each designed as a specific response to a particular video on YouTube."

[00:31:07] Similarly, in another campaign, you'll find that contextually (I'm having problems with some of these longer words) contextually relevant ads increase brand favorability and purchase intent in significant ways. In this particular case, a 2x increase in brand lift and a 3.7% increase in sales.

[00:31:34] Last example. "To drive awareness, we first served a 15-second pre-roll. Next, we presented every sale product, each with its own six-second bumper. 'Myer [?] six-second sale is on. Click for this incredible price. It's gone in three, two, one.' With literally hundreds of sale items, traditional production techniques would have been incredibly impractical in a five-week time frame. Using Vogon, we were able to automate the process. Vogon was linked to each user's profile. They were retargeted with a sequence of six-second ads, and once they'd seen the offer, they'd never see it again. In partnership with Google, we created the first ever clickable bumper and proved that six seconds was plenty of time to drive results."

[00:32:17] Yes, I just had you walk through a whole bunch of ads. Thank you very much. So to wrap it up: the five Ps in play, principles, people, priorities, process and product, in that order. Because I don't think you can start without principles first, and then you make sure that whatever it is you have at the end is actually good. Thank you.