Predictive Design: How To Be One Step Ahead Of Your Customers

Pedro Marques at UXDX Community: Amsterdam. Video: https://youtu.be/3mG03Rm0Ceg

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Prediction versus anticipation

[00:00:10] We're here today to talk about predictive design. We had introductions, so I was going to skip right through them and just go to what matters. Today we're going to talk about prediction, machine learning and research. We're going to touch on these three topics in a specific order. Basically we're going to talk about what those things are, why we use them, and how to use them.

[00:00:38] Okay, so we'll start with prediction. What is prediction? It's a Latin word. A prediction is basically a forecast, a statement about an uncertain event. When we talk about AI, machine learning and all those things, we're not usually talking about prediction. We're talking about anticipation. And the difference lies in the premise that a prediction is a forecast. We don't know exactly what's going to happen. And with anticipation, we can assure that this thing is going to happen, because I know that this pattern exists.

[00:01:19] So why did I call this thing predictive design? Because I've been working with anticipation for the past year, doing a bunch of stuff. But for prediction, you don't need to use the highest power of machine learning and all the buzzwords and all the tech things and so on. These are things that you can just take home and apply. Not take home: take to work, and apply in the things that you do.

Learning: the Nest thermostat and Gmail

[00:01:42] Before predicting or anticipating anything, you have to learn. The whole basis of this is learning. So let's start over there. What is learning? It's basically the first thing we do as we're born. You've got a baby, and they're just learning how things work, and they know that if I cry, I'm going to get food, and if I throw things away, if I have a good parent, they're going to, I don't know, slap me or something. So you start learning things as you go. This is a thing that we do all the time. We're always learning.

[00:02:17] One example of that in technology is this little thing. Who has heard of Nest? Oh, there's a lot of people. Okay, cool. So Nest is called a learning thermostat, and the learning part comes from the fact that it learns about your habits and then just changes the temperature of your house, and that's it.

[00:02:36] A small story. Why am I giving this example? I'm from Brazil, like I said, and I moved here to Amsterdam in the winter. Not good. I think it was like 45° when I left, and when I got here it was like, I don't know, 10, which for me felt like -100. So what I did was just make my house feel like Brazil. I had my thermostat set at like 25. Cool, that's what Europeans do, right? Apparently not. Next July I found out that I had to pay like 600 euros more on my bill because of that. Not that good. And this thing costs like 300, so I should have invested in this before. So yeah, small story.

[00:03:17] Why is it called learning? This thing you can see is when I open the app and I see the schedule of my Nest thermostat. As you can see, it has the temperature at specific times, and it turns on and off and so on. When I installed this at home, my friend told me, "Don't just go and mess with the schedule. Just use it as a normal thermostat, and then you'll see what happens." So what happens is, I wake up, I turn on the thermostat, because in the winter it's quite cold, and I do my things, I turn it off, then I go to work. Cool. I get back home, I turn it back on again, then maybe turn it off, and then turn it back on again before I go to sleep, and then turn it off again. So just using it like a normal thermostat.

[00:03:56] And then at some point it was like this. I wake up, and 15 minutes before that it just turned itself on, and boom, the house is warm. Nice. I go to work. As soon as it knows that my phone is not in my house, it just turns it off. Bye-bye, big 600 euro bill for forgetting your thermostat on all the time, like I did. So this is just learning. It's learning like all of us do.

[00:04:17] Another example I really like is this one from Gmail. If you write that you have an attachment and you try to send it, and you don't have an attachment, it will just pop up this thing for you saying, "Hey, you wrote something about an attachment, but you don't have an attachment. Do you actually want to send this message?" They made a blog post about this, and this is such a small thing, right? But the amount of data gathering they did, and how they found it out, is insane. Basically, if someone sends an email, and then seconds after that sends an email without text but with an attachment, they saw that most of the time they had talked about an attachment but didn't attach something. So what can we do? If they fall into the prediction that they wrote "attachment," we just warn them before they make a mistake. Simple, effective, and a great use of machine learning.

What machine learning is: the apple game

[00:05:10] And then we get to machine learning. Machine learning is a term that's been thrown around all the time in the past two years or something. And then I think they grew tired of it and just started calling it AI, and now everything's AI. It's like, "I'm going to AI the [ __ ] out of this product, and it's going to sell like crazy." Most startup pitches you see are just like that. Sorry for the cursing.

[00:05:34] So, machine learning. Let's start from what it isn't. Machine learning isn't magic. This was a GIF, but whatever. So machine learning isn't magic. Machine learning also doesn't do the entire work. To be honest, it doesn't do anything at all, kind of. It doesn't do the work for you. It's not like, "Oh, I'm just going to throw machine learning at this problem, and then everything's going to work fine."

[00:06:03] So what does it actually do? We're going to play a little game. I hope everyone can participate with me. Question: what is this? "Cherry." What? Okay, so the model is off by a little bit, but yeah, it's an apple. So it's an apple. Next, what is this? Sure? Actually, it's not an apple, right? Fine. Next one, what is this? I love this one. When I do these slides, I'm like, "What's the weirdest [ __ ] I can gather and then show to people?" In Portuguese we call this caju, but I have no idea what this thing's called in English. Cashew? Mhm, okay. It's not an apple.

[00:06:53] So what we did is we basically just learned what an apple is and what an apple isn't. And that is machine learning. If I show you this thing, what is it? It's kind of different from that one, right? Do you think it's still an apple? It's an apple. It's just green, right?

[00:07:18] So the thing that we did: we showed this machine learning model, that's me, so you're basically training me, these three things, and we said, "Okay, this is an apple, this is not an apple, this is also not an apple." This is what we call a training set. A training set is something where you just tell the machine, "Hey, this is this thing, this is not this thing." Kind of like a kid: "Don't put this in your mouth, this you can put in your mouth." The machine just learns basically the same way. And this is where most of the misconception happens when you're talking about machine learning, because this is manual work. This is something where you have to go and say, "Hey, this is this thing. This is not this thing." Do this a hundred million times, until the machine can actually do it itself.

[00:08:03] So basically it analyzes the training set that you're giving it based on other things, like color. The first one was red, right? The other ones were not red. So that was a characteristic that it looks at and says, "Okay, that doesn't match what you told me in the first place, so it's not that thing." I also gave it red and green, but you also told me that that's also an apple, right? So now I know that an apple could be green or red. I also gave texture. The texture of the apple over there is quite different from the other things. So you can basically identify, "Hey, this is the texture that I'm looking for. This is not. This is not." And I also talked about the shape, because it's round.

[00:08:47] Maybe apples are not round sometimes. Maybe they fell off a tree. They're still an apple, but they're a deformed apple for some reason. That might just fall out of my model, because I didn't train it with that shape. So these things are features. I have to tell the computer, "Hey, this is an apple because it is green, because it is round, and because it has this texture. This is not an apple because, even though it's green, even though the texture is the same, it's not round." So this is basically what we're doing with the features. And in the end we have the apple, which is the label.

[00:09:25] This is machine learning. That's it. Thank you. I'm just kidding. So this is machine learning, basically. All the things that people talk about, yada yada, it happens, but it depends on how verbose you want to be about the features and about the labels and so on. These are some examples of models that I use daily: if someone is price sensitive, if someone is location sensitive, if someone is date flexible, and so on. Weird terms, I know. We're going to go through examples later.

Why: friction points and personalization

[00:09:58] Okay, so now we know what it is, right? Yeah? Cool. So why? Why do you use it? I think someone said this before, I think it was Raquel: good design means you have happy users and a healthy business, right? But every kind of service or product has several touch points. And when you design it, you take those things into consideration. Like, oh, someone is in a discovery phase trying to look for a hotel. Someone is in a discovery phase in a hotel, trying to look for a room. Someone is in another discovery phase in a hotel: they know the room they want, but they want to know which package they want to get with it. So there's a bunch of different touch points, and it's actually kind of hard to predict all of them. But we see them on a daily basis.

[00:10:48] So with machine learning, or not with machine learning, just doing your job, you can basically identify where you find friction in this product. Where are people dropping off? Where are we creating bounce? Where are people simply just booking something else? Or where do you have a lot of doubt, where they keep opening things and closing, opening, closing, opening, closing, spending 20 minutes on the page, or just spending one second on the page? So at that point we know a bunch of friction points. And the best part of it is that we know what they did before. We know what led them to that friction point.

[00:11:28] When you know what led someone to a friction point, you can actually propose a solution. But not every user is the same. Not everyone wants the same thing. So you actually need the training set part. You need to give it information: okay, when people want to do this, this, this and this, it means they are this kind of people. So then we can show something to them. This is personalization, basically. That's the word for it.

Data-driven versus data-informed

[00:11:53] But how to do it? In the beginning I said that this wasn't about anticipating, it was about prediction, because it's something that we can do on a daily basis with everything we have. I'm going to give you a tip for this. When we talk about data, qualitative or quantitative, we always fall into the same old question. A few years ago the coolest thing that you could say about your company was that you're a data-driven company. "Oh, we believe in data." That means we have Google Analytics, and that's it. And now we do all the things right, yada yada. But those things are very, very different.

[00:12:28] When you're data-driven, the main pitfall is that you might end up optimizing the wrong thing. Like, "Hey, we made the button two pixels bigger." "What if you made it four?" "Great idea, let's do it." "Now what if you make it 10 pixels bigger? And 20? And 30?" So you're optimizing the wrong thing, just because you found a data point that points you somewhere you think you want to go. And I really like this quote from Andrew Chen. If they send out the slides, this slide has the link to the blog post. He says, basically, that not everything is an optimization problem.

[00:13:03] And what you do with that is: data is just one of the tools. It just helps you fill in the gaps in the story you want to understand. The big difference between being data-informed and data-driven is that when you're data-informed, you accept that you don't know the full story. You know a bunch of stuff, but you just don't know the full story. So you have quantitative data, you have qualitative data, and what I like to call the leading instinct. It's not an actual instinct, but I think Rachel put it very well. It's basically: where do we want to take this product? Where does the organization want to go? So this is direction. It's a bet. And you don't know exactly if it's true, but there's a North Star. Having that also helps you be data-informed.

[00:13:51] The main pitfall of being data-driven is that you end up optimizing the wrong thing. And the best thing about being data-informed is this. This is a local maximum. The button example: if I just keep increasing it forever, I'm going to reach a point where it's like, "Okay, the page is just a button, guys. I think it's not good enough anymore." So this is a local maximum. It's when you reach a point where you cannot simply keep improving the same thing. And if you don't accept this curve, you're never going to reach the global maximum, which is: "Okay, let's take a step back. Why does making this thing bigger work? Oh, because people were not seeing it. What is the problem? Maybe it's because it's gray on a white background. Oh, maybe that's the problem." So you're taking a step back to look at it from a different perspective, and you can reach the global maximum.

Research methods: surveys, street tests and diary studies

[00:14:42] Why am I talking about this? Two things. Practicing empathy, another hot word in the industry, is very important, and you can spin this out and just talk to your customers. I'm going to give some examples of how I and the design team at Booking.com do these things. And I'll show you how you can apply these things to the product, no matter how much traffic you have, no matter how much, well, it kind of matters, but no matter how much money the company is willing to invest in that.

[00:15:22] Online survey tools: cheap, fast and very economical. If you want to know something about your users, remember that we talked about the friction point, and about making the button a little bigger? Why not just add a little survey there, after someone clicks that button? "Hey," frame a question, what do you want to know? "Why did you click on this? Or why didn't you click it before?" Or something. So at that point you can just intercept them and understand. Now you know that the friction exists, and you can get more qualitative information on that thing. So, online survey tools: great.

[00:15:56] Street-level user testing. The Booking.com office here in Amsterdam sits on top of a Starbucks. And for a while, I'm 100% sure that they hated us, because what we would do is basically: I'm working, I build a prototype, and I want to understand how people use it. If I have to go and invite people over and get them into the office and run a usability test, that would take ages, yada yada. Why can't I just go outside and test? But then, "Oh, maybe he's not your actual target user. You're just testing with someone that's not actually going to use your product." I can bet 10 euros that if you walk around randomly in the center of Amsterdam, 80% of those people are tourists. Therefore they're probably potential customers. So for us it's very good. We just step outside, go to the street, and tell people, "Hey, use this," or "What did you think of this?" after they accomplished the task, and so on, and gather that knowledge.

[00:16:52] Then you have usability tests, user research, and diary studies. Diary studies, for the purpose of machine learning per se, are one of my favorite things. A diary study has several ways to conduct it. This one is what's called a structured diary study. In this case, we intercept people when they're trying to plan their trip, and a recruitment agency gets them, yada yada, and they install an app on their phone. And we send them a notification every day saying, "Just asking, have you done something about your trip?" If they say yes, they go and say, "Oh yes, I did," what they did, and how they feel about it. So we know what they did, what point in their journey they're at, and how they feel about that thing. And we do that throughout the entire journey with a lot of people.

[00:17:44] Why is this key? Especially if you're dealing with a lot of people, you will find that at specific points of the journey, people might have the same friction. And what led them there? What was the pattern that led them to having that friction point? So the diary study is very, very good for that, in this sense.

Finding patterns and turning them into models

[00:18:07] And all these research methods, everything that I talked about in research, in the end, for this purpose, the only goal is to find patterns. Because when you find a pattern, then you can actually start predicting something. If a customer clicks on Y, then they hover X, it takes them to X, then they hover Y, then they buy the product. Of course, it's very abstract, but you can see the path they took until they performed an action. So if they drop off the page, and before that they sorted the list by price, and before that they had three different errors when they were trying to type the name of the city in the search, if I see that other people are doing the same thing, I can just try to fix this problem before it actually happens.

[00:19:05] And it's just design work. It's what we do on a daily basis, especially when we have a lot of data to look at. But then you can take all these things and just feed them back to a model. These are some examples of models that we have. We have a data science blog for Booking.com, so if you're really into the nerdy data part of it, you can go there and read about these things. But in the end, they're just abstractions of patterns, where we can predict what someone will do.

[00:19:33] So I have those features, like the dates you set, the location that you're looking for, your past reservations if you're a customer. I have the bounce rates, the things that you do on the page. I have the average time on page, destinations, and whatever other metric you can think of. We will record it and we will have it. And then from there we can create labels by putting people in buckets. So if I see people that are looking for the date of today, so maybe they didn't bother changing the dates, and then they search for Amsterdam, then they search for London, then they search for Dublin, that kind of means that they're just shopping around and trying to get a feeling of where they're going. This is a guess. But when you look at the data, you can see that of the millions of people that go to the website every day, a lot of people do these exact same things. And if they do these exact same things, we can predict what they're going to do.

Real examples from Booking.com

[00:20:32] It sounds very abstract. The whole point today was just so we can get to the end and know what machine learning is, know what we can do, and maybe apply some of those things without having machine learning models and the yada yada behind them. But I was going to walk you through some real-world examples, work that I did and that some other designers at Booking did, so you can see how we can put this into play.

[00:20:55] This: if you're on a hotel website and you reach the end of the list, or you're just shopping around, and then you see, "Oh, here are some recommendations for you." From a UI design standpoint, it's just a bunch of banners with cities and so on. But the work behind it is that in this case we know that for people who are looking at a specific city, a bunch of other people who also looked at it also looked for this other city, and this other city, because probably they're doing a multi-electric [?] trip. So why not just show them what we know, almost for sure, they would do next? I know that people that are looking for London will probably look for Amsterdam, and they might look for Paris. And Rotterdam is a funny one, because there's an airport in Rotterdam, so a lot of people just go there to fly to other places. So you can kind of track what all these people are planning to do based on what other people did.

[00:21:54] Because in the end we're kind of like herd animals. We do the exact same thing that other people did. Not everyone, but most people just do the same thing. I think when we were talking about this, a friend made a joke that most Dutch people, when they finish college, go to Bali, Thailand, and then there was another city in the East. It's a very normal thing to do. A bunch of Dutch kids, after they finish college, do this kind of thing. So it's a pattern. And if you can predict it, you can offer it before they have to go and do manual work to find it. This is an example. It could be, I don't know, a prejudice. I'm sorry.

[00:22:33] Another example, very simple, just a small banner. This was something that Maria, a designer at Booking, did. If you're on mobile, that already tells a lot about you. It already tells us that you are maybe looking for something now, or you're actually just shopping around because you don't have high intent. So we already know a lot of things just because of the platform. And then you start shopping around, and you see one, two pages, you change this, you change that.

[00:23:04] In the end, based on a bunch of factors, like where you're from, where you're looking, and the platform that you're using, we kind of predict, "Hey, I think Pedro is date flexible, because he probably doesn't care a lot about when he's going. He just really wants to go to Amsterdam. He just doesn't care when." So I know that Pedro will go shopping around until he finds something, and Pedro might get tired. Pedro might just drop off because it's too complex, or because of other factors on the website. So Pedro, or millions of other Pedros, might just get tired and drop.

[00:23:43] So at the moment the model looks at the features, remember, round, green, yada yada, as soon as Pedro performs a specific set of actions from which we know for sure that Pedro is date flexible, instead of showing one of the results in the search results, we just show this: "Hey Pedro, we know that you're kind of shopping around. Here are other days that you can look at, because these days are cheaper," because maybe I identified as well that you are not a cheap person, but you are price sensitive. You care a lot about price. Which is my case. Not that much.

[00:24:18] So by doing this, you're just predicting what they would do. You're just making the action way easier for them, so they don't have to scroll all the way to the top and maybe get lost in something. They're like a butterfly. So Pedro might get lost in something. You're just predicting that he would do it. You're just making his life easier. He would just go, tap this, find the perfect dates, and then he will book. The results are incredible, because you're really showing something to a specific niche, a specific set of users. If I just put this there for everyone, interaction is not going to be higher, because maybe, I don't know, Lisa doesn't have flexible dates. So why am I showing this to her? I'm just wasting real estate on the mobile page, which is already very scarce.

[00:25:02] That's one example. This is another one. This was from a hackathon two years ago. We have an image recognition system that can basically tag what's in the photo. Like, oh, this is food. This is ramen, because it has eggs or whatever. We have a blog post about this as well, on the machine learning Medium thing that we have. So in the end, basically, what we can do is, I know that after you book something, I can just send you, "Hey, are you looking for ramen in Japan? Here is a big concentration of photos of ramen." Because photos have geolocation, and when someone sends a photo as a review, we know where that person took that photo. "Hey, this is a place with a high concentration of food photos, so if you like food, you should just go there. Here is a high concentration of photos of concerts. If you like concerts, you're probably going to find a concert over there." So it's just finding what is there, and then presenting that information to the users, and giving them a little bit of control over it as well. So this is another application. And this is one of my favorites, actually, just because it's ramen.

[00:26:15] Another one is something that I did last year. I love this thing. Everyone makes fun of me, to be honest. If I know that a person, say me, is going through every hotel that I look at, browsing the photos until I reach a photo that has food, that has breakfast, and then I drop off and go somewhere else. The other hotel, same thing. I go to the rooms list, and then I start hovering over information about breakfast, like the tooltip, to see what it's about. That kind of tells you that I'm into breakfast, right? That kind of tells you that I really want breakfast. So why not? As soon as I identify that, in the next hotel you look at, I can just put that photo as the first photo you see. Easier: you don't need to browse around. Here's the thing you're looking for. You can make your decision faster.

[00:27:14] So these are just very simple examples of very complex applications. But you can find the same thing in a user test. You can find the same thing in a street test. And as soon as someone performs specific actions, you can just move things around a little bit and serve that specific person.

How you can do it

[00:27:34] So how can you do it? Track everything. If you have a digital product, track every single thing in your product. Hovers, every div. Add something like a JavaScript listener and understand what happens: people hovering there, doing this, doing this. Save all of that in your database. And then you can sit with someone that knows how to query databases and create a trail of things. If you have Google Analytics, you can just create the events in there. And then in Google Analytics, the very friendly interface, you can start checking: someone that did this event, what did they do next? They did this and they did this and they did this. So just track the [ __ ] out of your product when you can. Sometimes you just can't, but try to do this as much as you can. Find those patterns that I talked about.

[00:28:18] One of the most important things: talk to your customers. Because sometimes you only see data for things that you already have. You don't see data for things you don't have. So when you talk to people, you can actually understand what they need, what they're trying to accomplish. And then sometimes you just need to create something new based on the things that you already had. But if I just look at this skewed data of only mistakes, I'm probably just going to make it easier for someone to make a mistake. I'm not going to be making it easier for them to actually accomplish what they want to do.

[00:28:51] See beyond your product. I talked about the, Jesus Christ, what was the name? The diary study thing. I really like it because in that case, most of the time when you intercept people for them to do that task, they don't know I'm Booking.com. So a lot of those people just end up booking somewhere else. And for me that's the most valuable thing, because why did they choose that? Why did they drop off at some point and go to someone else? Or why did they just hate something that I did? So talking to people and seeing beyond your product is insanely important. Especially travel, because travel is not just about a hotel. Just like it was mentioned before, it is the flight, it is what you're going to do, it is how you're going to move around. There's a bunch of things that you do that we can help with, because we have the data to do it, and we can predict things and just show you relevant stuff, not just try to push whatever we want to push.