Design for Simplicity in a Complicated World

Hi, I'm Ryan. Today, I'll talk about the Culture of Experimentation we have at Priceline that allows us to fail fast and learn equally as fast, but do so in a way that has enabled us to be value-driven and customer centric. There are more choices today than ever before. The average person makes 35,000 choices daily. This has 2000 decisions per hour or roughly one decision every two seconds. Here's a fun fact, around 227 of these decisions are food-related and the paradox of choice, why less is more Barry Schwartz makes a case that eliminated consumer choice can reduce anxiety for shoppers. There are a few key pieces to this. One is choice overload. If a person has presented too many options, he or she is actually the less likely to buy. The fear of choice. People have a fear making the wrong decision, which grows as more options are available. And this all leads to decision fatigue. The more decisions we make, the less rational they become. The struggle here is real. While this doesn't mean twice as bad, it does mean we have to be smart what the options we present. Today, we aren't going to focus on the psychology of choice instead. Let's focus on what we can do as product people to ensure that we are moving as much friction as possible when it comes to having our customers make important decisions. The number of choices someone has to make when shopping is absolutely mind boggling. This is true both online and off. Think about it. How many times have you been in store and felt completely overwhelmed? How about shopping online? A common mindset is the more the better. It's not only the number of choices, it's how we present those options. Think about the same in store experience and online experience. We just talked about the feeling of being overwhelmed. What could have been done to make the purchase decision easier. Could anything has been done in terms of messaging content, layout replacement? We should all have targets that we're shooting for. These targets can be KPIs, desired end states, or even generalized. Regardless, there's likely to be a piece of this that translates to something that people see and experience. We've talked about choice, meaningful interactions help steer a customer in the right direction. Layouts tell a story. Content is credible. There is a clear balance between too much or too little. The only way to get this right is by testing, learning, and adapting. Let's talk about how we give our customers what they need by continuously optimizing choice, interaction, layout, and content. There are roughly 1 million people that touch Priceline products every day. This is a lot of people. For each one of them, we intend to provide a positive, meaningful experience. Our mission is to be the best travel deal makers in the world. It's important that our travelers feel our sense of urgency and living up to our room, we tested a high velocity at any given time. There could be up to 100 tests running when you visit Priceline. In other words, if you're sitting next to a friend and browsing the site or app, it's very possible, you both will see slightly different variations of the same thing. Constant experimentation allows us to learn quickly. With these learnings, we can make informed decisions about what to do next. What to do next always needs to be centered around the needs and wants of the customer, regardless of what your business does, nothing is more important than making sure you are delivering something valuable. Value, is it the only about something you can touch or see in the moment? It's about the entirety of the experience. It's about making sure your customers want to come back. Think about your customer journey, some questions we need to consider. What does our traveler do on their first visit? What type of deal are they looking for? How does the purchase journey change after the first booking? What is someone most likely to do next after they select a hotel or a flight or a car? If someone is looking for a hotel, what are they most interested in? How about a flight or a car? These are just a few of the questions we need to ask ourselves. Within each of these, there are countless more questions to ask. We need to continuously learn. The journey never ends. The best way to make progress is by testing everything often. Great quote by Jeff Bezos, "Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day." Jeff Bezos correlates Amazon success with the number of experiments they run. While we're talking about Jeff Bezos and Amazon. Here's another good one. "If you doubled the number of experiments you do per year, you're going to double your inventiveness." What I like about this, it's another way of calling out the significance of being driven by experimentation. You can't learn if you don't try. Sometimes it's the unexpected things that turns out to be the difference maker. Before you can really get into experimentation. You need to have a culture in place that allows us to happen. You can have the best strategy in the world, but if your team doesn't have the desire, it'll be nearly impossible to win. In other words, the team needs to share the passion and enthusiasm, otherwise, the strategy will never work. Mission-based teams quickly respond to evolving opportunities. This like culture as a key driver of successful experimentation. One big ingredient of being mission-based is being cross-functional people with different working backgrounds, bring different perspectives, all of which need to be well, all of which are needed to be innovative. It's not about having creative or strategy and your title, the best ideas come from a team of people who share the same passion and enthusiasm for improving their product. This sounds overly simple, but there some truth here, any important metric can be tested and improved upon testing conversion. For example, it can be as simple as trying different variants of copy on a call to action or learning what performs better or a single page checkout or a checkout that has multiple steps. Want more email signup, you can see what works best - a modal, how long had the modal opens, whereas the modal, the right thing to do at all to capture email sign up. It's also important to look at trends that could have an influence on customer behavior. And we're going to talk more about the student. If you have physical locations, spend more time in store to see what people are doing, what questions they are asking, whatever your business model may be. Interview customers, talk to support, listen to your data, look beyond the obvious. During COVID, it was important we understood our market. People were traveling for a different reason. Things we would normally test had to get set aside. What questions would someone has when looking for a hotel? How clean might this property be? Adding cleanliness, badging and messaging, and helped with confidence. We had to understand our market and know our trends. People still had to fly. Adding messaging around COVID safety with the tool tip was helpful. This was a winning test. COVID safety detail. It turned out to be useful when looking at flight details, this was a winning test. We learned that plans change during COVID and people look for flexibility. Making flexibility, a promise filter help travelers feel more comfortable. This was a winning test. These examples are all small tasks that had a big impact. None took that much work and all were based off what our travelers needed in the moment. This is important just because a test wins doesn't mean the experiment is over. Winning tests make money, losing tests make you smarter. The lean UX cycle is a great way to think about getting iterations into the world of speed. If you're going to run a lot of tests, you're going to need a lot of ideas. And remember the best ideas come from a cross-functional team. The quicker you can get an idea into an experiment and think of an experiment is another way of saying MVP. The better the story is simple. Get your ideas into the wild. Look at the data, learn what's working or not, and make informed decisions about revising or pivoting. I highly recommend outcomes over outputs as a great quick read. Josh Seiden does an excellent job talking about setting goals and creating outcomes. Outcomes are changes in human behavior that drive business results. Outcomes are what we hope our creations will achieve. Outputs are what we will create in order to achieve our desired outcomes. Priceline has a product called price breakers, which launched in 2019. Let's talk a bit about how this product hit the market and evolved. Some of you might remember William Shatner as the Priceline negotiator. Aside from the brilliant ads, the negotiator of Priceline, we're known as a way to name your own price. Overtime, this concept of bought into express deals, which continue to be an excellent and popular way to get an amazing deal on a trip. Express deals are known as an opaque. Simply put you're getting a great deal that is relevant to your search on a hotel flight or rental car. The caveat is the actual product is revealed after he booked. For some, there is a missing element of control over what they will get. Another great deal is it the price breaker. With the price breaker, you see, for example, up to three hotels that are relevant to your search, all are a great price. Once you book, you find out which product do you get, one big difference between price breakers and express deals is what the price breaker, all your options are in front of you. There is less element of surprise. This is known as semi opaque. We launched fast. The buzz was great. We were happy and our travelers loved it. This doesn't mean we were done and that we were ready to learn and make it better. Let's talk about price breakers in the context of outcomes over outputs. The impact we are looking for where the objective is to offer travelers a different way to discover hotels that meet their needs while increasing exposure to available bookings. The outcome or result is to create another compelling mechanism for deal delivery or increased conversion by a certain percent. Resources activities and outputs are all things that happen within discovery and output something we need to create would be to add a layer of transparency and to do a selection, an activity something we need to do would be enhanced traveler confidence. They are getting the best price and quality product. Resources or what we need. Let's just say, we'll start with the mission-based team. You need to test your outputs to get to a desired outcome. During discovery, it's important to think about leading metrics, these inform how to produce a result. Lagging metrics measure what you achieve outputs need to continuously be tested until they produce the desired outcome. Some things to consider, spend as little time in discovery as possible. Fast and cheap is better and it is okay to fail. When you think about outcomes over outputs, there are certain questions you want to ask along the way. For an objective, what is the compelling end goal? In our case, offering travelers a different way to discover deals that meets their needs while increasing exposure to available bookings. A question for an outcome might be, if we're succeeding, what will people be doing differently? Travelers will be purchasing better deals at a higher rate or what action might we observed that predicts the behavior we want. Higher purchase rate for customers with a lower tolerance for risk. For an output we might want to ask, what can we do to make that action happen more frequently? Display the names and highlights of three potential hotels which can be featured at the listings level. This is our price breaker MVP . After lunch, we had a lot of questions asked. Does it seem too much like an ad? Can it look different than other listings? Are people looking at reviews or price? How about the hotel address? Would a map help?
Do we need to explain more early on?

This is where we are today. As you can see the display as it is today including more detail than what we launched with. We're mindful not to add clutter or make an already difficult decision even tougher. To hit our desired outcome, it's important that we ensure our travelers have what they need to make a confident decision. We can only do this by talking to customers, looking at data and working as a team to ask the right questions that inform iteration. Iteration allows us to try new things to continuously build a product. Sometimes we win, sometimes we lose but we're never bashful about trying something out. A few very important takeaways. Talk to your customers. Know what your data is telling you. No sacred cows or hippos. Test big and test small. Winners make you money, they also make you smarter. Losers always make you smarter. And finally be customer centric. Customer centricity starts with understanding the needs of your customers. Customers are hard to predict instead of predicting work towards outcomes, do that by having a strong culture of experimentation, but know that cultures are hard to change. Good luck. Thank you.