Design Thinking Process And Strategy For A New Product

Roberta Virzi at UXDX Community: Europe West. Video: https://www.youtube.com/watch?v=o7RLfY5HDkQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction

[00:00:00] Hi everyone, my name is Roberta Virzi, and I'm going to talk about design thinking process and strategy for a new product. I'm currently a senior UX designer and team leader at Booking.com, which is one of the world's leading digital travel companies, making it easier for everyone to experience the world.

[00:00:23] As you might have noticed from my British accent and the fact that I can't say the "th" sound, I'm Italian. I'm from Rome, but I moved to Amsterdam 4 years ago to work for Booking.com. And you might think that when Italians move to northern Europe, the things they miss the most are the sun and the food, but actually no, it's that.

[00:00:46] Jokes aside, what does my current role mean? It's a hybrid role, as individual contributor and manager. But as you can see from the timeline here, I didn't start from this position. I developed the skills that helped me to grow as a leader over time, with experience. And that's why today I will walk you through the steps to implement a product that allowed me to step up and gain experience in that direction in a big company.

A quality rating for vacation rentals

[00:01:18] In my team, we faced this problem. There is a star rating system that is used worldwide to classify the quality of hotels. This hotel star system doesn't apply to vacation rentals, so they don't have a similar rating that defines quality. Also, stars is one of the most used filters on our platforms. When guests filter by stars, they can't find and book the variety of accommodation types that Booking.com provides.

[00:01:54] To solve this problem, we created a first-of-its-kind machine learning system that defines the quality of vacation rentals. And when I say vacation rentals, I mean homes, apartments, bed and breakfasts, and all the other property types that are not hotels. This rating is based on factors that are important for the guests, including facilities, size, location, services, established through user research and guest booking behavior. And the effectiveness of this system was validated with massive A/B tests.

[00:02:30] So why is this so important for my company? Booking.com is well known in the industry for having a full range of hotels, but we actually have almost 25% of accommodations that are not hotels, but apartments, homes, bed and breakfasts, etc. My team takes care of the experience of the guests that find and book these different types of accommodation.

Pain points for guests, partners and the business

[00:02:59] So back to the product. We basically have three main parties that benefit from it: guests, partners, and the business. What are their pain points? On one side, guests are not able to compare vacation rentals based on a quality indicator. For instance, how would you compare these two properties in terms of quality? You can look at other factors like photos, price, amenities, user reviews, but there was nothing that differentiated one property from another in a similar way to star ratings for hotels. So guests had to spend time looking at different features to create in their own mind the quality of the property they were looking at.

[00:03:51] Partners, which is what we call the property owners, get excluded from our search results page, which is the page you land on when you search for accommodation, when guests filter by stars. So how could partners achieve more visibility for their property? And therefore the business misses an opportunity to get a booking, by not showing the full range of accommodations that we provide.

[00:04:21] So we wanted to better support guests in their decision-making process, to find a property that fits their needs. And we wanted to support partners to get visibility and to set the right expectations for guests. Now, you may think this looks quite straightforward, but I can tell you it was not. I faced many challenges, and I had to come up with a communication and UX strategy. And now you will understand what I meant at the beginning when I said that this project allowed me to grow as a leader.

Challenging a business-driven idea

[00:04:58] Let's go back in time to the beginning of this project. I showed you before this win-win-win framework. Scratch it. This didn't exist at all. This project started from a business need. So when I was told about this idea at the beginning, I challenged it, because I couldn't see the value for our users. And I asked, "Where does it come from? Is that something that we learned from user research? Is that a guest pain point or a need?"

[00:05:31] I had discussions back and forth with my team members, because I initially couldn't find the value for the users. Since everybody has a voice at Booking.com, independently of their role and their seniority, they understood my concerns, and they gave me the time to analyze the facts. So I started writing down what the value would be for the guests. And even though my team worked on a customer-facing product, I had to find the value for the partners as well. So I came up with the win-win-win framework that I showed you before.

[00:06:07] But I wasn't satisfied yet. There were a lot of questions that my team needed to find the answers to. What if the guest doesn't need it? What if partners leave us because they don't like the rating that we assigned to them? What might be the legal implications? Are we actually allowed to build this product? How do we create this machine learning model? How do we create something scalable for all countries, something that is objective and that we can experiment with?

Facing hard conversations with senior designers

[00:06:39] In the meantime, others in the company heard about us developing this rating system. And I still remember one day, one of the principal designers booked a meeting with me for the day after. There was no description in this meeting, so I immediately got very nervous. I understood that something was going badly. And you have to imagine that at the time, I was neither senior nor team leader, and I was close to promotion.

[00:07:09] So when I entered the room, he told me, "Look, Roberta, I know how you usually work, so I want to inform you that there are principals and seniors worried about your team building this product." And I was like, mamma mia, how can this be? My career is over. I will never be promoted by these people. Maybe something similar has happened to you, and you can relate to it.

[00:07:34] After being a bit desperate, I decided to face conversations with them. And I think the important point is how I faced this conversation. Since there were people external to my department, I spent time writing a presentation that listed out everything I described to you before. And I wasn't defensive; rather, I was very eager to get their feedback and improve the team plan. So I flexed my communication style, trying to understand how they preferred to have this conversation, trying to understand what was important for them. And I reassured them about the fact that we were committed to thinking about partners as well. And I also committed to keeping them updated over time.

Understanding: competitors, guests and partners

[00:08:26] I've described to you the UX strategy, but now let's deep dive into the design process of the team. We spent a lot of time understanding, as is usual for a new product. We started out with a competitor analysis. We wanted to see what others were doing on this topic. Do they have a similar challenge? And if yes, what are they doing to solve it?

[00:08:57] And this was already the first difficulty that we explored, because we found out that there wasn't something the way the team was thinking about it. So what we did there was an audit of similar programs of competitors, and we went through forums to gather more data and check if there was actually a need, from the partners at least. And we were surprised to actually see that users felt they were missing something like that.

[00:09:33] At this point, an important step was to validate the value for the guests. We also wanted to understand their mental model, what they were associating quality with, and where in the user journey quality was a decision-making factor for them. So what we did is we ran a focus group with several guests, and we found out that quality is associated with different factors, such as specific amenities and services. And actually, guests would find it valuable to have an objective system like this in place.

[00:10:18] Next, we needed to do the same for the partners. We wanted to validate or invalidate the value for the partners as well, and we ran user research with them too. We found out that partners were actually excited to have a system similar to hotels. And we understood that they also wanted to know how to improve their assigned rating once it was established.

Exploring: the model, the icon and alignment

[00:10:49] Afterward, it was time for us to move to the exploring phase. In this phase, the first challenge that we got was the product definition. What do we base the machine learning model upon? And how do we build something scalable globally? What we did there was base the system on what we heard from the user research: what are the features that are most important and associated with quality for the guests. We grouped them and we weighted them. And our algorithm weighs the factors differently depending on the location of the property. As you can imagine, for instance, air conditioning might be something very important in Bali, but not in Norway. So that's the difference.

[00:11:46] And then it was time for the visual. The visual was very challenging, because we had the requirement not to use stars, for transparency. So what other icon that is not stars can be associated with quality and match the guests' expectations? And also, where and how do we show this? And where and how do we explain to the guests and the partners what the system is based upon? What we did there was test out different icons and different placements. We tested them using moderated user research. Later on, we validated the winning icon through A/B tests.

[00:12:37] Another big challenge was how do we align with other teams? Because, as you can imagine, we are a big company, and everybody is split into product teams. So we needed to define who the external stakeholders were, what other teams we needed to collaborate and coordinate with, and how we make sure that we collect enough feedback. So we made a list of stakeholders to keep posted and coordinate with. And for the feedback, we ran several UX critique sessions to gather it from other designers. In this way, we could make sure that the user experience was holistically good.

Creating: pilots, iteration and a north star

[00:13:30] And then it was time for us to start creating. We wanted to define a pilot to test, and we started with a guest-facing implementation plan through A/B tests. Indeed, at this point we had enough data, and we wanted to collect quantitative data to validate the qualitative. We started with the guest, and then we moved to the partner side. We will deep dive into this plan on the following slides.

[00:14:02] Afterwards, we collected the learnings from these A/B tests, and we iterated on them until we were satisfied enough with the metrics. We also defined what the north star for the product would be, what we wanted to reach as a final goal in terms of visuals, explanation, and impact for guests, partners, and the business. So what does success look like?

The product strategy

[00:14:30] We've discussed the UX and communication strategy, and then the design process of the team. Now I'd like to deep dive into the product strategy. How did we implement the product? As it was for the user research, we focused on guests first, defining a pilot for one country and for a specific property type. We did that because we wanted to validate the product first where we had the most supply. Indeed, this exploratory stage was fundamental to make sure that we were not hurting our guests. And at the same time, the team needed to be set up for success. That's why we picked the country where we had the most supply.

[00:15:20] Once we did that, we were ready to scale the product globally. We still wanted to keep only specific property types, because from our user research we understood that we needed to deep dive a bit more into the differences between the property types. So at this point, we went back to the understanding phase, because, as you know, design thinking is not a linear process. Basically, here we wanted to make sure that we were setting the right expectations for guests, and so we needed to take into account these differences among property types.

[00:16:01] At this point, the product was there for the guest. And this was the right time for us to implement the partner-facing product and communicate to the partners what we implemented, gather more feedback from them, etc. As a next step, we thought that we needed some more qualitative data before expanding it. So, as I said before, we ran specific user research on the differences between the property types and the guest expectations. And we decided that it was safe enough to implement an A/B test to expand the product to all property types.

[00:16:44] At this point, the product was running and functional. But there was an opportunity for us to improve partner performance. Basically, we wanted to suggest to them what they could implement to have the possibility to increase their rating. And this was also challenging from a UX and machine learning perspective, because if you think about it, there are things that we cannot suggest to the partners so easily. We cannot just say to a partner that has a property in the center of Amsterdam, "Oh, you can build a swimming pool, or you can build a balcony." Of course, it doesn't have the space to do that.

[00:17:25] Did you imagine there was all this work behind a simple icon? A product that is based on many factors, driven by machine learning, and frequently updated. So we can say that we achieved our goal so far. We validated the system through massive A/B tests, impacting millions of our users and showing strong evidence of significant benefit for all the relevant parties. So the product is there, it's live and running, but we cannot say we're done. At Booking.com, we look for continuous improvement, so we keep on making features better for our users.

Five learnings for growing as a leader

[00:18:11] At the beginning, if you remember, I told you that this story was an example of actions that helped me to be promoted and that defined my growth. So I would like to leave you with five learnings that helped me to grow as a leader, all related to the three Cs: craftsmanship, communication, and commercial awareness.

[00:18:33] First of all, take a holistic point of view and adapt your design process to your product. Define a win-win framework. Map out the actions that you need to take to reach success. And when you design, look at the whole environment around your product. The important thing is that you keep the user at the center, and this is even more crucial when you work with machine learning.

[00:19:06] Advocate for the users, but balance their needs with those of the business. If you don't agree with something, my suggestion is: challenge it, analyze it. Don't be afraid to face hard conversations. But eventually, you will need to find a balance between them and strive for a compromise.

[00:19:29] Flex your communication style. Be humble and ask for feedback. Understand what's important for the people you want to convince, and how they like to talk. And really prepare yourself. This is the key for every conversation. And eventually, accept that not everybody might agree with you.

[00:19:51] Make sure to be involved in your team's strategy, even if it's not your responsibility. Once you're comfortable in the team, don't limit yourself to doing only what's expected of you. Find a company that allows you to fail, take risks, and learn from your failures.

[00:20:11] Demonstrate impact according to what's important for the company. If you want to become a leader, understand what the company and department goals are, and find opportunities that go in that direction, that you care about and that you're passionate about. Take the lead, drive the product.

[00:20:32] Thank you, and that's all for today. If you want, you can find me on Gmail or on LinkedIn.