Preventing Churn Through XPs
Preventing Churn Through XPs
We live in a world where retention is king, users get to switch between products seamlessly. Building products that are relevant and that stay relevant with consumers’ ever-changing needs are critical. Continuous and quick improvements drive innovation and retention.
This session will look into:
- Listening to your customers is only the beginning
- Setting your team up for success when it comes to continuous experimentation
- Guidelines for setting up experiments
- A case study from Uber Eats in South Africa
Daniele Joubert, Executive Head of Growth and Consumer Operations SSA,Uber Eats
Hello, I'm Daniele you read and I will be chatting to you guys about preventing churn through XPS. I currently work at Ruby eats in South Africa. And I'm the head of growth and consumer for Sub-Saharan Africa. So it's been an incredible journey working at it. We ate and we learned so much. And I'm really excited to take you guys through everything around XPS and how to actually implement it and build that into your daily strategy. So to start with UberEats, Uber Eats delivers beauty products like flowers, toothpaste, firewood baby necessities, bed food, and all of these things that are not food. And it's definitely not in the name. And I think a lot of people around the world asked the question but how did we go from delivering and fulfilling restaurant food as hot and as quickly as possible to the clients to delivering everything else that is not food? And that's really the crux of my talk about how you innovate and build better products for your clients, not just listening to them. So the agenda for today is listening to your customers is only the beginning. And then I'll take you guys through setting up your team for success when it comes to experimentation. We'll also chat through some ideas around guidelines for setting up experiments. And then I would like to show you a case study in South Africa of an experiment that we ran again that just proves the results is that the behaviours of eaters are sometimes a lot different than what we expected them to be. So listening to customers it's only the beginning. So how did Uber Eats get where we are today? So the business started as a ride-sharing business which you will know this whole idea of can you get a ride? Can you go where you need to be at the click of a button? From the year we started you know creating and we were one of you know the creators of this idea of a gig economy and building and creating opportunities and owning opportunities for people in our network. And because we've solved this idea of you know moving people from point A to point B at the touch of a button this idea of delivering food started you know we started tasting it in the US can we get a dry cannot take and our solutions and our logistics you know solve not just moving people but moving food from point A to point B as effectively and as efficiently as possible. But let's jump to 2022.
Suddenly the world got hit by a pandemic and a lockdown was announced and no restaurants were able to operate. And as you'll be eating this is what we were doing. But as a business, we've learned to adapt and diversify and make plans. And we were really able to enable our clients to get what they need during difficult times you know by ensuring safety and less contact between people. So we went from delivering food to delivering pharmaceutical products and groceries and liquor and you know anything that consumers needed that did not that they could not get because of the lockdown restrictions. So this idea of listening to your customers if we listen to our customers we would have stayed in a virtual food court. This is what we eat. It was a virtual food code. One place where you can get all your favourite brands, order your favourite food, and get it delivered in time. But by not just listening to our customers by running XPS by testing you know new product features. By doing trials we've actually become an entire virtual Mall. If I were to ask people five years ago hey do you think that we should be delivering jewellery or clothing? Most people would have probably said no no no that that's not a product fit. And yet we are delayed today delivering just that anything at the click of a button in the convenience of your own home or a location that you know where you need that specific thing. So how did we get it? How do you set up a team for success when it comes to experimentation? How do you set up your team to be able to build it through mutations and learn what they eat? Do the clients like what they want without them being able to articulate it themselves? So this entire idea starts with understanding your users in a data-conscious world. So the first thing is getting to know your users. What is the information that you Have about the behaviours, decisions, and choices that they make in their experience with your product? For us, it could be when they order what they order what is the basket size? How frequently do they order? What are some of the things and the pain points and decision-making points for your consumer in the journey? The second thing is to centralize your data. Yes, keep personal information under lock and make sure that is as safe as it can be. But give the person your team and the rest of the business access to information. The only way to truly create a data-driven culture is by giving them access to the information that they need in order to make better decisions than data-driven. And then finally if you know your customers and you have access to your data run experimentations, build cohorts based on behaviours, run experiments as quickly and measure and measure them as accurately as possible and really use the results to adapt your strategy. But what are the fundamentals of making experimentation part of your business? The first fundamental is it needs to be entrenched in what you do every single day. The number of problems and the number of unanswered questions that you have about your current user's needs and your future user's needs and behaviours just increases every single day. So in order for you to truly understand them better to really improve your product and your experiences. And that's just your business behaviour and performance overall you really really need to make the experiment experimentation part a critical part of your strategy and execution. It shouldn't be something that's an afterthought or once a year or on big changes. But it should be entrenched in every single decision that you make. The second thing is that in order to make experimentation entrenched in your business you need the technology to support it. Running multiple experiments weekly at the same time is very time-consuming. Focus on building technology platforms that allow teams to set up experimentations. And experiments very quickly. Make sure the technology not just allows them to set up the XP but also to run the results and XPs analysis of basic sets. But if you need and have to ask your team to run these results manually every single time it reduces their ability to run faster and create more experiments and ask more questions. So build the tech that allows them to do that on every single product that you develop on every single question you have every engagement and as quickly as possible. And you'll see how quickly you can learn. And then the last fundamental is the focus when experimentation becomes entrenched in your company. And when you have the technology that allows you to quickly execute and validate your assumptions the world becomes your oyster. But as soon as the world becomes your oyster the skill becomes in your ability to prioritize your assumptions. Make sure you are focused on only trying to test one thing at a time but run as many tests as possible. Focus on assumptions that could drive higher returns on investments. This could be either from a financial perspective like conversion or it could be from an experiential except you know it's from an experiential perspective. So just make sure that there is one specific thing that you focused on for that experiment and only try to validate that assumption. So why does this matter? In the past acquisition was king. So a company's ability to get a user on the platform was the most important thing because the friction to move between products or companies was high. So most companies invested a lot of money in getting their customers onto the platform, getting them to use it, and getting them to download that. And once they were sticky but because in today's world and the technology environment that we work in it's so easy for most consumers to jump between products between companies and banks something that was very hard to move between a few years ago is a click of a button. So in today's life retention is king and the only way in which you can really drive and improve your attention is to drive loyalty to improve their experience is to diversify your product to create the light moments Add they love and also product updates.
But you can only achieve this if you entrench and make experimentation part of your life in order to give your consumers updates and experiences and loyalty experiences that they didn't even know they want. So be less dependent on market research and taste your assumption and ideas in a real-world environment. So what are some of the guidelines for setting up XPS? So the first guideline is your objective of what you're trying to achieve. And your hypothesis is two different things. Things. Your objective needs to be linked to a business question that you're trying to answer or achieve or solve. And your hypothesis should be linked to what you will be testing specifically. The second thing is to spend time formulating your hypothesis a lot of the time something that might seem very straightforward ends up being very complicated. And something that might seem very complicated ends up being very straightforward. But if your hypothesis is clear and you know exactly what you are testing for it is much easier to set up your entire experiment. The second thing is to clearly define your primary and your secondary KPIs. A lot of the time in experiments the results could show improvements in your primary KPIs. But where do people get caught up in these unintended constant consequences? So ensure that you consider both the good that you were trying to achieve. And that you might have achieved as well as what might be some of the matrices that had a negative effect? What was that? And how do you balance the positive with the negative to truly understand what the impact of this change would have on your business? The fourth thing is creating a controlled environment. As much as it's feasible. This is a real-world experiment. This is not a theoretical environment. So yes, exclude your treatment groups as much as possible from other elements and other engagements that could influence the results. But do not try to isolate them completely so that the results of your experiment are actually skewed. And it's not really a true reflection of what the results would be if you had to make it live in a real-world environment. The fifth thing is to think about the execution before you launch the experiments that you do that need to be linked to something that you can change and implement and you know the impact. So if you test assumptions that provide incredible results but it's not something that you can live with, you will actually be wasting your time. So again when it comes to the focus make sure you're testing something that will drive business value as well as something that you can implement or that can impact your strategy in order to get the most from it. And then lastly test one hypothesis at a time you test one hypothesis using different treatments versus trying to date different hypotheses using one treatment. So really focus to ensure that you know what you do with the results because this again will impact what this means for you going forward and the business results that you're trying to achieve. So I want to end off with a very very interesting case study that we did in South Africa. So the case study was around how we can use choice architecture to improve our conversion rates on promotions and the promotions that we ran. So the business objective is improving our conversion on promotional campaigns in order to increase our orders and our gross bookings and just, in general, our top-line performance. So the hypothesis that we made is we really wanted to taste and utilize behavioural heuristics, one specific one which is called third choice architecture in some of our campaigns in order to make sure or to see what the impact will be on our conversion rates for the users that we talked with. So how we decided to set up this experiment was for treatment one we will be sending three promotions in the same campaign to the same cohort of eaters and allow the user to actually choose which of these promotions they would like to use and then to see whether they end up using it whether they end up placing an order and also what the value of those orders would be for the business.
The second, third and fourth treatment groups will each only get one promo code and each one of those promo codes will be linked to the same value as one of the ones or one of the promo codes in treatment one. So our primary KPIs were really we wanted to test what the incremental impact would be to our business. So what was the user uplift? What was the trip outflow? What was the gross bookings uplift that we would achieve? If we run these promos and also across the different treatment groups. Our secondary KPI however was efficiency which doesn't help us target our treatment one but it drives a lot of growth. But it actually costs us a lot more than what the other promotional engagement would have. cost us. So quite interesting if you see some of the engagements that we built. So on the choice promo and treatment wine, the users got emails and push notifications and in-app experience experiences all linked to the choice Hey pick a promo that speaks to you to get 75 Rand off because you're a solo eater or get 150 Rand off because you're ordering for a family. Or if you're spontaneous, get 99% of your order but then spend it in the next three hours. So again gratifying their choices but really giving them the option to select one of the options. This is an example of just treatment for and the 99% off promo. So it was like Hey come into that and get 99% off your next promotion. Again we had emails and push notifications. And we had the same amount of communication on both treatment one as well as all the other treatment groups that we targeted. So if I had to ask you person X if you were a logical person and you got either one of these promotions, which ones do you think would have received the most orders, and which one would have driven the best business performance? Right, on the one hand, it's kind of busy. There's a lot of information. You know the title is all about the choice and picking an option versus foe which is very much straightforward 99% of I mean I don't even think you can get a better deal than that anyway. But the results were so interesting. So I took if you take treatment one and you compare it versus the other the average of treatment to two for the user uplift was 200. And focusing more on the GB uplift for 61% more. And on the secondary KPIs, you know the cost per incremental eater and the cost per incremental cruise booking was a lot less. So the conclusion of this experiment was that by giving people a choice between three options instead of just giving them one, we got a higher uplift. The campaign drove a lot more efficient results and we created a gamified experience which also increased our retention and engagement of the consumers that we targeted. So this for me is another example of and just a proof point of how important it is not just to make assumptions about what your product updates or initiatives will have on your eaters or your clients as behaviour but rather really embedding experimentation in your daily life in order to test your assumptions and really see what the impact of this would be on your business. So this is my conclusion for the day. I hope you enjoyed it. And I'm really looking forward to seeing a lot more companies build experimentation into the product design as well as just into the daily operations. Yeah, thank you so much for having me, and have a wonderful day.