Unlocking a Truly Great Driverless Delivery Proposition

I'm going to tell you a story that starts where a lot of us in UX research are wanting to be. A lot of the time I hear UX say, I wish I'd been involved earlier in the conversation, or I wish I'd known about this 3, 6, 12 months ago, or I wish I'd just been there at the very start of this product. But it's something I've said a few times. And this is the story of what's been happening since my wish was granted.
It all started a few months ago after Ocado announced that it had partnered with a couple of companies working in the autonomous mobility space for the software behind self-driving vehicles. Now, it doesn't take too much of a leap to work out as a grocery logistics company, which we as Ocado might be planning on using self driving vehicles for. So at this point on this news drops a few months after we had happened, a PM was brought into the group and was asked to start looking at things from a product perspective. And she reached out to our UX team to see if she could borrow anyone to support for a couple of months. And I with embarrassing enthusiasm and speed kind of threw my hat into the ring to help out. And a short while later after one of my team's prioritization meetings, which involves a lot more looking at JIRA backlogs that are interesting for a conference talk, it was agreed that I would be able to work on the project.
So wish granted, I get to be the UX researcher at the very start of a product's life. Now, obviously, this is a huge new product domain. And so we decided to step one, we would immerse ourselves in the current delivery product space. So myself and the PM planned a day trip to shadows and drivers in a London suburb to understand what it is our distribution centre would look like for those vehicles, and then take a trip out and do a day's worth of deliveries.
This will give us a good understanding of the elements of a driver let experience that might be missing from our new driverless offering if we didn't work out a way to bake all that good stuff into it, was a fascinating day. What it taught us was that our distribution centre was not at all set up for autonomous vehicles, it was already incredibly crowded with our current fleet of vans, the vehicles operate almost constantly, and there really isn't much in the way of storage for our vehicles. So charging them was going to be a bit of a problem. Also, the current wave we load goods into the back of our vans needed to change as that wasn't going to work for the new shape of our vehicles. On a day's deliveries had reminded us just what a nightmare parking in suburban London really is.
So it's all good. At the end of that day, all we knew we needed to do was change up our entire distribution process, build ourselves an entirely new distribution hub from the ground up that could work with our vehicles. And of course, most crucially, and easily of all, fix the parking crisis gripping suburban London. Easy, right?
So step two, we had a bit of a panic, overwhelmed with everything that we've seen before step three, deep breath, we realize that, yep, we can fix all those things. But we dive too deep into the roots, right? At its core, like with most of the product you're working on right now, wherever you're watching this, it's going to be a service. So it doesn't matter too much about fixing those background processes. Because if we don't offer a truly great customer experience, then all those amazing improvements we make behind the scenes don't really come into play if no one's using it. It's obvious, it's the basics of our job. But it's kind of worth remembering, especially when it looks like there's too much work to be done on something.
So while focusing on a great customer experience is obvious, what was kind of less obvious when we were thinking about this? And before we focused our thinking is kind of what this service was actually going to be. Because while this kind of driverless vehicle looks cool, or terrifying, depending on where you sit on that kind of scale, that sort of conversation we can have in person, you're not going to get to ride around in one of our driverless delivery vans, sorry, to kind of burst that bubble at this point. So while this is kind of a cool concept, this sort of Robo taxi idea, and what springs to mind when we start talking about driverless tech, this isn't really what we're offering, or offering could be something closer to this significantly less cool, and really bad if we don't get it right.
So if we're only focusing on the tech, when the tech actually has very little to do with what we're offering the customer, this could be the situation we ended up in. Now that's it. I know what you're thinking. It's called Tech. Right? So surely, there's an old field of dreams, philosophies, or that required, right? Build it, they will come. It's cool. It's innovative, people will try it. Sure, fine. We know that novelty and emerging tech spaces will win you a lot of business early on. But we have to also think about the long term sustainability of such a customer offering. So while this might be a useful way to calm us down and think about it, maybe we should be working closer to this kind of philosophy. If you build it right, they will come back, because that's the game we're all playing, right?
Cool. We've covered the exposition, we've reminded ourselves of the basics. So let's get into the good stuff. What are we actually doing now the groundwork is laid? Now that my wishes have come true, the real work begins. So looking at this, we're currently looking down two branches of research, we're looking down the proposition, or what service it is, we're actually offering to customers. But we're not really going to touch on that today. Because we're also looking at a second branch, a more intriguing research branch, which is about how people are going to interact with that vehicle itself, how they're going to be able to unload and collect their groceries.
Now, this side is more intriguing, because currently, there's no vehicle. So I've got my wish, I'm finally working on a product right at its genesis. But there's a newfound occupational hazard that this is meant to be a physical piece of kit to interact with. And we have to build it smartly and intuitively so that people can use it. But we have no idea what it's going to be at this point. All we have to do at this point then is research and the usability of a product that doesn't exist to make sure by the time it does exist, we've made it easy and intuitive to use. So people want to keep using that service. Not a daunting prospect at all. So we find ourselves in a bit of a chicken and egg situation.
So not to be deterred. After briefly toying with the idea of maybe building a time machine that we could take our vehicle from the future, and bring it back into the past usability test, which my product manager said sounded fun, but slightly impractical. I remembered a quote I'd read from an author, Gillian Flynn a long time ago. Now, Liam Flynn says, There are no really new stories anymore. And this has kind of always made me think about how everything is an amalgamation of other bits and pieces, the very few things are created truly uniquely, it's possibly a weird way to view the world. But in this case, it's super useful for us, right? Because we don't have an AV solution to test. But we can strip back the façade of what that end product will be, and work out the basics of what needs testing. In this case, an autonomous vehicle delivering groceries, it's basically a locker system on wheels, right? And stripped further back than that, a locker is just a shelf with some bags of groceries on it.
So a shelf with some groceries on it. Now, that is something we can use for the ability test. So viewers are ready to marvel at our first stand-in self driving vehicle for usability testing. So what did we do? Well, we borrowed some of our delivery crates, we got an average sized Ocado order delivered to our office, and we put out a call for volunteers to take part in the first generative usability test of a non-existent product that I've ever done. But again, there are no new stories. I've run usability tests before, I've run generative user interviews before. So with our table, with our self driving vehicle loaded up with some shopping, and a bit of a discussion guide focused on the information we felt we really needed to know right away at the beginning, it was time to start.
So this is roughly what it looks like. Across the day, the day was split into three sections with each user, section one we called Imagine the day, all we did was ask a super broad, very basic question to get them to start telling us a story. And that question was to imagine you've got your normal online grocery delivery arriving today. And you've selected a self-driving vehicle to deliver it. Just talk me through how you expect the process to be. And then we got to talk and see what it was they expected as a customer with no expectation of what this service would look like. We'd previously had a chat about what we kind of expected the delivery flow would be that you would be waiting for your groceries and notification that the vehicle had arrived would come through to you, you would go to the vehicle and verify who you were to get your shopping, unload your shopping, and you would carry your groceries home, and the vehicle would leave.
So while we got people to sort of self drive us through that journey themselves, we also had some follow up prompts on each of those points, just in case they didn't touch on any of them. Having done that, we then, as silly as it sounds, ask them on a piece of paper to do a very simple pencil sketch of how they imagined the vehicle to look. So in this way, they've started thinking about the experience they expect and what they are expecting to see. What we also wanted them to do was talk us through how they expected the vehicle to present the groceries today.
So from that, we then moved into section two, where we used our very high tech autonomous vehicle for the day. That white table you can see right now, some boxes on top of it. We then took their scenario, how they said they expected groceries to be presented, and played it out. We told them that this was their normal shopping. That the door to the room we were in was their front door and all they had to do was unload the shopping and get it into their house as they felt they would if this was kind of a real world scenario, it's slightly contrived. But it would teach us an awful lot about how they would go about unloading this vehicle, their thought process through what they were thinking.
We then followed this up with a few more of our own scenarios that we decided in advance where we varied the height of groceries, the orientation in which we presented them, whether we presented all the shopping at once, or a set of three bags at a time. And also whether this was a big weekly shop for a family, a large amount of shopping, or maybe something small and Topkapi, maybe something in the immediacy space. And we just watched, and let them talk through everything they were thinking and feeling and doing in this moment to get an understanding of how a customer might react when this was what they were actually doing with one of our vehicles.
In the final section, we then ask them to come back to the table, alter the drawings they've done, and kind of chat us through how they found the experience.
Now, we did this for two days, across a bunch of people. And it raised a whole bunch of questions for us as well good generative research should, we learned a lot from these customers about what they expected the proposition to be, which was kind of a useful input for that other research stream. But for our interaction research stream, how people would engage with the hardware, it really helped us identify which of our upfront assumptions and questions would be key in the customers mind when doing this task, and what things we thought might be important, that never came up when they were doing this. So we've got a whole bunch of data. It was broad, it came from a very semi structured, unstructured type setting.
But it did give us some clarity on a few avenues of where we could go next. At this stage, we then took our findings to the hardware design team to get a little bit of guidance on which of those avenues was worth exploring first, essentially, we were trying to find out what the first decision they needed to make and get set in stone was to get rolling on designing the vehicle, and whether or not we could give them some help and guidance from the UX perspective. And there was one clear winner from what they said, which is what is the optimal height of grocery presentation or less fancily how high should our shelves be. And this was the first step in building a great feedback loop between ourselves in the product space, and the hardware design team. To make sure we were not only focusing on the product aspects that we felt were most important to understand.
But we were also ensuring that the hardware design team were getting the data they needed in the right order so that they could be more confident in their next decisions. So on once more, with our trusty red customer crates, and a whole bunch of groceries, we put out another call for volunteers. Again, kind of hoping that the prospect of helping out on this driverless delivery project might excite them enough to look past the task, which was effectively to retrieve three bags of shopping from a red box at slightly different heights across a 45 to 60 minute period, while we quizzed them about how they enjoyed retrieving three bags of shopping from a red box and their comfort around it.
Now, across that day, we collected a lot of qualitative data. Realistically, we were hunting for heart numbers with this test. And those numbers were what heights form an optimal kind of zone of grocery retrieval in relation to the height. Now I've tried to make that sound fancy because realistically, it was a day of filling in these Google sheet tabs, where the first column is the participants height. And the subsequent columns represent different comfort heights across three different ways of presenting boxes to customers, who say no taking needs to be boring. Now, at the end of the day, we had a relatively good appreciation of what kind of heights would work well across our sample of very usefully diversely hated people. And what heights did not work so well? As with a lot of user testing, though, we'd spent a whole day getting data, and we've gotten quite deep data. But it was only from a handful of people at this point.
Meanwhile, armed with some qualitative understanding of how people expected to interact with the vehicle from our table testing days, as well as this range of potentially good heights, and of course, the design team's own expertise, they then got to work on developing a whole bunch of really cool concepts for how the groceries could be presented by our autonomous vehicle. Now, sadly, I can't share those concepts with you today. But here is a publicly available image we've previously shared that might help you imagine what one of these things will look like on the street. And you can imagine underneath this, how the groceries might be presented within lockers.
Now, from this, the design team came to me and the product manager and said it wasn't going to be practical to physically create prototypes of everything they'd come up with and they were busy scoring their concepts on a variety of scales, including technical feasibility, cost of reduced, complexity of mechanism. But they were also keen to get some user opinion and include that in their scales as well. And so they asked if we could go and get some feedback on the concept of race. And so we went, and we did just that, right, we ran a set of focus groups to understand very quick first impressions from users about each concept. And why a focus group? Well, we felt that a good focus group would really tap into that kind of creative bouncing off each other, that works so brilliantly in a well facilitated group, the kinds of things and concepts that unlock in those conversations that you just don't get in a solo user interview that you really need people bouncing off each other. And from these sessions, we were able to add user feedback into the mix of all the other scorings.
So in other words, we managed to achieve that good product development approach, where we gather data from every useful angle as a team to make the best possible decision on which things to build first and test next. Now, building the prototypes was going to take a little bit of time. So this gave us a much needed moment of pause and breathing space to review the work that we've done so far. Now, by and large, we were pretty happy with how we tackled the whole testing of a non existent physical product gig. You know, the mixed methods generative first piece have taught us a lot and cemented a lot of our thinking in the proposition space, our focus groups had led us feed in UX metrics towards the next part of the iterative hardware development process. And while we haven't talked about any of it so far today, we were also pretty happy at this stage with how our proposition research was coming along.
And much like our hardware research findings, from that, it had started to blur into this branch as well. All in all, things were fairly positive, except for one aspect that I didn't feel we kind of robustly covered enough at this stage, which was that height of groceries piece. So on the day of the height research, we had a lot of potential heights to test and only one day due to a variety of factors to test it in. So while the data was really useful, it wasn't very broad. And I certainly wasn't too comfortable that such a fundamental aspect of the product might be guided on the basis of a handful of users. So we decided we needed to do a similar sort of experiment, testing, but bulk testing for heights that made it into that optimal zone.
So what does that mean in practice? Well, it means we set up shop in our headquarters reception area, with our best representative autonomous vehicle for the day, here featuring one of my lovely colleagues, and we use the footfall of that busy part of the office to grab any passers by, to take part in our very simple gorilla testing, essentially, roll on, roll on, come and grab the bags out of the boxes, and let us know if it's a comfortable height for you. And once they take apart, we then ask that height, which it turns out is certainly an interesting metric to gather self reported data on. The point remains, across the day, we managed to go from under 10 data points for each of our heights in that optimal zone to nearly 200 data points across once again, a conveniently diverse and well distributed set of heights, even if it was self reported. And at this stage, we were a lot more confident in our data on this aspect.
We've done good deep user testing, to narrow that thinking down. And then we'd both tested those assumptions to validate them as best we could at this stage. So what happened next? Well, we're kind of about time, and things started to get closer to normal testing after this point. So let's end things here. But say it's the end of part one, as it's by no means the end of this process. There's more we've already done. There's certainly more to be done. But hopefully, this has given you an interesting kind of time to amusing insight into the wild world of researching a physical product that doesn't yet exist.
Before we wrap up, though, because I find a lot of time at conferences. I'm kind of left inspired but slightly adrift in the message I meant to take away from people's talks. Here are a couple of things I'd love for you to remember from this short presentation. Firstly, focus on that customer experience. Even if there's a whole load of background processes happening behind the scenes. If you're working on a b2c product or a product that has some b2c element to it, your experience is the primary concern at the start to get people to come back. Remember the modified furniture is philosophy. If you build it right, they'll come back.
Secondly, remember Gillian Flynn, there are no new stories. It can be super daunting jumping into an emerging domain, but just remember to take a breath and where you can't test the actual thing. Just try and find something comparable, strip away the idea until it's in its most basic rauris form, and test that cuz sometimes even a dull white office table can be an autonomous vehicle, even if it's just for a couple of days. Finally, never forget, at its core UX research is about getting some quality data to be that little bit more confident in the product decisions we make. We hide tests with less than 10 people. It gave us confidence to start drawing up some concepts, but it certainly didn't give us the confidence to double down on those for full production.
So we went, we got more data, always test and retest and retest again. So the next time you need to make a product decision, stop and work out a way, no matter how silly it seems to quickly get some data, make those most unknown things a little bit more. No, because I guarantee that little boost of confidence it adds to the small decision you take will translate into huge gains in your overall product. Thank you for your time and attention. And if you're in person this week, I look forward to meeting you later.