Building Products At Postmates

Bianca Curutan, Christine Song at UXDX Community: San Fran. Video: https://www.youtube.com/watch?v=CmzZw8bThqQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

The traditional development process and the three-sided marketplace

[00:00:05] Speaker 1: Before we jump right into Postmates' products and processes, I'd like to take a step back and briefly review the software development process as a whole. Most companies, especially larger ones or those with more of a corporate culture, tend to have a process that looks like this. It starts with gathering and recording product requirements and designs. After that, coding takes place in the development and implementation phase to create a functional product. Once coding is done, testing can begin. This can be internal or external, or both at the same time, to report and fix issues, and no new features are introduced during this phase. Then, when the product is finally complete, or at least deemed deliverable, it gets released out into the wild. Finally, the engineers continue with any follow-up maintenance and updates. The one-directional arrow here indicates that this is a sequential methodology, or waterfall, with few if any opportunities to return to a previous phase, jump ahead to the next phase without completing the current one, or provide feedback. We'll return to this process later for a review.

[00:01:20] Speaker 2: Before we get into that more specifically, I'll explain the business ecosystem that we have at Postmates right now. Postmates operates in a three-sided marketplace: we have the fleet, the sellers and the buyers. As you can probably tell, each market has an intimate relationship with the other two, as changes to one can affect the economics of the others. It's basic supply and demand. The seller and fleet markets create supply, and the buyer app generates demand. So if we were to take a holistic approach to our overall ecosystem, we have to find a way to surface the supply of the sellers to increase and maintain the demand of the buyers, while ensuring smooth delivery between the two with effective usage of our fleet.

[00:01:56] We also need to find a way to measure this progress and to ensure that we are achieving what we want to achieve. So what is the metric by which we should measure our success? We ended up settling on the goal of growing the number of users in each of the markets. This is a metric that would scale proportionally across all markets: if there are more buyers, there are more couriers in action and greater activity for the merchants. This goal was broken down into two measurable metrics: onboard new users, meaning make it as easy as possible for new users to convert, and keep existing users engaged, meaning encourage them to keep using our platform. These were the metrics that we had in mind when building the products that we decided to build in each organization.

Onboarding new couriers

[00:02:51] Speaker 1: First up is onboarding new users. One of the many ways that we aim to onboard new couriers to the Postmates Fleet app is making signup available on multiple platforms. For example, users can sign up using the mobile apps, which are available on both the App Store and the Play Store, or they can sign up using the web and then eventually jump onto the mobile apps to complete their signup process. Either way, signup is laid out with very clear steps, with very limited information asked for up front. For example, the very first form that users are required to submit asks for nothing more than their first name and last name, so they're not overwhelmed with having to provide too much information all at once. Another feature of signup is the ability to log off or close the app and then return to it at a later date on that same step. This is especially useful for things like ID verification, where users need to go and retrieve their ID, which they may not have available when they start signup.

[00:03:50] Another objective that we aim for is low activation time. Typically, automated profile photo approvals take only 2 minutes, and background checks take five. This, along with the ability to do prepaid deliveries, allows couriers to essentially start delivering immediately, without having to wait for their physical Postmates card to come in the mail. Otherwise they might give up or join a competing app or service.

[00:04:17] Finally, there's in-app education. With in-app education, couriers are able to go through a mock delivery, so they know what to expect when they're doing deliveries. Additional tutorials also walk couriers through how to check out their earning details, which include promotions, wait time and distance, and that can actually get complicated unless you have the visual in front of you, or unless you have a very clear understanding of all the different variables that come into play. In addition to these two tutorials, we have also built in-app education in such a way that we can add more tutorials at a later date with ease.

Onboarding new buyers

[00:04:58] Now on to the buyer side, which we approached slightly differently. Buyer functionality is available through the mobile apps and the website, meaning buyers can also place orders through the website, whereas couriers, or fleet, cannot deliver using only the web. One feature of the buyer app is group ordering. Users can start an order and share a link with others. A use case for this might be if you're hanging out with a group of friends or colleagues and you want to put in a boba order, which can get really complicated really quickly. In this example, one person can start the order and share the link with everyone else, and everyone can add their own order without the first person going, "Do you want tapioca? Do you want jelly? How much sugar do you want? Do you want ice?" It definitely reduces user error.

[00:05:50] Another feature that we built on the buyer app is Postmates Party. Unlike group ordering, Postmates Party allows buyers to jump onto another user's order, even if you don't know who that person is. The benefit of this, especially for new users or solo users of the platform, is that there are no delivery fees and no Blitz pricing, even if it's a busy time. The TL;DR of all this is that there are low barriers to entry for both apps, and we provide information and incentives up front to reduce user uncertainty and costs.

Engaging existing couriers and buyers

[00:06:29] Speaker 2: When it comes to engaging existing users on the Fleet app, the features that we've implemented focus on creating tools to reduce user errors from the couriers and optimize their time spent on the app for maximum profit. A good example of this is heat map polygons. Heat map polygons are areas on the map that show couriers what parts of the map have high demand, so they can go there and maximize their time spent. Delivery details and package count are examples of organizational tools that we've implemented on the Fleet app that help Postmates see the delivery details, which show you what you can expect when you're picking up an order. For example, this is helpful for vehicles with limited space, like scooters or bicycles. You open the app and you see that you're about to pick up 17 pizza boxes and four drinks, and you're on a bicycle: you're probably going to want to decline that order. Package count also helps couriers know what they have to deliver at the destination they arrive at. It'll tell you things like, "You have four onion rings and three side dishes to deliver for this order," and they can see that without having to keep that information in their head, making sure that they get the order right when they arrive.

[00:07:42] Pre-tipping is not something that we implemented on the Fleet app, but it is something that we implemented on the buyer app, where a user, upon ordering, can choose how much to tip their courier beforehand, so that the courier immediately gets their tip upon completion of the order. The other flow is that upon completion of the delivery, the buyer will see a modal pop up on their app asking how much they would like to tip, and this can be days or weeks from the time of the initial delivery. Pre-tipping ensures that fleet gets paid out more, or faster, and it's more efficient, and it's actually an option that a lot of our buyers use.

[00:08:22] The features that we've implemented on the buyer app are less UI-specific and a little bit more technical on the back end. A good example of this is that we designed a feed algorithm so that the app will surface a greater variety of places to buyers. This ties back into the earlier concept of increasing the supply, and the variety of supply, that you're showing to your market, so that demand also hopefully increases. Another example is passwordless login. This makes it easier for users to sign into their account. No more messy "I forgot my 18-character gibberish password that I wrote down on a piece of paper and lost three weeks ago, and I have to do forgot password every time." Instead, you can just log into the app, put your phone number in, we'll send you a four-digit code to your phone, and you can log in with that. It's much faster.

[00:09:05] Quick reorder is a component that we introduced in the feed. It's a place group that has the merchants you have ordered from recently, and when you click into the merchant, it deep links into the most recent cart that you ordered from, so your most recent order is there. I'll explain a little bit more later about why we chose to implement this specific feature, but the overall point was to reduce the barrier to entry: fewer clicks for users to order their complicated boba orders, or Postmates or Chipotle orders where people know exactly what they want, the same bowl every time. It just makes things easier and more efficient to do.

A development process with a feedback loop

[00:09:52] Speaker 1: Now let's take a moment to revisit the development process that I displayed earlier in the presentation. This is what I showed, and it might be more like what you're used to working with. However, at Postmates we prefer a different way of approaching it. These processes might look the same on a high level, but as you can see, we also adopt a constant feedback loop. Some concrete examples of providing feedback within the company are having individual or team discussions, reporting features, improvements or bugs, prioritizing pull request reviews, or commenting on requests for change or specs. Some ways for external users to provide feedback are messaging through the app, Twitter or Reddit, believe it or not, which are just a few of our online resources, participating in focus groups, being beta testers and joining the Fleet Advisory Board.

[00:10:44] Sometimes stakeholders easily agree on the feedback, so the steps to make the necessary changes are very clear. However, it's also common for there to be disagreements, which can create a healthy tension between the individuals or teams involved. In the long run, this can lead to a more effective and productive environment. The teams at Postmates are relatively small, so this tension I'm talking about, and what we do with it, can really make an impact. And let's not forget the other change here: measuring outcomes. Building and delivering products is all well and good, but without analyzing and understanding the impact it makes on end users and the company, it's very difficult to plan for what's next and determine how we can make our current state better.

Why quick reorder beat the new feed algorithm

[00:11:30] Speaker 2: Bringing it all together: I mentioned earlier that I would talk a little bit more about why we decided to create this quick reorder feature. I worked for some time on reworking the feed algorithm that we use to surface merchants on the buyer app. We had this initial hypothesis that surfacing better places and helping discovery of new places would not only attract new users but would continue to engage existing users. Over time, we revamped the algorithm to choose places that had higher delivery ratings and were favorited by more users, and used the user's order history to surface places that were similar to what the user had previously ordered from. If the user did not have previous orders, we surfaced tried-and-true places, which were popular across the board with our users.

[00:12:13] We defined an experiment that would roll this new algorithm out to a percentage of users in some of our larger markets, and we let the experiment sit for a couple of weeks. We did several experiments of this kind, so if you've noticed something different with our app compared to your friends', maybe we're A/B testing on you. We tweaked the algorithm to favor different properties of the places and did multiple iterations of this experiment. But as time passed, we started noticing, surprisingly, that there was no change in user conversion. It didn't increase, but it also didn't really decrease. That was interesting; we were very surprised to see this happen.

[00:12:51] So we decided to go back to the drawing board and test our initial hypothesis that surfacing new places would attract demand, and surprise, according to our data it really wasn't true. It turns out that a good chunk of our conversions came from searches, which means people would go onto the app, search for what they wanted, and order from there. Either that, or they would order from places they had already ordered from before. Given this, we realized that in this specific instance it wasn't really about increasing supply and the variety of supply to increase demand. It was more about finding out what the demand is and creating features that feed into that demand and make it easier for users to get what they want. That's how we ended up with quick reorder: users can quickly see the orders they've done in the past and reorder them. After running some queries against those place groups and their success rates, quick reorder is one of our most popular and successful place groups.

Lessons learned

[00:13:59] After 20 minutes of us talking, I'm about to summarize our presentation in 30 seconds. If you only take one thing away from this, take these lessons learned. Experiment, experiment, experiment, and analyze the resulting data and feedback from users. Experiments are not always successful, such as exposing new and innovative places. We are creatures of habit: a lot of our users like to order from the same places over and over again, and that's okay. Sometimes there's just not enough user conversion, so we need to turn the experiments off and revert to the older way of doing things, or try a different way that may prove more successful. In addition to that, we should not be afraid of removing code that's no longer useful to us: Marie Kondo your code.

[00:14:48] Not all changes make a big impact. Even if the experiments do work out, they might not make a big impact, but that's also okay. Just because they're not making a big impact doesn't mean they're not useful; sometimes smaller changes can build incrementally to create a bigger impact. And finally, we are never done growing. We want to keep improving the user experience by innovating and releasing new designs and features. Here are some links for you to explore and learn more about the company and the work we do. We hope you enjoyed the presentation and the insight into building products at Postmates. Thank you.