Building a Seamless Online to Offline Experience

Susan Stavitzski at UXDX USA. Video: https://youtu.be/BMh1aVm3EwQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Online and offline experiences

[00:00:00] Susan: Thank you. Hi everybody, thanks so much for having me. I'm Susan Stavitzski, I'm a senior product manager at CarMax, and I'm very excited to talk to you all today about building seamless online to offline experiences. At CarMax we have our online experience, carmax.com, and we also have many store locations across the US for our offline experiences. We also have experiences that cross over. Today I'm going to tell you a story about how in one of those experiences we didn't do such a good job, and what we did to fix it.

[00:00:36] But before I jump into that, I want to take a minute and talk about what an online experience is and what an offline experience is. Quick show of hands: how many of you have purchased something online? Yes, everyone. You probably bought your ticket to come here today online. So we're all familiar with this experience. You have a need, you search for it, you go online, find what you're looking for, buy it, and either download it or in a couple of days it's at your door. You have expectations for that experience.

[00:01:08] Another show of hands: how many of you have bought something in a store? Yes, thank you. Thanks for humoring me, guys; it's after the break, you should have some energy. Again, very familiar, whether it's groceries, or new shoes to come to an event, whatever it might be. We are familiar with that experience: have a need, go to a store, find what you're looking for, take it home.

[00:01:33] Each of these experiences is very unique, and they fit a need for your customer, whether it is speed, convenience or personalization. Another thing that makes these experiences unique is the teams behind them. Many of us here in the room are probably on the online experience: product, design, engineering. We're thinking about how a customer is going through their experience on our site. How do they get here, how are they searching, how do they find what they're looking for, and ultimately become a paid customer?

[00:02:08] On the in-store side, or offline experience side, there are similar goals but different teams. Now we're talking about sales, marketing, operations. They're also thinking about the customer workflow, but in a physical space. How do the customers come through the front door? Where are the shopping carts versus the cash registers? How do we lay out our inventory? Same goal of becoming a paid customer.

[00:02:33] Where this can get tricky is when these two worlds come together. It's not a new experience, but certainly one that over the last couple of years has become more popular: this concept of buy online, pick up in store. Now we are creating a handoff point. We have our online experience that knows about our customer. We know what they bought and we know about them, and we need to tell our offline partners, "Hey, they're coming in, they bought something and they're going to pick it up today." Our customers now also have an expectation for this experience.

[00:03:08] What's been really cool over the last couple of years, with this experience becoming more mainstream, largely because of the pandemic and people not going into stores as often, is that businesses are noticing that this experience doesn't just need to work. It has to be really great. How can we make this experience really great for our customers?

CarMax, the principal field lead and an iconic customer experience

[00:03:30] How does this relate to CarMax? If you aren't familiar with CarMax, that's fine. We are the largest used car retailer in the United States. Many of you might be like, "Oh yeah, I know them, they're the guys with the towers." No, we're not the guys with the towers. We also are not the guys with the fox, and we are not the ChapStick company. I just want to put that out there. However, we do have a wonderful partnership with the NBA and WNBA, so you may have seen some of our really hilarious commercials with Steph Curry, Candace Parker, Sue Bird. Really fun; if you haven't seen them, check them out on YouTube.

[00:04:09] I've been at CarMax for a little over two years now, and I've really enjoyed working there for two really big reasons. The first one is CarMax is very technology forward. We think about how we can build technology into our experiences. Where are there opportunities to put technology into what our customers and our associates are doing? This is actually a picture of our wall in our office, where we have our product principles in plain sight. We look at this when we're building experiences, we think about it all the time, and we reinforce it, which is really great.

[00:04:45] Something that we do at CarMax that really puts technology in practice is the way that we format our teams. Tim called out earlier the product triad: you have your designer, you have your product manager, you have your lead developer. Well, CarMax has a fourth role in our core team. It's called the principal field lead. A principal field lead is someone who has worked in the field, meaning at a store location at CarMax, for many years, and they now sit on a product team. They understand the customer experience in our stores, they understand the challenges of our stores, and they understand the associate experience in the store.

[00:05:25] Let me define an associate for you real quick. When I talk about associates: I am an associate at CarMax. Anyone who works at CarMax is an associate. From the people in the field, to the people at our corporate office, all the way up to our CEO, Bill Nash, we're all associates. So when we build experiences, we're not just thinking about the customer who's buying or selling cars with CarMax, we're also thinking about our associate experience.

[00:05:50] Our principal field lead brings a great perspective to the team. They are also our arm back into the field. When we're thinking about how we're going to build an experience that is going to impact our stores or our customers in an offline way, we also keep in mind how our principal field lead can help us execute that learning. It makes our life, and a lot of our researchers' lives, a lot easier too, because we can get in front of customers quickly.

[00:06:21] The next really cool thing about working at CarMax is we are obsessed with driving an iconic customer experience. I'm going to say that one more time: an iconic customer experience. We want you to remember how you felt when you worked with us. We want you to tell your friends how you felt when you worked with us. And we challenge each other whenever we're building new experiences, if we don't feel like we're building an iconic one: how can we do that better?

Instant Offer and the handoff point

[00:06:50] I'm going to tell you a story. A few years ago CarMax went through a bit of a transformation with their appraisal process. If you don't know what an appraisal is, that's fine; I didn't either before I started working here. I hadn't sold a lot of cars. But if you're looking to sell your car, you will want to go get an offer, somebody to tell you how much money someone would buy your car for. You'd want to go get an appraisal. Up until a few years ago this experience was completely in person at CarMax. You'd bring your car to CarMax, one of our appraisal specialists would take a look at it, the mileage, the condition, take it for a test drive, the whole thing, and we'd give you an offer: this is how much we'd buy your car for.

[00:07:27] Well, one of our digital teams said, "Hold on, what if we put a portion of this experience online? What if we are creating friction here for customers that don't live near a CarMax location? Maybe it's hard for them to get to a store, because they don't know what they're going to get for their car. Maybe they don't want to come out. Let's see if we can get them an offer online." So they did that. They came up with this tool called Instant Offer, where in literally under two minutes, from the palm of your hand on your phone, you can get an offer on your car. Answer a couple of quick questions and you'll know how much CarMax is willing to buy your car for. This was huge. Wow, this experience that previously was completely in person, now a portion of it is online.

[00:08:09] At this point we've created a handoff point. We've taken a previously all in-person experience and put half of it online, and then half was in the store, because if you choose to redeem that offer, you've got to come to the store. We'll review it and we'll buy your car; you've got to bring us your car, of course. So there is now this handoff point.

[00:08:29] The digital teams did all the right things. They did their market research, we had our expectations, and we knew we were unlocking a new market. So we said, "We're going to have more volume from this, we're going to have a lot of people coming in. Stores are ready? Yes, let's do it." In early 2021 they launched this product. The date is important, because if you remember what was happening in early 2021, we were at the height of the pandemic here in the US. We also had interesting things going on in our market: the housing market was hot, the car market was hot, interest rates were low. Looking back now, this was the perfect storm to launch a product like this.

[00:09:11] And that's what happened. Against our expectations for the product, this thing skyrocketed. We knew it was going to be a success, but it went so fast and so high that we were like, "Wow, this is amazing." We were excited at first. We want to have more customers sell their cars to CarMax, we want to continue to be the largest, this is great, they're using this tool. But what had happened was this higher volume of customers strained our handoff point, and we had a lot of people come into the stores to sell their cars.

[00:09:43] This is a picture of our Miami store, where a lot of people were out trying to sell their cars to CarMax. They were excited, they got a great offer online: "I want to sell you my car." There were lines out the door. I don't know about you guys, but this does not look like an iconic experience. We might have missed the mark there.

[00:10:04] So quickly we sprang into action. Of course our sales, marketing, operations and store teams got on it. How could we move people around? Remember, pandemic: we're trying to still social distance, we're trying to do all the right things for safety. How do we move people around, how do we add signage, how do we take care of this volume? It's a great business problem to have, but this was a bad customer experience and we wanted to make it better. And our leadership team said, "Hey, how can technology play a role here? Tech teams, what could we do?" That's where my team got involved.

Finding the opportunity: check-in

[00:10:35] My team sprang in and we said, "All right, the first thing we want to do is look at the entire customer experience from the moment they arrive at the store with their offer. What happens?" We check them in, we verify other details, we confirm their offer, we do the paperwork, and we get them back out the door. Where are the opportunities for technology? What could we do here to improve this experience? Also, how long is this taking, because there are a lot of people? So we wanted to understand the whole flow.

[00:11:05] When we dug into it, we found a lot of opportunities, a ton of things that we could do with technology, which was great. We were excited to say, "Cool, let's go do some of these things." But we cannot do all these things. There was a really good session on prioritization earlier. We have to think about what is going to drive the biggest business value and the biggest customer value out of all these opportunities. We got down to three, and we said, "Okay, out of these three, what portion of the experience do these make up?" Pretty good: 83% of the experience was in these three sections. It's a lot.

[00:11:43] Many of you are probably quickly going, "Well, with the limited information I have, you should go after paperwork. 60%, holy cow, do that." Well, you're right, that is a huge portion, but it's also a giant boulder to move. So we said, "Okay, even though this is a huge portion, what we want to think about and focus on first is that iconic customer experience. Where can we have an impact on our customers' experience going from online to offline?" So we honed in on check-in, and we said, "Yes, this is 10%, but we want to make that handoff point strong. We want customers to be able to come in and feel welcomed."

[00:12:26] What was going on in check-in, and where could we improve this experience? Well, we'd actually created a bottleneck at the check-in, which we didn't mean to do. Previously, before the volume, we were able to check in customers one by one with an associate. They'd collect basic details, ask you a couple of questions about your car and get you on your way. Now, with this volume of people, we couldn't do this one-by-one check-in. What was happening is not only were people waiting in long lines, but people were also leaving because they weren't going to wait. So we were losing out on the customers that had done the online experience and now wouldn't wait to transact with us. This was bad.

[00:13:07] So we looked at the patterns and things that were going on in the market, and we said, "We know that customers are providing this basic information to an associate. Why don't they just give it to us themselves? Why don't we create a way that they could self-serve?" We recognized this as an opportunity: today associates are doing it; could we have our customers do this? And we said yep, there are patterns in the market. There are plenty of people now, especially with the pandemic, checking in from their phones or the parking lot or whatever it might be.

[00:13:40] But we stopped and said, "We don't want to make an assumption." We don't want to assume that a CarMax customer is the same as a Panera customer. Sure, you can go and pick up your Panera and check in and get your sandwich, but we're not talking about an eight-dollar sandwich, we're talking about a car. It's a lot bigger, a lot more personal. Would this work for our customers?

A demand test with a QR code and a Microsoft form

[00:14:01] The first thing we did is say, "Let's run a demand test. Let's just see: if we provided self check-in options to these customers, would they even use them?" We don't know; we've never offered it. We said our goal for this demand test was just: could we get half? Could we get half of the customers that come through, who could use this product, to use it? That's half the time we could give back to the associate who's checking them in.

[00:14:27] The first thing we did was go out in the market and look at what's going on. I did go to Panera, I got myself a sandwich, and I saw their check-in flow. We also found other people that were doing different things, like QR codes. And we said, "Okay, for this demand test we don't want to code anything yet. We just want to know, are we going in the right direction? What is the smallest, scrappiest thing that we could do to just get a gut check that we're on the right track?"

[00:14:52] We said, "Let's test out this QR code thing." If a customer was to come in, and remember they got their offer in the palm of their hand in two minutes on their phone, what if we continued that experience and they could check in straight from their phone? We got with our marketing team and said, "Hey, we have this idea, we just want to run this test." Our marketing team is fantastic, and they said, "Yes, let's create a sign with a QR code on it and let's just test and see if people would use it." Again, we're not coding anything at this point. We came up with a Microsoft form, and people go from the QR code to the form to check in on their own. This form doesn't do anything; it's just a Microsoft form, but we are behind the scenes checking them in. Customers would come in and do this, and it made the customer feel that they were self-progressing. They were providing the details. Would they do it?

[00:15:42] This is us when we ran the test. We ran this at our Maryland store, and here's a picture on the far left of the sign out front. You can scan it. Again, remember social distancing: some people didn't even want to come into the store, and we wanted to provide that option to them. If you did want to come in and meet with an associate, we would always be happy to help you. Iconic experience, whatever it is you'd like. And then here's me with my team: me, my designer Jairus [?], we're the ones behind the scenes putting all the information in, and our awesome principal field lead, who was instrumental. He helped us pick the right store so that we could get volume quickly, to really get that gut check and know we were going in the right direction.

[00:16:18] Our goal was half: just half of the customers that went through this experience, and we ran this for a couple of days, would go through this flow. How do you think it did? Oh, thanks. 57%. So we just made the cut. But we were excited about this, because again, this was that gut check. We just needed to know: is it worth investing more time? Because if it's not, let's scrap it and start over. Small, quick, think quick.

Discovery, prototyping and the first version

[00:16:46] At this point we said, "Okay, we feel good, we're going to keep moving forward, we're going to think about this some more." But we'd just tested demand. We hadn't tested anything else, and we hadn't done any other discovery. Now it's time for tech discovery, data discovery, user discovery, customer discovery. Let's go start all of that, because now we know we're on the right track. So our tech teams jumped in and said, "If this is going to actually do something behind the scenes, what does it need to do? How do the systems come into place?" And then we started thinking through the usability.

[00:17:17] We put together a prototype. We said, "All right, let's design something. What could this look like? What could the flow be?" We put together a basic prototype. You can see here, after you scan your code you'd enter your offer code, you'd confirm the offer you had on your car, and then you'd put in your information. We said, "Okay, we haven't tested usability. We talked to some people that were going through the test, but it was a form. Let's put this on user testing and get a bunch of people to go through it, all different types of demographics." Anyone who's selling their car to CarMax should be able to go through this flow.

[00:17:48] We did, and we had a lot of different users go through it, and we iterated and iterated and iterated. We got amazing feedback on language and flow and order of information, and just kept iterating on this prototype. One really interesting piece of information that we thought was hilarious was that customers don't like spelling their street name out loud. When they had to check in with an associate this could be awkward, and they loved that they could type it in. We thought, well, I guess if you have a really hard street name or a weird street name, maybe that's convenient for you. That was a surprise and delight moment where we learned that about our customers.

[00:18:32] From here we did all these iterations on the prototype, we got our tech discovery going, we had our data discovery going, and we came up with our first version. This is the first version that we launched. You can see here we added a landing page at the beginning to really drive home the value prop of why you should do this. You scan the code and you see "Skip the line, check in on your own," really focusing on saving the customer time by self-serving. They put in their information and they'd be on their way.

[00:18:58] We launched this first version at our first pilot location, down in Fort Myers, Florida, and on the very first day of it going live, 75% of customers who could use it used it. That was awesome, and it was a huge testament to all the iterations and all the learnings we had done over the last couple of weeks, from our initial test to our prototype to now.

[00:19:21] Here are some pictures of when we were on site launching it at our pilot location. Down here you can see the sign out front of the store; again, if you wanted to check in from outside, you could. Here's the up-close of the sign. On the far right you can see a couple who came inside the store and were checking in, and here in the middle we have an associate helping these customers check in. Again, iconic experience, however you want us to meet you.

[00:19:45] While we were on site it was myself, my designer and three of our engineers. As we were there, we were talking to customers, we were talking to our associates, we were making iterations on the fly, pushing out code. It was great. We were having a really good time making iterations and learning on site, live, right there, to make sure that we were meeting the needs of the customers with the version that we had released. We rolled this out at three more pilot locations, so that we had a bunch of different perspectives and customers going through the experience, which was great.

Results and takeaways

[00:20:14] We got some quotes. The top two are from our customers: "It was really easy and fast." "I expected to do this myself." We were meeting customer expectations. We were not creating a new check-in flow; this was a pattern that had existed in many places, but for CarMax customers it was something that was benefiting their online to offline transition. And then from our associates: we set out to build a customer tool, but our associates were benefiting from it too. "I don't feel overwhelmed at the desk anymore." "I have time to help other people." They are getting a benefit from this tool as well, having the load of people taken off of them while customers could self-serve.

[00:20:55] I'm excited to share that this experience is live at all CarMax locations now, and on average it's at about 65% adoption. When you scale things across multiple different places there are variables, but we're really excited that we hit that 65%, because that's 65% of time that we could give back to our associates while customers continue on that flow on their own. And we had estimated it at about 10% of the experience, and that's in fact what we were able to decrease the transaction time by as well, which was great.

[00:21:24] There are a few key takeaways here if you find yourself building an online to offline experience. Focus on that handoff point. Understand the teams that are involved. Get in front of your customers and follow them around. We literally followed people around with stopwatches in the store. Follow them around, see what they're doing, and understand where things are falling apart. Then recognize the opportunities you have. What could we do about this? What are ideas that we could test and try? And rank those opportunities.

[00:21:55] Then think small and fast and scrappy. I work at CarMax, it's a huge company, and we still do things really small and really scrappy and really fast. It is possible, I promise. But focus on the overall experience you want to deliver. What is it that you want the customer to feel when they're done using your product? How do you want them to tell their friends about their experience using your product? Focus on that. Thank you.

Q&A

[00:22:26] Host: That was incredible. I really appreciated how you were talking a lot about thinking about the end-to-end user experience and not just the part that you focus on. I think sometimes we can get so wrapped up in the team that I lead, or the feature that I own, and forget that our user doesn't realize that I'm the PM of this part or that part. They don't care. So think fast, iterate and quickly deliver. I love that, it's amazing. I have the mic here, so pass it back to me. I am not great at throwing, so be careful. I would love for us to be able to use this mic, just because folks are online and they can't actually hear the questions. Any questions for Susan? Right here. Oh, all right, careful. This is not a good idea, guys. I'm sorry this happened to you.

[00:23:31] Audience: My name is Irene. I'm just very curious as to why you decided to keep the check-in experience offline, in the stores, rather than having it as a follow-up stage after getting that offer.

[00:23:46] Susan: You need to be physically at the store to check in for us to be ready to look at your car, so that's why we left it at the store. We actually did a test, and I didn't share it in this presentation, where customers who made appointments could check in via their email. People were checking in at five in the morning, and the stores open at 10. So we're like, "Well, you're not actually here." We had to figure out whether there was a geolocation option where you couldn't actually check in until you're physically in the parking lot. There's a lot of iteration that teams are still exploring now, but that was why we went with this one, where you need to be on site, because we need to know the car is actually here to check you in.

[00:24:20] Audience: My second question was whether you considered, and I guess you did, having an appointment system, whereby you would give appointments so that people didn't group outside of the stores and wait there, but rather get their appointment time and then just go into the store.

[00:24:40] Susan: Yep. We do have some customers who make appointments, but a lot of them just show up. The offer's good for seven days, so whenever they feel like popping in is when they pop in. We would love it if everyone made an appointment so we could plan for you, but that doesn't always happen.

[00:24:54] Audience: Okay, thank you very much.

[00:24:56] Host: That's a good question. Right over there, can you pass it to her?

[00:25:04] Audience: Thank you. My question, let me remember. Oh my gosh, I've completely forgotten. Ah, yes. The take-up rate: I was interested if you had quite a variance between locations, and if you looked into that, whether there were any insights around that between stores.

[00:25:28] Susan: The overall adoption rate, you're talking about? Yes. Definitely some of our stores have higher volumes than others, so they were very adamant, like, "Hey, use the self check-in," and they helped instruct customers to do it. At other stores where they only had a trickle of people, it was one of those where, well, if no one's in line, I'll just check in with an associate. So it depended on volume and store location, store type, those types of things. That's why we had great success when we launched in Florida at 75%, but then when we looked overall, over our 240-plus stores, it got to a 65% level overall.

[00:26:03] Host: Awesome, thank you. All right, ready here in the front. Can you take the box?

[00:26:10] Audience: Once you automated the check-in process, how did you then manage the handoff from the digital component to the in-person component?

[00:26:17] Susan: Once you checked in, we actually had a system in place that associates were already using for monitoring cars that were going through this experience. So those customers just blended right into something that was already going on in the store. It wasn't a new pattern we were introducing; it was just a new channel for getting that information into the system.

[00:26:36] Host: Do I throw the box that way, or up front?

[00:26:41] Audience: Thanks for your talk. I think this is a great example of how to use small tests to de-risk ideas. I'd love to hear more about whether this is your standard practice. Do you have a certain way of doing that at CarMax? Because it's something that we want to replicate where we are, but haven't quite gotten there yet.

[00:26:59] Susan: That's a great question. It is, as much as possible. We have digital teams, we have in-store teams, we have teams that cross over both, but we are always encouraged to think as small and quickly as possible, to keep moving forward as quickly as we can. There's a lot of great support from our leadership team; very much, read the Empowered book. How do we empower these teams to think on their own, create tests, and go and prove themselves right or wrong? Lots of failures, open to failure. How do you think quick and scrappy? Some things we have aren't as easy. It's not always as easy as throwing a sign in a store and seeing if people would use it; other experiences are a lot more complicated. But for the ones that can be small and quick, a lot of our teams use that practice.

[00:27:43] Host: All right, we have time for one more question, and then of course you can always find your speakers out on the break.

[00:27:50] Audience: Hi. I just have a question. I noticed that you said you were in a rush for this project. You described the entire process; how long was that?

[00:28:02] Susan: We were able to do everything, from first going into the store just to watch people, to launching our very first pilot, in a quarter. That's where our principal field lead has a huge impact, knowing: what do you want to learn, how quickly do you want to learn it, what stores have the highest volume? If you go and just spend three to four days in a store, will we see enough people to know we're going in the right direction, or will we need to go to other stores? So that role on our product team is huge, just knowing that information and helping drive us in the right direction. But yeah, it was quick, because we had that pain in the store. We had customers in line, and we needed to get something out, at least to test and try, as quickly as we possibly could. But we scaled it nationwide over a couple of quarters, because we obviously had to think about operations and rollout plans and all that.

[00:28:52] Host: Thank you, good question. Very good. We'll give another round of applause to Susan, please.