The UX of Automated Processes

AJ King at UXDX USA. Video: https://youtu.be/wkMPoJQPv3Y

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

The robot beer machine

[00:00:00] Good morning, New York. Thank you so much for being here, and good day to everyone who is watching online, wherever you are in the world. I want to start today with a little bit of a story. A couple of months ago I was at a music festival with my dad. Don't worry, I'm aware this isn't a standup routine. It's going to seem like it to begin with, but we will get on to something relevant in a minute. Between acts we decide to go and get a drink, as is standard, and my dad turns around and goes, "Oh, you're going to love this. You're going to love this. They've got these robot beer vendors here."

[00:00:31] I was like, that's cool. It's cool for two reasons. One, I work in robotics, and that's cool. And two, this is a man who struggles to download things from an app store, and if he thinks it's good, it's probably good automation. So naturally my brain assumes something like this is going to be greeting us outside, this kind of handling robotics. It's going to be really cool. It's not, because design-wise this is not particularly practical. It looks a little bit more like this. It's not a fridge, it's kind of vending machine shaped. It's got a bit of a screen and an interface, and a cup drops down behind a barrier, pours you a beer, the barrier disappears and you grab it and go. So imagine something like this.

[00:01:07] We drift out of the venue and head out there, and there are 10 of these machines in this small area, stacked up, and seven out of the 10 are being used. Because I'm very cool, my first thought is: that's some good capacity planning from the service designers there, well done. I wander straight up, and within a few clicks of a button I've managed to order me and my old man a beer, and I've put down roughly the price of a house deposit for it, at a music festival.

[00:01:36] And I start to get a little bit worried, because something like this screen pops up. Not this screen exactly; I didn't take pictures at the time, because I wasn't smart enough to realize I'd talk about it at a conference. I'm not a designer. No one's going to hire me to be a designer off the back of this. The cup pops down and it starts filling with beer. I used to be a barman, and I can tell that this is not going to be a great beer. I'm going to end up with half a cup of beer and half a cup of foam, which normally, being British and polite, I'd probably passive-aggressively take, mumble under my breath, and go about my day. But it's a music festival, so I paid a lot of money for this.

[00:02:09] Before I can start worrying about this, some guy rocks up from behind. I assume he worked at the place. He goes, "Don't worry about it. This always happens on the first pour after the machine's had a break. Let me fix that for you." He clicks in the top left of the screen, where there is no button, but there's a secret admin button. Amazing. It pops up a little login code, and he puts in four digits that, if I was smart, I'd have tried to work out so I could get another beer later at slightly less of the price.

[00:02:40] It pops up this admin interface that any designer looking at would have cried over. It was filled with buttons, and the copy on the buttons was huge, and it told you exactly what it was going to do. But it's perfect B2B design. This user comes in, they press this, they put the login code in, and he has access to do anything on this machine to fix the issue that I've got. So within a few seconds he's managed to rectify the situation. I take my beer, he hangs around to see if the second one pours correctly, and it does, which I'm also very impressed by. His edge case of "this only happens for the first beer after a break" seems to be true.

[00:03:16] So we walk away, myself and my dad, beers in hand, to enjoy the rest of the evening, having actually had a very good experience with automation where it had mostly gone wrong. And that's what I want to talk to you about today. I'm going to talk about what happens when you are asked, whether it's in software or hardware, to essentially automate out what a person used to do and deliver just a user and a service. Rather than this, you end up with this. We're being asked to do this a lot, across software and across hardware, because it's good from an operational expenses perspective.

[00:03:56] What I'm going to take you through is the process that I go through at Ocado Technology to try and bake in essentially what you lose in this, which is how you retain human intuition. Because as you automate people out of your processes, what you're actually automating out is your best defense against edge cases: human intuition, to look at a system and go, "I can fix that and work around it. Don't worry about that, let me just do this and ping it." So, four steps, 15 minutes, four minutes a step, slightly over time. Perfect.

Step zero: the happy path

[00:04:29] We are going to go through, starting with step zero of one, two, three and four, because I know how to build slides. When you are doing your discovery on this, step zero is the same as it would be in any other product discovery: work out what your main workflow is going to be and your happy path. Let's take the beer example. The customer chooses a drink type, in this case a beer. They choose a brand and a quantity. The UI shows you a total price that, once you've balked at it, you confirm. You tap a card or a phone or a watch or your face or a ring or whatever it is today for payment. Then payment gets confirmed and you have your outcome, which is a well-poured drink.

[00:05:08] This is your happy path. Now, when you're automating something like this, if this is all you do as discovery, you are never going to build something that can adapt beyond its happy path. You end up with me walking away from this machine with half a beer and quite a lot of frustration. So this is how you fix that.

Step one: learn what you lose by removing the person

[00:05:29] Step one: you've got to go and learn what it is you're losing by getting rid of your user. That can feel slightly awkward in some situations, because you're basically going and doing massive contextual inquiry with people that you're potentially going to scale back. But it's important to go and work out what they're doing right now. The things you're looking out for when you run this contextual inquiry are the little nuances of the job. You spend time with them, and you look for the little things where they go, "Ah, don't worry about it, that kind of happens, but this is what we do to fix it. This is my workaround. This is my hack."

[00:06:03] You're trying to come out at the end of step one with a list of problems, a list of the common edge cases that you will get. When you're automating things and you don't have a user to use intuition to get you past those edge cases, you actually do have to start baking this in from early discovery. In an automation world, we are no longer in a safe enough space to go, "Don't worry about the edge cases, we'll worry about them later." If you are removing the intuitive human who can get you past those things, you are going to run into a world of pain.

[00:06:41] What you also get is some of the current solutions these people are using, the little things that they do that get them through this and make it work. So essentially that's it for step one: go and do a big contextual inquiry. It might be in a warehouse. I do a lot of them in warehouses. You might get lucky and get to go and do it in a nice bar like this. If anyone's hiring to do automated UX research with bars, call me.

[00:07:02] What you should have done by now is mapped out your happy path. You should have your main end-user goal at the front of your mind, as you would normally with product development. You've worked out where things are going to go wrong, and you've started to have a bit of an idea about how you might be able to solve your way around them.

[00:07:23] The first time we did this, it caused untold amounts of stress in my and my product team's heads. We went to look at a driverless delivery system and how we were going to integrate it, and by the end of our "what are all the things that could go wrong" exercise, we thought we were going to need an admin interface with a thousand buttons on it to fix everything. We also thought we were going to have to re-architect an entire building, which is slightly outside of our remit. So it was a bit of panic.

Step two: tier your edge cases

[00:07:51] But we realized that it's absolutely fine, because step two is something we do all day, every day, and have to be good at. Once you have your list of things that are going wrong or could go wrong that you now need to bake into your product in some way, you have to work out which battles are worth fighting and which aren't. All of you are aware of this; all of you do lots of prioritization. We know that features are not created equal, but in this case your edge cases are not created equal either. The issues you've got also need to be prioritized in some way.

[00:08:21] The way I like to do this, because I love a whiteboarding session: I love getting people into the office and writing all over the walls, sometimes forgetting which ones are just paint and which ones are the whiteboard. Sorry, facilities. Get product in the room, because they have the vision side of things. Get engineering in the room, because they can bring you back to reality. Have UX in the room to come up with crazy ideas, crazy researchers and designers like me. And really crucially, get your commercial team in this room as well if you can, because they will turn around and say, "We're not paying for that."

[00:08:52] You have to convince them that they should maybe pay for that, but bring them along on the journey. It's easier to convince them right now, when you can turn around and say, "If we build it just to the happy path, no one is happy. Here are all the things we need to bake in." Pop all the issues up on the wall. Stress, panic, there'll be so many of them to look at, but put them all up on the wall and then review them in light of the goal, in this case "customer receives a well-poured drink autonomously."

[00:09:17] At this point I like to tier these into four sections. You can use this tiering method if it works for you; if not, your own prioritization techniques will probably work. For me, tier one is any issue where, if it happens and no one is there to solve it, your user goal is entirely unmet. That is something you have to bake into your solution. Your automation has to be able to do this, whether it's a software chatbot or a beer vending machine. It has to be able to meet all of those problems without someone being on hand, or you've completely wasted your time trying to automate it.

[00:09:55] Tier two is a little bit like my example, where the goal is sort of met but my customer is not going away happy. If you can bake these into your first iteration, great. If not, think about what support users are still going to exist during MVP and work out how it bakes into their workflow. More on that in a bit. Maybe it waits until version two before it's baked in.

[00:10:19] Tier three problems are where your user's goal is met, but the journey to it is really painful. We don't like these things, but we probably have to admit that for the first build we may not solve them. At this stage of discovery, these go straight to a support user who is going to deal with them until later down the line. And then, beautifully, there are a few things that you get stressed about that, when you actually stop and think about them, don't matter. We overthink everything, in personal life and work life. This stuff is safe to ignore for now.

[00:10:53] So tier your things. This is very theoretical, so let's go back to the example: customer receives a well-poured drink. A tier one problem: the machine is out of drink. That is a big problem. The machine doesn't know it's out of the drink, you buy the drink, and it doesn't appear. Tier one problem; solve that. My problem, the machine pours a drink badly: sort of a tier two problem. Maybe it's a tier one problem if you're less British and polite. A tier three problem: the machine can't take a card payment. Maybe it's like, "Yo, you've got to put some cash in this," and I don't know what cash is anymore, so I get sad and go to another machine and it's all a bit awkward. A tier four problem: it won't print me a receipt, or the bill, or the check. No one probably cares about that anymore. It might still be an issue.

[00:11:39] Once you've got all of those things up, you prioritize your tier one issues and then do some solutionizing around them. This is where having everyone in a room is great. The machine is out of drink, so the designer will be like, "That's fine, we'll gray it out. That means there's no drink left. Perfect. UX wins." Then your engineer is like, "Yeah, but the system's going to have to have a sensor for when it's out of drink, and I don't know what that level is." The product manager goes, "I don't know what that level is either. Maybe we'll set a maximum limit: you can only order three pints. The sensor is set so that when the beer in the machine is lower than three pints, great, gray it out."

[00:12:25] Then commercial is like, "But then no one's going to buy the beer. We need them to buy the beer. So maybe we tell them about another machine: go buy over there." Then your engineer comes back and is like, "That's great, because our machines are linked. They're in a network, they're in a fleet, they understand stock between the machines, and we can pop up and say, go to machine 4, machine 4 has your beer." It's this kind of wacky ideating that pulls together and means you can very quickly ideate through some things that might work.

Step three: prototype and pretend to be the automation

[00:12:50] At this point, oh my God, on time, I will go very quickly through the summary. We've done the happy path. We know the user goal. We've prioritized our issues. We've done some solutionizing. We have some solutions that might cover many workarounds, and we have an initial idea of the stuff our support user, that bloke who popped up behind me, is going to have to do. Now we get to do the fun bit, the bit that I as a UX researcher love. We get to prototype it. On the software side of things, that's fairly simple: low-fidelity prototype, usability test, unmoderated, moderated, all this sort of stuff.

[00:13:24] With hardware, though, lots of people turn around and say, "How do you do UX for hardware when the hardware doesn't exist?" Honestly, you can prototype anything. You can Wizard of Oz absolutely anything, and you should be doing it, especially if you're in a hardware space. This is us. This is an autonomous vehicle: it's a table with some boxes. This is also an autonomous delivery. That car isn't that autonomous, but never mind, same thing here. This was us user testing a concept. What you can do is strip back the technology, because the technology is not massively important, as long as you retain what the user needs to do. In these scenarios, the user just needed to pick up bags. So we simulate as much as possible, but we can strip back some of the hardware and some of the technical aspects of it.

[00:14:08] Then you get to pretend to be the automation, which is lots of fun. Usually in a UX session people are like, "I don't know how to do this," and you're like, "How do you think you should do this?" What's great is you don't even have to say that when you're pretending to be a robot. You just stare at them blankly. It's great, it's so much fun, but maybe I'm a bit of a sadist. I don't really know.

[00:14:27] What you want to do in this situation is simulate all of those tier one issues. I would have loved to have been in the beer example. I'd have loved to pour a really bad pint of beer and just stare at someone going, "What do I do? How do I do it? Where do I go?" The reason why this is good, as much fun as it is, is that it also tells you so much. If you hold firm on not responding, the questions that they are asking you, that you don't respond to, show what worry is going on in their head. It's showing you what their thought process is.

[00:14:57] Then when they start trying to tap frantically at whatever tablet you've got on the side, with a crazy wireframe on it that makes no sense, that's what they're going to do in this scenario when you're not there. That means you now know how to bake something into the copy to reassure them that everything is going to be fine, because you know what they're thinking or what they're worrying about. And you've also worked out a workflow, because they've shown you how they're going to troubleshoot these things.

[00:15:26] This is an IP-sanitized version of one of the first projects that we did when we got to the full end-to-end workflow. We had never tested this fully with its technology, but we had tested it in little bits and chunks, and we knew all the chunks worked. We had no idea whether, when we put it all together, it would work. And actually I was amazed. I've never seen this many green ticks when something comes together. This was a full end-to-end test, and the only thing that didn't quite work at the end was just because of the context of where we'd put the solution, and we hypothesized that once it was out in the real world where it needed to be, it would be fine.

Step four: design for the shadow user

[00:15:59] So what have you done so far? You've worked out the happy path. You know the user goal. You know the issues for all your tier one and maybe some of your tier two things. You've ideated on solutions. You've had so much fun in the lab, and you've got so much good data from the lab; it's not all just about having fun. Ideally you've tested your solution end to end before it goes out to your customers, but you don't need to have done so, because ideally you've chunked it up in a state where everything works. What's great about this is that in these little chunks and little iterations, you've basically created your go-live product build, and you've tested it and you know that it works, and you're actually not that stressed anymore.

[00:16:36] The final thing you need to think about is all of those edge cases that you've not baked into that solution. This is possibly the hardest bit. This is where you have to craft what I'm calling the shadow user, in an attempt to sound cool at a conference, but realistically it's just your support person. Never mind. This is the person behind the scenes who keeps everything running, because automation does not get rid of everyone. That is not the purpose of automation. It's to reduce the operational expense of lots of people down to very few. They are basically the Batman of your product. They are unseen in the background and will rock up to help when needed.

[00:17:11] You have to actively think about their experience as well: what their workflow looks like, and what they, as essentially a product, are delivering for you. There are three key things you have to think about when you're trying to design for your support user: how they find out there's an issue, which sounds obvious; how they then get to engage with that system; and finally, the really crucial commercial part of this, what your milestones are for the automation-to-support ratio.

[00:17:39] Taking those very quickly. Alerting is very simple. Your user is either physically present, and if they are, great, something visual or auditory will alert them that something's going wrong, like the little beacon on top of self-checkout machines. It might be that they're in a space where they can't see everything, so you might give them a handheld that alerts them. That is very similar to when your user is going to be remote: they have to have some kind of notification that says these are the issues. If you really want to do this properly, automate the prioritization of those issues, because your support user should be fixing problems. They shouldn't be working out which problems to fix. You should know the importance of the issues that are popping up, and you should take that control away from them so that they can focus on the actual job.

[00:18:23] Then what you need to provide them with is some sort of good system to engage with. This UI popped up with a million buttons and would have genuinely stressed any of my design friends. But it was easily accessible. It wasn't pretty, but the guy got into it very quickly, a consumer couldn't get into it, and it had absolutely everything that this guy needed to fix everything. It was function over style. Everything was on one page for him to get very quickly to a solution: a grid of very well-described buttons. I'm standing over his shoulder expecting a solution, and also expecting that this shouldn't have been an issue. So it doesn't have to be pretty, as long as it's clear.

[00:19:00] The final thing, and I won't dwell on this too much because it's one of the harder parts, is that you also have to think about the cost aspect. Ultimately your automation is aiming to reduce your operational expenditure. So think about the ratio of support people you have to your automation. If you have a large ratio, lots of automation and very few people, you get faster ROI and commercial are happy with you, but there's more room for bad UX if you haven't thought about this. If you have a smaller ratio, your ROI is going to be much longer and it's going to be harder to talk about with clients, but your UX is better.

[00:19:36] It's a gradual thing. Set yourself some milestones and stress test through development. What you can do with that is work out the frequency of your issues, the necessary support for good UX, and the impact on ROI timelines. What's great about that is it makes resourcing conversations easier: "You want an ROI of this? I need this many more people to bake this into production. If you don't, as we roll forward, support is a bigger issue." And that, ladies and gentlemen, is everything. Thank you so much. Here is some stuff you can come and talk to me about that's outside the talk that I like.

Q&A

[00:20:10] Host: AJ, as a favor, I'd like you to repeat after me three words.

[00:20:19] AJ: Uh-huh.

[00:20:20] Host: Massive.

[00:20:22] AJ: Massive.

[00:20:23] Host: Autonomous.

[00:20:24] AJ: Autonomous.

[00:20:25] Host: Aluminum.

[00:20:27] AJ: Aluminium.

[00:20:28] Host: See, it wasn't that hard. We got there. Is it okay for us to differ on this?

[00:20:35] AJ: I'd like to think we disagree and commit.

[00:20:39] Host: I love it. He's not wrong. So what is it like to meaningfully engage with your users?

[00:20:49] AJ: This is a great question, and it comes into more of the specifics of how you build that prototyping session. How you want to engage with the users in this moment depends on how much we know about what the solution is going to be. We normally set it up to run the happy path and not talk to them about any issues that pop up; we simulate those. Then we'll start simulating some problem issues with those users, and from there they will start to go, "Okay, well, what about this? What about this? I might fix it this way, I might fix it this way."

[00:21:22] Then you almost improvise. It's more of an unstructured usability test, because again, you can Wizard of Oz anything. If they turn around and go, "I think the best way to do this is to have a foot button that, I don't know, retracts the drink into the top, pours another pint and starts again," you're like, "Cool, all right, let's work it through." The drink comes down, it's not pouring very well, do what you think is right. And they're doing this with nothing, on the floor, and they're like, "Yeah, it's weird, it doesn't feel intuitive." So the way I engage on that is to get them to come up with crazy ideas and then role-play them out.

[00:21:55] Host: Wonderful. When it came to the nature of the prototyping that you were doing, can you tell us a little bit about what some of the goals were, as well as a little bit about the context, maybe some of the environment?

[00:22:08] AJ: The goal of that concept testing is getting comfortable that once you, as the UX researcher, or your product person, aren't standing over it, the thing can be done. Once you're comfortable that you know how the thing can be done, then with the box of technology you put around it, you just have to be in touch with the engineers, chat with them and make sure that they don't change what the core engagement is in the way they build the automation around it. In the delivery system side of things, ultimately you strip all the technology back: it's boxes, you pick up some bags. Providing whatever automation we put behind that doesn't hide the boxes, turn the boxes upside down or do something that means they can't literally go, "Let's go," then you're onto a winner, because you know that in its simplest form the user can do it.

[00:23:03] But it's about having that cross-functional thing as well. When we're doing this, we will be in heavy communication with the designers, with the engineers, with the product team. It's one unit that has to make decisions, because everyone's got their expertise. We UX researchers bring in the very basic "can do thing good." Everyone else does the more technical stuff, but we make sure that the core, simplistic need is still met no matter what crazy bells and whistles we put on it.

[00:23:31] Host: I'm debating where the mic drop was there, whether it's "like all things, multidisciplinary product development," or what was it, "make thing good, user do thing." Right, good. And that's the UX goal. AJ King, ladies and gentlemen. Thank you so much.

[00:23:52] AJ: Thank you. Appreciate it, mate.