Outcome Driven Ideation

Eric Greene at UXDX Community: New York. Video: https://youtu.be/txKe9-qlBn4

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Outcome-focused ideation and the Aer Lingus story

[00:00:05] So yeah, I'm the product design manager at Aer Lingus. We have four UI designers and two UX designers, and today I'm going to be talking about outcome-focused ideation. That really means getting your entire delivery team, be it a product or a project, so your BA, your QA, your developers, your designers, the researcher, everyone, at the very beginning, before you've even estimated anything. Everyone's there at the very start to look at the outcome you want to deliver and what this project is. I'm going to go through a couple of case studies of how we define what this outcome is and how we measure it.

[00:00:37] Okay, so Aer Lingus was founded in 1936. It was the national airline of Ireland, and that was the very first plane it launched. So it shipped that plane; I suppose that's called the order. In 2001 it launched its first website. The value proposition for Aer Lingus is essentially still the same: we sell flights online. Nothing really has changed. Everything that's offered here, this is actually in many ways more accessible than the current website, because it has all the data on there, and everything that's a link looks like a link, and everything that's a button has an indicator that it's a button. So we've actually gone backwards in how we do product design.

[00:01:20] And here's how we look today. I joined in 2016. We still have a value proposition where we sell flights online, but we also have an iOS and Android app. Okay, so it's not rendering on the projector, so bear with me. That's very good. This is the default of what I had, so I feel your pain.

How the app team set its own flow

[00:01:42] So, the one-minute ticket. This was the vision statement the mobile app team came up with. They were the first product team in Aer Lingus, which is a project-run organization, and this is how the team mastered research to drive bookings. They're an 18-person dedicated product team: multiple specialists, designers, developers, QA and an architect, when they all started. I worked on this with Rory Martin [?] when he was in everythingís [?].

[00:02:05] This is how the team defined its own flow. You should never really hire a team and then give them your organization's flow, because really you're denying the organization the opportunity of learning from the good stuff these guys have learned in previous organizations. So they defined their own flow. But the most important part about this is, at the start here, there's goals, there's ideation and there's prototyping, and the entire team is here for this. Everyone who is represented the whole way along is there at the very beginning. Before any prototyping has happened, everyone is briefed on what the goal is, so everyone's there at that meeting at the start. And if your QA can't be there, reschedule it, because they have all the scenarios. If they're not there for ideation, you'll miss a scenario. You'll start prototyping, they'll come in later on and say, "Well, what about this scenario?" and all of a sudden your prototype doesn't work.

Measuring success with the Voice of the Guest survey

[00:02:56] Okay, so how we measure success. There are two ways. The first, the most important one for me, is the user experience metric. We survey 10,000 customers every quarter, and we do this with what's called the Voice of the Guest. It's a digital survey, so if you complete a booking, you might be one of the lucky ones who gets asked, "How was your experience? Would you give us some feedback?" and then that basically gets taken into this survey. The other one is Google Analytics, so conversion: do people book, or don't they book?

[00:03:25] This is what that survey looks like. Last quarter we got 20,000 [?] respondents, so people like Aer Lingus. They're happy to give feedback, and we do value the feedback, and the feedback people give in this survey becomes actionable insights for us. The kind of questions we ask in this area: how would you rate the overall flight booking process? We're a value carrier, which means we're more expensive; we're not a low-cost carrier. So our standard is up here. Aer Lingus is a premium airline; our customers pay more and they expect a better experience. So we want them to be answering at eight or above. Five or below is a fail, and this is something we have to fix if it's orange, at six or under seven.

[00:04:05] So I have a quiz for you. A, B, C, D and E are all different digital channels where people can book. One of these is a product team; the rest are project teams. Who can tell me which of these is the product team? You can't answer. Hey, it is A, of course it's A. But you can tell, because they've been together three years and they've been continuously innovating. That's what happens: something doesn't get forgotten when the project ends. It actually just goes into a backlog, and that improvement eventually makes its way into the product. So the product team are happy with the product, and the customers of the app are happy.

The one-minute ticket

[00:04:45] One of the first things the app team did three years ago was they were given the remit of coming up with how to fix the current app, basically, and make it a real conversion machine. This is what it looks like now, and the customer satisfaction rating from that survey is 90%. That's way, way beyond expectations. They did it because they had the vision of the one-minute ticket: from search to pay, I can do it in under a minute.

[00:05:12] So how did they do this? Back in 2017, this is what the scores were, and then by December 2019, in the same survey, the scores are like this. What they did was we all got into a room, dug out all the survey questions and analyzed them. What do these survey questions mean? What are we asking the customers to answer? Because basically what we're asking the customers to answer is what we're going to be measured on as a team. Because we're a product team, we share the same KPI. Basically, our Christmas bonuses are given together; we're all contributing to it. That's an important part of a product team: they have to share the outcome of what they're going to deliver.

[00:05:51] So we looked at these and came to the conclusion that these are all related to speed. Overall flight booking process, overall these reviews, conclusion: the booking flow is not fast enough. Speed is the number one driver of customer satisfaction, not conversion. Customers are happiest when they can do it quickly, but you also get conversion up later. The conclusion on this one is compatibility with my device. It's a native app, but compatibility with my device is only 68. That's terrible. If you have a native app that isn't 100% compatible, people should be answering that at 100, because it's a native app.

[00:06:23] The next one is a good one for our designers. We ask the customers how they liked the look and feel of the experience. That's not a metric that's tied to everything, and not all of our stakeholders care about good design, because it's a kind of fuzzy metric. But the real value of that look-and-feel metric is when you're designing something and it's got a really snazzy transition, and the developer says, "That's really complicated, I don't think it's worth building," and the product owner says, "I don't really see any value in that transition." The designer can say, "Well, hang on, look and feel for this is only 63 [?]. I'm trying to inject a bit of personality and really manage this loading screen for this user on this route." There is value in that. It's called look and feel, so there's a metric for that.

A clear brief built on lean UX hypotheses

[00:07:07] Okay, so they analyzed all the data and were able to come up with their own vision statement for what they wanted the app to be: by having a native booking flow that's fast, accessible and easy to use, we can raise the booking customer satisfaction score to 80%. That was the goal. Then we used Jeff Gothelf's Lean UX hypothesis statements to draft this up. If you've ever read the Lean UX book, it's great; it's like a how-to for doing UX.

[00:07:40] So we came up with this: we believe it takes too long for guests to make a booking, and this is hurting conversion. A faster booking flow will increase conversion. We'll know this is true when we see research on a prototype scoring 80% in overall booking satisfaction. So we build the prototype, get users to come in and test it, and after they've done it, we give them the survey we give digitally, but we just give it to them in person and they fill it out. If they fill it out with the scores we want, we've basically mirrored the digital survey we do, and this will prove the KPI. Voice of the Guest: time to make booking is 60%, expected to be 80%.

[00:08:13] This one-pager is a clear brief for the design team, and it's a clear brief of what we think value is going to be. You get that statement together, and it can be on anything, but it has to live in the Voice of the Guest metric. Otherwise you're just doing a vanity project. If it doesn't affect one of these metrics from the survey we do, we can't really attribute value to it. Well, we could, there are other ways, but this is the one we focus on today.

Everyone in the ideation room

[00:08:37] So you've got your clear brief and the team's assembled, so there'll be UX, QA and so on, and then you need an ideation room. This is what it looked like when we took the app and put it all onto the wall, every booking flow, and the entire team was there to actually analyze it, to see how we could speed things up and where we could make any kind of improvements. There's an early draft of design principles here on the wall, and lots of sketching. Then afterwards, a big list of improvements we could make, and then how these might look. Simple things, like buttons should look like buttons. If I press something, it doesn't need a confirmation: I press it, it changes color and moves to the next screen. That speeds up a two-click process by just having a transition and an animation instead.

[00:09:24] This is a summary of what the booking flow used to look like on the left, here's a sketch from multiple iterations from the ideation session, and then here's what it looked like in a clickable prototype. The time from ideation to technical prototype is actually really fast, as long as you make sure everyone's at the ideation session and you get all the questions out early about what this project needs to be a success, because that's how you stop rework later on: you do all the iteration at the start. Google says that Allison knows for every yes [?]; in a way they'll release something for a year and then carnac [?]. Try and get your tabs and those in the ideation session [?], whether you need a day or two or a couple of hours.

[00:10:06] People are always in a hurry to get to this stage. Don't be in a hurry to get there, because once you get there, people take a position: "I've designed this, I'm going to defend it." And then people don't want to have a battle, they don't want the politics. But there's no politics when you're here, so try and do all that early, at the start. Oh, my... this is too many slides.

Small improvements that sped up booking

[00:10:28] Okay, so this is the transparency of pricing. That's it on the left, at 66. So we cleaned it up and made it much easier to compare fares. Transparent pricing should be transparent. It's not deliberate that it's not transparent here; it's just the way it was designed. So just having a compare fares button and pinning it, so that it's always there and someone can press it, that's probably what increased the score: someone could see easily how to compare fares.

[00:10:56] And then on booking, this is about non-native components. None of these are native iOS components. Simply by changing them to be native-looking components, even if they weren't native, made a difference to how people felt about the overall process. And then, instead of having to select a calendar here, where this page loads and breaks, you just choose dates, and the calendars are there in the first part of the flow. These are just real simple things, to see how we could... The goal is the one-minute ticket: how can we speed up this conversion flow? That's the vision. The vision's been articulated, and everyone on the team knows what it is. We're talking about speed, to satisfy the customer, because the customer wants a fast experience.

[00:11:35] Another one we did: we had these two screens, one was review and one was purchase, but they both kind of duplicated the same purpose and repeated your flight details. And then we had this warning to complete the purchase at the end, so people were waiting on this page, because we made it into a warning. We want them to complete, we want them to pay; we don't want them to be scared of paying. So just "Pay now." That sounds final: pay now, there are no more screens after this. But that's just the cognitive load, how long it takes someone on this page, which is reduced a lot. And they were disabled from paying [?]. Now it just continues, and we're adding things like Apple Pay and different ways to pay, and summarizing the flight currents to and from [?].

[00:12:14] So there are lots of tiny little improvements. But this is what a product team can do, because they can keep continuously improving over and over, over a couple of years, but it's hard for a project team to do that. What we learned from this is that customer satisfaction with the booking flow is 90%, and app conversion has continued to double since 2017. So fixing customer problems increases revenue. I'm not even looking at revenue or conversion as a metric here; I'm just looking at the customer satisfaction scores. But they're the ones that actually drive revenue.

[00:12:46] They really invest heavily in customer research in Aer Lingus. You can see that with the surveys, and even with the people they hire. They have customer research embedded in the organization, and there's a lead researcher [?]. So if you get data in, you're going to get outcomes from your product teams. You have to invest in good research.

Bringing the approach to a project team

[00:13:10] Okay, so this is the project: everything I learned from that product team, trying to implement it with a project team. It's a huge project: deliver a responsive website in six months with a fixed team. It's a fixed deliverable team. This finished last week, and I just spotted something I should have fixed and didn't. I contacted a developer; he's on another project. Getting that fixed actually happened with something on the boarding pass; I'll show you later on. It's a big team, but with the same approach. The vision statement: by having a mobile site that's successful, fast, convenient and easy to use, we can grow our mobile web channel by 105%. That's obviously optimistic, but the Voice of the Guest scores are orange and red, so there's an opportunity.

[00:13:54] Here are examples of why they need to be fixed. It's not responsive. It's confusing. It's not a UI anyone's ever seen before, so they have to figure out how to use it. That's cognitive load; it's taking them time just to understand what they're looking at. And it's the same here; this is obviously complicated anyway. And then this is a non-responsive website, and the features don't act or behave natively, so it just zooms you in. That's the kind of problem you get in 2020 when your website is still not responsive.

[00:14:27] We started with some competitive analysis for the project team. We started with the seat map; they did comparative analysis on who does a good seat map. The reason we started with the seat map is that we did a kickoff ideation, and the devs said they think the seat map is the most complicated thing, can we look at that first? Absolutely. They haven't given any estimates for this project yet, because we're doing all this team ideation at the very beginning, before you estimate. So they'll have had some ideation before they've even given any estimates. Otherwise it's just a finger in the air, and they're making it up, basically. At least this is grounded.

[00:14:59] Then this is the ideation of what a seat map might look like. These are done by designers and developers; the whole team is in the room. The result of that ideation session was: we need to get into the code and have a look. So they did a hackathon the following day, for a day. They've looked at the outcomes, they've done ideation, they've done a hackathon, all before they've given estimates on how complicated they think the project is.

[00:15:22] Ultimately, what we did for the booking flow, which is not released yet, is we just made it the same as the app, because we know from the app that it's flying along. So why would we do anything different? Inherently, we've mirrored the same steps. Okay, ease of adding seats: that's poor on the left. It's not released yet, but our user tests are coming back at around 80. We won't know until three months after it goes out, when we get customer feedback, whether it worked or it didn't.

Running a silent sketching session

[00:15:56] Okay, so that's the talk about outcomes. This is the ideation session. Has anyone ever done a design sprint? Yeah. My favorite part of a design sprint is this ideation session. I can't fit design sprints into my schedule; nobody really can. They're not really practical to bring into an agile methodology, but you can bring in an ideation session, and that's really well covered by Jake Knapp in Sprint.

[00:16:22] Here are the instructions, pretty much out of his book. Successful time-boxed ideation with the team needs a clear problem, a range of specialists, a room to work, markers and paper, a stopwatch, and silent 10-minute ideation, sketching. The most important part is the silence, and that nobody comes in with any ideas in advance. Because again, if someone comes in with an idea or gets to talk, they take a position, and other people then follow it. So everyone needs to come in cold. Just give them: here's the problem we want to solve, and show some of the research, and then silence, ten minutes, and see what sketches come back.

[00:16:57] These are just different ones from designers and developers who were in the ideation session. The great thing is that the best idea can come from anyone, and then everyone votes on it, and then you do another iteration, where everyone iterates on that idea. Eventually you don't even really remember who came up with the idea first. The main thing is it's collaborative, and everyone's doing it together at the start. It's not design showing UI to dev and telling them this is what they have to build.

[00:17:23] This is basically a winning idea. It gets voted on with these dot votes, and it gets taken to a bit higher fidelity by the UI designers, but it's still only pen and paper. Then a UX designer can wireframe it up, giving it a bit more clarity and realism. But really the only purpose of this is to test basic patterns of functionality and content. We want to test: does the content read well, and do people understand the content? Because as soon as you put on colors and interactions and things, you're actually losing the testing of just the content.

[00:17:53] So we have the UX designers do wireframes of different interactions, and then the UX copywriter modifies the copy in this form, and this is how we test iterations of the copy. The designer then isn't having to do high-fidelity changes. It's changed here, early, because it's faster, it's easier, and it's less work. Then that gets user tested, and the feedback from the user test was that this wasn't 100% clear; it could be more obvious to users that that's an add button. So when we took it to prototype, we took that feedback in. This is the prototype design, and we just added an add button. That was really the only iteration, apart from the content changes.

[00:18:37] That's the prototype that was tested, and then this is the design spec that was handed over to development, and then this is what the developers built. It's basically the same, because they were involved from the very beginning. It wasn't like we were handing this over to them. They tested the prototypes with us, we built it with them; we were together as a team from the start. So there's no re-briefing. We're not having to brief them and go through everything again. They know about it, because they've been working on it with us since the start. And this is a project team, not a product team.

[00:19:10] So if you apply this outcome-focused ideation at the start, with everybody, you can get the outcome while the project is active. But once the project closes, you still face the same problem Rory addressed earlier: how do you get something fixed? This is what it was on the left, and this is what it is now. When we released it, it tested well, but we won't know until September, when we get the customer survey results.

What a project team can't do, and how to measure success

[00:19:34] This is my last slide. What I spotted, and I fixed it: there was an error in padding on one of these, this one, I think. I wanted them just to fix it, and he came back and said, "We can't, I'm not on the project anymore." Whereas that would have been a two-second change for a product team, now it's out into the world, and I thought, no, it'll never be prioritized, because it has no value to do as a piece of work on its own. Those little bits, that's what you get when you have a product team, because they care about the product. They're dedicated specialists, and they're dedicated not just because they're permanently assigned to the team, but because they own the product and they like the product they work on. Otherwise they wouldn't do it.

[00:20:22] So how we measure success: there are three ways. There's the Voice of the Guest digital survey, there's booking conversion, but you also measure success by how happy your team is. A motivated team. Outcome-focused ideation means less rework and the team contributing to the vision. So iterate with pen and paper and prototypes to avoid rework, and you'll end up with a happy team. What I've learned in the last few years at Aer Lingus is: a team informed by research builds better products, iterate with pen and paper and not with pixels, and a team that ideates together stays together. Okay, so that's everything from me. Thanks a lot for listening.