Designing Voice Driven Experiences

Noelle LaCharite at UXDX EMEA. Video: https://youtu.be/ttAHyvcZ7Q4

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Meeting Alexa and an early skill builder

[00:00:06] Good to see you guys. What a nice crowd we have; we'll see how this goes. You've seen this before, yes? Raise your hand: have you seen an Alexa device? Okay, put them down, that was everybody. That's awesome. So you saw the Super Bowl, then, maybe a commercial or two. How many of you own an Alexa device? A smaller number, but well done, you guys. Love it. Okay, cool.

[00:00:33] As you can tell, this Alexa device has a nice red ring on it, which means Alexa is not listening to me right now. However, I have connected Alexa to the RTS Wi-Fi. All right, cool, let's see how this goes. "Alexa, what time is it?" "The time is 5:12 a.m." In Boston, which is where I'm from, so if I'm not completely coherent, now you understand. "Alexa, tell me a joke." "What's the pirate's favorite exercise? The plank." Ha ha. And she's full of jokes just like that, so go running and get yours today. I'm going to mute Alexa again.

[00:01:28] Oh, actually, well, I'll come back to this. We have a couple of interesting things I want to share with you today. My name is Noelle LaCharite. I am an early adopter of Alexa. I built some of her very first skills; I've been building for Alexa since 2014, I guess. Some of you might know my skills, and if not, you certainly will want to. One is called Daily Affirmation, where you just ask it for an affirmation and it will say you're awesome and wonderful and people like you, and you can go on your way.

[00:02:03] Do you guys know Saturday Night Live? Well, I'm not dating myself, but there's this guy Stuart Smalley. Anyway, you can look him up. I built it around that, but then it took off and people were actually seriously using it, so I got serious about it. I ended up building One Minute Mindfulness, which is exactly what it says: one minute of mindfulness. Yeah, I know all of you are thinking one minute is not nearly enough. Well, that's exactly what my reviewers have said, but I stand firm in my marketing and I have not elongated it, because it's advertised as a minute, so it should be a minute. But my customers, to the tune of, I don't know, 2,500 reviews, have said it should be longer than a minute. Don't fault people for doing exactly what they're telling you they're going to do. But I like it, so you should try that one.

[00:02:58] I've built many skills. Probably 20-plus skills published on Alexa I have either built or helped to build. My goal over the next 20 minutes is really just to share with you my journey, what was very different about it from a UX perspective, and then hopefully give you some inspiration to try some of this fun stuff yourself.

The next big shift in interfaces

[00:03:21] This is us, really. I've actually been part of probably the majority of this slide. When I first became interested in software and software engineering, we were just on that cusp of moving from things like mainframes, which are still alive and well, thankfully, into a more user interface world. And then we were all part of that wonderful 2006 launch of the iPhone, which changed us forever. And now we're seeing another one of these big shifts. I remember early on I used to teach for IBM, teaching mainframe people how to write the Java programming language, and at that point they had never used a mouse before. I'm teaching them object-oriented technology and they've never used a mouse. It was a very bizarre time.

[00:04:17] However, I'm in the same time now. I'm teaching people to write user interfaces, and when we say UI, people instantly think of a screen. Now, we do have a screen in Alexa, but we very rarely design for it. If it happens to have a screen, awesome. If you happen to be able to show a card or some kind of display, even better. But very rarely do we consider that as part of our user interface. I'll actually be on a panel later this afternoon talking more about some of the intricacies of how those decisions changed as we moved from a mobile-centralized, focused world, even at Amazon, into VUI, or voice user interfaces.

[00:05:03] We do believe that voice, and I apologize, we can't read it all, but it's like one of those tests where we give you some of the words but not all of them and see if you can still understand it. We believe voice represents the next major disruption in computing, similar to when screens became a thing, and then when little screens became a thing. Now no screens are a thing, and we've actually found that it's extremely delightful for our customers.

[00:05:30] I started with Amazon three years ago, which means I'm a dinosaur in that sense. I have worked for AWS; I now work for Alexa machine learning, and I work on the brain, or the heart, of Alexa: how does Alexa get smarter every day? I can't really talk to you too much about all those pieces, but I'm happy to discuss with you. I'll be here all day and on the panel this afternoon, so I'm happy to answer questions over coffee or whatever as I see you. Oh yeah, feel free to take Twitter pictures and stuff. I hope I get a copy of that picture.

Voice everywhere, from eldercare to the kitchen

[00:06:11] I don't fully buy into this, but it just sets the stage for what we're talking about: soon touch could be gone and all we do is talk. It's similar to what my dad used to think. He's an old Asimov, Golden Age of science fiction fan; he loves the ideas that were presented in those works of fiction. He's in his 70s. He actually was hit by a car, so he's cognitively probably more in his 80s. But I bought him an Alexa device and he uses it every single day to turn on his lights with his voice, and he finds it fascinating.

[00:06:52] Especially, any Star Trek fans, you don't have to raise your hands, but Star Trek lovers, I know you are: we now have the ability to use a different wake word other than Alexa, which you heard me say. We also have "computer." So now he can say, "Computer, turn the lights on." He can even say, "Open the pod bay doors." I should totally try that for you, but it works; she has a response. It's interesting, because he never thought in his lifetime that would happen. When he read those books when he was 12 and 15 years old, he never thought in his lifetime he would be speaking to his software and having things actually happen.

[00:07:31] He navigates music like the best of them now. He plays random classical music from composers I have never heard of, and I'm pretty much a classical buff, but he's like, "Oh, I went to that concert in 1952," and I'm like, wow, incredible. So it really opens up the world, not only for eldercare but also for little people, as well as anyone with disabilities who maybe can't see a screen. It just creates a huge opportunity. So though you can imagine a day when we never have screens, Amazon's not looking to end the screen. Even though Fire Phone was a rough experience for us, I owned one, I loved it. But we're not looking to end the screen; we're looking to really augment it, add to it, and this certainly has turned out to be a good experiment.

[00:08:23] Voice is going to be everywhere. You can imagine a time, and even today this happens. I used to live in Seattle before I moved to Cambridge, Massachusetts, and in Seattle I was part of this wonderful ecosystem in Amazon where I could, at work, call Amazon Prime Now, which is our service that gets you anything within an hour, and say, "All right, I'm on my way home. It's Tuesday; we do tacos on Tuesday." I think it's actually International Taco Day today, I'm just saying, you might want to google it. So let's say I wanted to make tacos. I would get on my phone and say, "Here's what I need for Taco Tuesday."

[00:09:03] You could imagine a time when Amazon's smart enough to know exactly what I mean by that. There probably will be a time where it's, "Oh, you order the same exact 12 things every Tuesday, or every Monday night," or whatever. So it goes out, grabs that stuff and starts shipping it to my house within an hour, while I spend an hour in traffic getting home. By the time I get home, the stuff I ordered is sitting on my doorstep. I pick it up, walk inside and say, "Hey Alexa, I'm ready to cook." Alexa already knows I made that order, Alexa already knows what I want to do, and says, "Great, don't forget to set the temperature on your stove." You can imagine a time when your stove is smart enough to hear you say that, and it can turn itself on. Scary stuff, I know, I know.

[00:09:48] The smartphone was a very scary thing when we first got it. It all starts scary and then it's cool. But you can see where I'm going: it becomes a conversational state you're always in with your device to help you get through, in this case, meal preparation. As I mentioned, maybe there's a time where I forget to make the order when I'm at my desk with my phone. So instead, maybe I want my car to be able to take that order, or I want my car to be able to navigate me, because I'm in traffic, find a detour, and I don't want to have to open my phone. I don't want to have to click a button. I don't even want to have to click the button on my steering wheel. I know, first world problems, but it could happen.

[00:10:36] And I do find it's quite delightful. I have a Dot. I actually have one here, so I'll show it to you guys if you see me after. I have a little Dot, it's like a hockey puck size, and it fits conveniently right in my cup holder. I tether it to my phone and Bluetooth it through my car speakers, and it's awesome. I can have it read me a Kindle book, I can have it play Audible. It's amazing. And of course I can have it randomly play any music I can think to ask for. So I think it's really cool.

[00:11:08] The last part is really about bringing Alexa into the home, the living room. Today we have products like the Show. I've actually become very fond of it, because it allows you to see the time and what recipe you're on, and you can even use the screen to scroll through. So it does give us the ability, in addition to voice, to design for what we now call a multimodal experience. A multimodal experience is the future. The customer wants to be able to communicate in whatever way is super convenient for them at the time.

[00:11:42] Jeff Bezos always says we'll never be in trouble, because customers will never say, "Oh, I don't want more features," and "Oh, I really wish these were more expensive," and "Gee, I wish it was less convenient." People always want better stuff, more convenient, and available immediately. So if we're always indexing on those desires, we'll always be okay. That's why our mission at Amazon is not about Alexa, it's not about books, it's not about really anything specifically. It's about customers. We want to be Earth's most customer-centric company. So we look at our customers, and all of our roadmaps in our organization, including UX, are designed by the customer. That's really what changes the way we do business.

Inside the Echo and why speech got good enough

[00:12:31] That's the inside of it, in case you were ever wondering. It does have seven microphones. We use echolocation, and you'll see over here there are seven of them. When I start speaking to Alexa, you'll notice there's a light ring. I used to wear it on my shirt, but I kind of have a light ring on my shirt. Alexa. You can see this light ring; some of you in the back may have trouble, but you can see the light gets a little bit brighter. That's the microphone that's actually focused on my voice and beamforming what I'm saying into the device and sending it to the cloud. Eventually Alexa realized I was not talking to Alexa and shut down.

[00:13:11] There's what's called endpointing that occurs in the cloud, which decides if you were actually speaking to this device or if you were accidentally saying it. I'm actually on a panel, I think today or maybe next month, with someone named Alexa, which is always fun. Anyway, that's just to give you an idea of what's in there. It's not much. That is all of Alexa right there. Alexa's actually the software in the cloud; this is just the Echo device. The rest of the device, other than that little piece at the top, is speaker. So that's ultimately what you're paying for. Yes, it's a really good speaker.

[00:13:49] Speech advancement is another reason why UX is really important for voice right now. Even if you're not doing it, you should be thinking about this right now. Because if you were thinking about this 10 years ago, even with the technology you had, you probably wouldn't have had a great experience. I won't name the experiences we might have had back then, even just maybe 7 or 10 years ago, but our voice experiences were not delightful. Customers did not believe in them, and they didn't end up working out very well.

[00:14:22] There's a reason why Alexa today is doing as well as Alexa is doing, and the reason is because our speech and language understanding has gotten exponentially better. And now we have even more players in the market, like Google and Siri, and there are others, I'm sure. There are tons of startups building their own; there's this one, I can't remember, it's like x.ai or something like that. Anyway, there are tons of machine learning tools being built, all of which require a good user experience, and most of which were not created by people who excel at that. It's a huge opportunity.

Skills, devices and Alexa everywhere

[00:15:03] In the store, one of the things that differentiates Alexa from any other platform, really, is the ability for us to improve Alexa and make Alexa smarter ourselves, as individual contributors. As I mentioned, I was not working for Alexa when I started building skills for Alexa. My Daily Affirmation skill was actually built as part of a challenge at work, but none of us worked for the team at that time. At that time I think there were 140 skills in the store. There are now over, I think, 16,000. It's growing massively. It'll be 20,000 in a month; it'll just grow exponentially from here. This is your opportunity. Just think about the App Store when it only had 10,000 apps in it. Now it has billions, and we're still leveraging it quite heavily every day, at least I am.

[00:16:00] The skills concept is how we build applications for Alexa, how we make Alexa smarter all the time. These are our devices. Some of you might be Kindle owners or Kindle Fire owners. One of the really interesting things that happened recently is your Kindle Fire became a multimodal Alexa device. We released an over-the-air update, and that update enabled the device to operate this way. It wasn't what we call far-field, meaning I could stand far away and say, "Hey Fire tablet, turn my TV on." You couldn't do that. But what you could do is go up to it, press a button and say, "Open Kindle," or "Start my latest audiobook," or "Resume my last Kindle read." Rather than you having to press a button and slide and click and go down a menu, it would just instantly do whatever I was asking of it. That's what we're looking to provide: that contextual instant gratification of "I ask, I get," as opposed to me having to navigate to any degree.

[00:17:11] These are all of our devices. This is the Echo thing I've got right here. It does have a battery base, just so I can carry it around, but that's an accessory; some company made some money building accessories for Alexa. I invested in the Tap, which is the one that is portable and small, but why would I carry a small Alexa traveling internationally? I didn't do that. And I do have a couple of Dots with me, so I can show you guys those as well. That's just to give you an idea of where we are with devices.

[00:17:39] This is the longer tail of what we're doing with Alexa. You can imagine Alexa on pretty much anything. Why can you do that? Because any company has the ability to leverage the Alexa APIs in their hardware. It's how I started at Alexa: I was a solutions architect for our Alexa Voice Service, which meant I worked with companies building in-car entertainment or infotainment systems, building refrigerators, building televisions. Today we currently have a collection of these devices, either in development with these partners or already on the market, allowing you to integrate Alexa. For example, of course, phones: apps are available that have Alexa built in. Earbuds [?]. Our Kindles are also available. Alarm clocks, where I use my Dot. Fridges, I don't think we're there yet, but you can see where we're going. It could be in anything. Ultimately it's our Alexa everywhere, really voice everywhere, methodology.

Why brands should build a voice-first skill

[00:18:41] How does this happen? It happens through the Skills Kit. You're probably thinking to yourself, "I'm so excited, how do I start right now? I've opened my laptop, I'm ready to build a skill." If you're not, you will. Skills are the apps you build for Alexa. It's the one thing you can do today. Let's say you have a brand in mind, your own brand, or you work for a marketing agency, or you work for a brand. What's one thing you could do right now that might be delightful for your customer? Try and think about it. I'm going to answer the question, but think about it.

[00:19:16] The customer will get a new way of interacting with your brand. You might not even be providing a brand new, flashy service or a new piece of functionality. It's something they might do every single day, but instead they get to do it with their voice. You'd be surprised how childlike people become when they realize they can say something and it happens. Like I said, with my dad, who is 70-ish, and I have four children, and my youngest two are about two years old: the same level of excitement occurs when we turn the lights on in our house with our voice. We've been doing it for a couple of years now, and still every time it's like, "Wow, it's so cool." And it is cool.

[00:20:05] Imagine how amazing it would be if you could provide that level of excitement, even for the smallest thing, to your customer. It's just that little thing, and it creates a connection with your customer they really can't get anywhere else. Like I said, it's pretty darn delightful. We're creating what are called voice-first apps. We say voice-first because, as I mentioned, we are doing multimodal devices, so it's not just voice. But it should be voice-first, because you just don't get that level of delight with anything else, just being able to say, "Hey, rather than go three levels down in a menu, can I just ask for that?" and accommodating those requests.

[00:20:46] This is the Alexa app. I think this image is actually old; it might be the one that's in the Kindle app store. We have an app on the phone. If you own a device, you would have this app, because you need it in order to set up your first device. This is how you configure it; I just did it in order to configure Alexa to work here. And this is what the store looks like. I didn't want to be self-gratuitous and have all of my apps on display up here; that would have been a little bit too much. But there are lots of them to choose from, and I know most of these developers.

[00:21:25] I built many of the tutorials we currently have to help you. If you go out and google how to build an app for Alexa, chances are, if you're on Amazon's website and developer blog, you'll find me. So you're welcome. Now you and I are close friends, and you can email me or hit me up on Twitter. I'd be happy to answer any questions you have, and I totally do it. You'll know if you're in my Twitter circle; I'm super responsive.

[00:21:52] This is how people find your brand. You'll notice there are these big banners. Every so often, if you create a relationship with the Alexa marketing team, they will actually feature you big time. I've never been featured that way, but it's just as meaningful to get featured in some of these smaller areas. It increases the utilization of your skill by something like 900%, and I'm only speaking from experience on over 20 skills: at least 900%, sometimes even more. So it's nice to be able to get there.

[00:22:28] You might ask how you get there if you're in with 16,000 or 20,000 other skills. Well, we have one thing we're looking for, and that's customer satisfaction. All of our top-rated skills get featured. You'll notice they're all pretty much in the fours and fives. Keep in mind, not a whole lot of the skills in the store have that, because they didn't have some of your skill set. I know this audience isn't totally UX people, but some of these developers weren't even developers. We handed them a GitHub repo and they just published something. So there's a lot of opportunity to even go to some of these skills and offer to assist.

[00:23:12] The framework we use: we have the Alexa Skills Kit for you to build skills for your brands, and we have the Alexa Voice Service if you happen to have hardware. Let's say you're building your own earpods, or your own tablet. We tried the Fire Phone, so you could do that. If you try to go into any hardware market, you can build Alexa into it, and it'll instantly add a level of interestingness. The reason I say that is because we do lots of hackathons, thousands of them in a year, and almost always the one who wins has somehow integrated Alexa. Why? Because it's just cool. Customers think it's cool, and being able to leverage that relatively easily, without shifting any major active development in your organization, might be a useful thing to think about.

Tips for building skills people love

[00:24:02] Here are some things you should think about when you're building. High value. Here's my super sneaky trick. I've built several brand-based skills, and what I did is I went into their websites, and you DevOps people might be able to help with this, and identified the most popular click trails. What are the most popular things on a website or mobile app that people click through? Pick one and get that working. In my case, I usually pick ten, because I like a challenge, like demoing live. Nobody really does that, but I do. So pick those top ten click trails, things people click, click, click to get to on your website all the time. I guarantee you, if you create an intent structure, in other words the ability for someone to execute that same request with one question to Alexa, they might be happy to do that, and it might work out really well. It has worked out for me. That's my first and foremost tip for building.

[00:25:01] The other thing is: think about all the different ways someone would talk to Alexa. I'll talk more about this this afternoon on the panel, so I hope you'll join me if you're interested in this world. There's a lot of work that goes into making sure that if someone uses a slightly different phrase, Alexa doesn't fail. So your skill should evolve. Your users should be able to speak naturally; that's part of accommodating them. The difference with this UX is that it's not like a button, where the button looks pretty much the same to everyone. Everyone talks differently, and you have to accommodate them all.

[00:25:38] It's not called computer intelligence, it's called artificial intelligence, and guess who does the artificial intelligence? That'd be us. We build what makes it look intelligent to the user. A developer does that work. So, yay for us, job security. But it's tough work. It's enjoyable and delightful, but it's not easy.

[00:26:02] Alexa should understand most requests. Your certification testing process will ask you to test your skill, and they're not just saying "it works on my Alexa device." They're asking you to build a beta program and have a bunch of users test it. Those users should not get a script of what to read to Alexa. You should tell them what you want them to do and let them say it however they want. I guarantee you: I was working with a financial firm, and we never knew that someone would use "bucks" in a transactional context on Alexa. But they do: "I want 20 bucks transferred to this account." Okay, so we added that. We ended up adding 2,700 variations on how to transfer money to accounts. And skills should respond in an appropriate format.

Start simple with a minimum remarkable product

[00:26:49] Really, if you're interested, if you're thinking, "Man, that sounds kind of cool," or you're super interested, or even if you're only a little bit interested, you really should think about how to start with Alexa. We encourage a crawl, walk, run concept. We encourage you to go through the process of building an MVP. Anyone know what that means? Minimum viable product. But we don't like that, because viable is not good. For our Amazon customers, viable doesn't fly. So we call it a minimum remarkable product. That's why you go and figure out what your data tells you about the most important things people want, and build that. Build the minimum remarkable product. Build only one if you have to, but build one, and then add stuff over time.

[00:27:38] Daily Affirmation, I've evolved it, I don't know, a hundred times since it came out. I still have an average of 20,000 users a week, which I think is crazy, and it's mostly at lunchtime, just so you know. Your friends need some love at lunchtime. It's just cool. We've got lots of metrics too, so you can decide how you're using and building your skills.

[00:28:00] This is what we say: start simple, find something meaningful and build for that, and then just remember you're not building for the screen anymore, so you actually have to talk to people. But that's okay; you can start by talking with me. We won't have a huge amount of time for questions; we're going to open up for a couple. But I will definitely be here all day, at coffee hour and after the panel this afternoon. So let's go ahead and do that.