Always In Beta: Using Scenario Driven Design For Better Understanding Of User Behaviour

Pritish Krishna Panda at UXDX Community: Dublin. Video: https://youtu.be/H3-kp1EsUxI

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and mixed pair product development

[00:00:00] I'm Pritish Krishna Panda, and yes, my last name is Panda, like the animal. Three out of three people actually asked me this: is it really Panda? And I said yes, it's actually Panda. Thanks to my parents.

[00:00:24] What we are going to talk about today is Always in Beta: using scenario driven design for better understanding of user needs or user behavior. Today I'll be discussing a case study, a product where we saw some usability errors, and then how we changed it or how we enhanced the user experience. Before that I will talk about my team, how we work and why this approach is the best approach for us.

[00:00:55] It was last year. Namir, who is my CTO, invited me for lunch. He was eating, I was just sitting there. And he showed me the portal: this is our software, how can you improve it? I said, well, okay, the icon looks wrong. He said, yeah, attaboy, that's what you have to do for the rest of your life. I said, okay, cool.

[00:01:20] He hired me as a UX designer, and now the ratio in our company is one designer for every five engineers, and those five engineers include both front end and back end. For any project, whether it's a small project or a big project, we try to divide it into small chunks so that it is easy to manage. And for each project, we'll have one developer working very, very closely with one designer.

[00:01:54] We call this approach mixed pair product development, where a developer and a designer work together and literally sit next to each other. It actually happens. And although the icons suggest holding hands, we do not hold hands. It's a problem. So do not get confused with that. It's mixed pair, and it works all day.

[00:02:16] But why is it really important? First of all, when I started working with the developer guys, I came to know a lot about how a developer's mind actually thinks. Just try to go to a developer and say, okay, make the fellow less pink. He will hate you. They're like, what pink? They understand the hex codes, they understand when the animation is happening in transformation, and all the transitions, how much duration, what kind of curve. So know how to talk their language. That's very, very important.

[00:02:50] After working very closely with them, it gave me insights about how they think, and they also came to know how a designer thinks. And that helped a lot with skill transfer. For example, when a designer wants to make a change to a button, what will he do? He will find the button, he will go to maybe Illustrator, he will change the color. That's not how a developer does it. He will go to the web page, he will right click, he will inspect, he will go to the Chrome developer tools and there he will make the change. It is much easier and faster, and that's how you can know what HTML code is, even if you are a designer. He taught me many things, not only him, both people sitting around.

Scenario driven design: people, context and purpose

[00:03:38] Scenario driven design. The main focus here is three aspects: people, context and purpose. As designers, we are very good at understanding what people need, and we are able to empathize with people. Developers are good at content, or product; they know what the product is. But they don't care much about all this design thinking stuff. Trust me, in my previous company there used to be workshops where they were teaching about design thinking and everything, and the developers were going, who cares?

[00:04:20] Purpose is where people and content come together. For example, if I say, make a booking on a website. In that case, the user will go to the website, he will interact with the website, and he will perform a booking. That is something really interesting for the developers, because they want to know how their code is actually performing. So instead of saying we are going to do design thinking and everything they hate, we just said, we will create some scenarios and you will see how your code is performing. And they were really excited about it.

The call flow designer and its problems

[00:04:59] This is the application we are going to discuss today. This is the most complicated application in Blueface. And by the way, those are not the real colors. Those are from my local machine, so if they are not looking good, it's fine. They're not the actual colors.

[00:05:15] What is a call flow? Since we are a communication based company, using a call flow you can design your calls. If a call comes to your number, where will it go? If that number is not able to answer, where will it go from there? You can program your entire call in a call flow. Can anyone tell me what to do next, just by looking at the screen? If you have to design a call flow, there are some elements. The page is called designer. What can you do? Any answer?

[00:05:51] Drag and drop. Drag and drop. Any other answer? You are smart, but not all the users are. They were not able to figure out what to do just by looking at it. That was one of the biggest usability errors. So first we wanted to know what the problems with the system were.

[00:06:13] Always ask the people what their current pain points are. You can do this using surveys, or just chat. Casually chat with them, and you will get some feedback. Listen to it. About the call flow, these are the problems they listed the most.

[00:06:33] Deleting an element is too tough, and I will tell you why. To delete an element, you have to drag and drop that element into the corner of the page, where there is "delete this element". So you will drag an element from here to here. I know it sucks. Most of the people complained about it.

[00:06:52] Second, in the call flow there are certain rules: after some elements, other elements cannot go, or they do not belong together. How can you know, since there are about 15 elements? That was one of the problems. Sometimes, working with one call flow, they want to switch to another call flow. How can they do it? That was also one of the biggest problems. And fourth, they can't view the number linked to a call flow. In a call flow, you can link a number. I am not going too deep into the technical stuff. But these were the problems, and we had to come up with a solution to remove them.

Validating feedback with small scenario tasks

[00:07:38] Apart from that, always validate your feedback with some tasks. This is my favorite part. You got all your feedback on where people are facing problems; then always design small tasks around that and go back to the user. For example, if they are not able to delete an element properly, just ask them, okay, create a call flow and delete your elements, and observe where they are having the problem.

[00:08:02] We always create these small tasks, with three scenarios per exercise. Very small tasks like add an element, delete an element, switch a call flow. The maximum duration of the exercise will be 20 minutes, so that anyone can literally join us and do it. And five participants for each task. We try to minimize; five is the most optimal number, trust me. Why, I don't know. Actually I do know.

[00:08:40] We created these small tasks, we told them to perform the tasks, and we observed what problems they were facing. Because we knew about a few problems, but that was not the entire picture. Sometimes, if you ask something about a product, they will tell you what's good and what's bad. But that's not 100% of everything. As designers, we try to observe where people are actually facing problems, and these are the things we noticed.

[00:09:08] New users getting stuck in the beginning with what to do. Some people can figure it out, like you, smart, drag and drop, but not everyone can. Deleting elements by mistake: you dragged something, dropped it, deleted it, but you did not want to delete it. So what to do next? By the way, these are some basic functional usability errors.

[00:09:31] And this one is one of my favorites: users using older call flows to create new ones, thus saving time. There was this guy, and we always used to give him tests: okay, perform this task. And he always used to do it really, really quickly. Really fast. And we asked him, how are you doing it? All the other participants are taking a lot of time. And he said, oh, I just copied one of the call flows, it had all the elements, I just changed the data, it was fine.

[00:10:00] And errors are tough to read, because whenever you try to put an element where it does not belong, it will show you an error, and it was very tough to read. By the way, these are problems which no one mentioned in the interviews or in surveys. These are the errors which were discovered during the scenario testing.

Designing high fidelity prototypes around scenarios

[00:10:25] Now, coming back to design. Now we know what the problem is, and we have to design. About the developers: they do not like wireframes. Trust me. Just go to them with wireframes which are not very detailed and they will say, I am not able to understand it. They need to know the exact specification, what the color code will be, where it will be. You just cannot go to them with a pen and paper drawing and say, okay, build this. They do not like it.

[00:10:58] How do you achieve high fidelity prototypes in a rapid prototyping mode? First of all, create libraries. Whenever you are hiring a new designer, ask him to create his own library. Not his own, but the one which is existing, and he will recreate it. I prefer Adobe XD for it. I have an Adobe XD page where I created all the buttons, all the drop-downs, everything, when I joined the company. It helped me to understand all the UI aspects of our company, so it is very important.

[00:11:30] Whenever you hire, just ask them to create a library. It will help a lot because, first of all, developers like high fidelity prototypes, and using Adobe XD you can create very high fidelity in very little time. Build the library first, and design keeping scenarios in the center.

[00:11:51] I always keep saying that scenarios are more important, because a scenario combines both the user and the context. The user alone is not what's important to us. And the website alone, when no one is using it, is not very important. When the interaction happens, that is the most important thing. And we always try to build the scenarios around the interactions where people are facing problems.

The redesigned call flow

[00:12:22] We got all the reviews where people are facing problems, and now we changed it. These are the real tickets we were working on. Deleting an element: now you don't have to drag and drop. There is one menu where you can just delete. This was one of the solutions. This was the best solution.

[00:12:41] And while adding elements, the system will now tell you which arrangements you are allowed to make and which you are not. If the element placement is valid, it displays the color code for a valid one, and when it is not valid, it shows a warning color.

[00:13:13] Our error was just bad. It used to disappear after every two seconds, and no one could read it. Can anyone frankly read it? No. So we changed it to an error alert. Three lines, not more than that. If you want to know more, click to know more. It was so easy. And it was not disappearing automatically; you have to click it: okay, got it. Because we wanted people to know what error they are making so that they won't make it again. An informed customer is always a good customer. So make this point.

[00:13:48] Whenever the call flow gets really huge, you won't be able to know what each element is doing. So we added this function where you can add small notes to the elements, so you can easily recognize them when you see the call flow. This was one of the best usability features we introduced. And now there's this tag: drag and drop elements to create your call flow. When you see the call flow, now you know what to do next.

[00:14:20] This is to generalize your flow, so the user won't be stuck anywhere. We wanted to make sure that as soon as you reach a page, you know what to do next. And we also introduced this undo function, where if you delete an element by mistake, you can undo it.

[00:14:36] We just tried to find out what problems the customer faces, and we designed scenarios around those problems. We were not focusing on redesigning the whole product. We just wanted to find the problems the customer is facing and remove them. That's the best thing. When you don't need to reinvent the wheel, don't reinvent the wheel.

[00:15:01] And there was this user who was creating call flows very easily. We asked him how he was able to do it, when other people were taking a lot of time. He said, oh, I'm just creating a duplicate and copying it. So we thought, okay, we can use this as a feature. We already had all the stuff; we just needed to put a button there. When you click on the button, it will ask, do you want to create a new one or do you want to copy? And that saved a lot of time for lots of customers. About 80% of the time.

Testing: easy to discover and easy to understand

[00:15:35] Now, testing. Since we always keep running different versions, how do we test? We check for two very important things: whether something is easy to discover, and whether something is easy to understand. These are very basic tests. How can you test if something is easy to discover? Just a first click test. If you have changed a button somewhere, give the test to the user. The scenario will be: find the button and do something. And just see how long it takes the user to find that button. That's how you will know. If he is taking a lot of time, that means it's not very easy to discover.

[00:16:20] And this is my favorite, because my master's thesis was around this: think aloud usability testing. How to actually know what the user is feeling. Just give them some scenarios and look at them. By the way, this is the best thing about being a product designer. When normal people keep looking at you, they are called creeps. But when we do it, it's called research. So we can do it.

[00:16:47] Just ask them to do something and look at how they are doing it. Observation is the key here. The more you observe your actual users, the more you will know about your product, and the better the product can be.

Culture and hiring

[00:17:01] And about culture: testing is not just a process, it's a culture. Never ever ship a product which is half built. Always make sure that all the components are well tested. And keep the developers in the whole process. Since we always try to make sure that the developer and the designer work very close to each other, there is always a problem with how they think and all this kind of stuff. For this, hiring is really important, and I will talk a little bit about how to hire the people who will be fantastic.

[00:17:37] I am a designer, so I know about designers and I will be speaking about designers: what kind of designers are the best to work with. There are people who do jobs from nine to five. It's perfectly fine. And there are people who go to a job just to get their money and pay their bills. It's also fine, not a problem. But there are some people who are never not designing. They are always involved. Whenever they see an update in any of their favorite software, they will go crazy.

[00:18:11] Those are the kind of people you actually want to hire, because they never stop designing. Even if you tell them to go home, they won't. That's a bad thing, I know. But trust me, for you as a product manager or anything, those are the kind of people you want to work with, because they are crazy about the product. Not only about the product, but about the whole design process.

[00:18:31] This is one of my favorite quotes, and it's very relevant to testing. Ever tried, ever failed, no matter. Try again, fail again, fail better. Thank you, and questions. Thank you.