The Role and Responsibility of Prototyping in Product Development

Alex Radu at UXDX EMEA. Video: https://youtu.be/QdxZLYBlP8Q

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

How often do you prototype?

[00:00:07] Alex: So we're going to talk about the role and responsibility of prototyping. I have mixed feelings about this, because I don't think it's just the UXers' and researchers' responsibility, so we're going to see what you all think today. I will need you to scan this lovely QR code. It's also where you're going to be able to give feedback at the end of the session, and we do appreciate your feedback, so please take some time for that, as we all love to grow. So we're going to see how this is going to change.

[00:00:39] Alex: All right, let's see. So we've got always, most of the time. Oh, we have a lot of people that prototype here, I love this. Some people sometimes, some people never. Most of the time has gone up. We don't even have percentages anymore.

[00:01:00] Alex: Okay, people keep voting. So most people say that they prototype always and most of the time. Is there anyone who would like to answer to always? So if you prototype always, what do you actually do and what is your role? Are you a UX researcher, are you a designer, are you a content writer? Anyone that said always? And don't feel pressured, we're in a safe space here. No one, no hand raised, come on, it's like 22% of you over there. Anyone, hand raised? Okay, the gentleman in the front, please introduce yourself and tell us.

[00:01:38] Audience: Yeah, my name is Rafael. I always prototype, like, even sometimes the low fidelity designs, to pitch it to the developers.

[00:01:49] Alex: Okay, so we have an idea, and then?

[00:01:52] Audience: From my perspective, prototypes are the time for us to discuss with them the feasibility of it. We can have an amazing idea that will cost them a year of development.

[00:02:01] Alex: I like that. So you quantify how much effort it would be to implement that. Do you talk to product at all?

[00:02:09] Audience: Yes.

[00:02:12] Alex: So you talk to them about the importance to the customer, I'm guessing, and whether this is really needed, what's the value, and how much do we need it?

[00:02:19] Audience: Yeah, absolutely.

[00:02:21] Alex: Okay, awesome. Anyone else who said most of the time? Why most of the time, what do you do? You need to raise your hand high because I can't see, because of the blinding lights in my face. There is someone, someone. I can see Maria walking. Okay, gentleman, I will move to see you.

[00:02:40] Audience: Hello. All right, sometimes you're just doing smaller things that you can just roll out. It's not major improvements, you just know what to do, so just do it sometimes. And most of the time you can just roll back if it doesn't work. So of course if you're doing major things you have to validate it, but sometimes you can just test it. If it takes two days to do it, just do it, and if you have enough data you should see if it's succeeding or not.

[00:03:08] Alex: Yeah, awesome. And what is your role, if I may ask, what do you do?

[00:03:12] Audience: Product manager.

Why we need prototypes

[00:03:13] Alex: Product manager, awesome, love that. All right, so has anyone ever been annoyed by something like this? I hate it with a passion. And do you think that these people prototyped and tried it out with people before they rolled out this? A raise of hands if you think they've prototyped. No. A raise of hands if you think they just didn't care and thought it was a good idea. Awesome.

[00:03:41] Alex: Okay, so why do we need prototypes? Because we don't want to annoy people, and we want to make sure that we're building the right thing for them. So we're going to go briefly, is there anyone here who doesn't know what a prototype is? I wanted to level set, but I'm just making sure that we're all in the same place. Okay, no hands raised, most people are familiar.

[00:04:02] Alex: So our friends at UXPin say that a prototype is an early version or a model for you to be able to actually test before you roll it out to the masses. Like the examples of those doors, which obviously they didn't really care about, they were like, well, we're not going to fix it now, we've reduced all of the doors with those handles, kind of late. And I think it's important to understand that a lot of the teams might consider that it is the responsibility of the researcher or the designer to prototype, but actually it's more than that. You can actually use it regardless of what role you have. So whether you're a product manager, whether you're a designer, whether you're a developer, you can use it to get early feedback and iterate on what you're already doing.

What are prototypes actually for?

[00:04:48] Alex: So what do you think is the primary purpose of prototypes? And I am warning you, I will ask some of you to share why you picked what you picked. So do you think it's about ideas, do you think it's about handing off to developers to see if it's feasible, do you think it's about usability, accessibility, aligning your stakeholders around something so they can actually visualize what you're trying to build?

[00:05:16] Alex: Okay, so a lot of people said all of the above. I'm going to challenge you on that, because if you said all of the above, how many of you actually do it for all of the above reasons? So if you said that, is there one of you that has done prototypes for those four different things? And I'm not talking about utopia, I'm not talking about what should prototypes be for. What do we actually use them for?

[00:05:50] Alex: So we have usability as a big, big item. The percentages have disappeared again, which is awesome. So I need two volunteers, for all of the above and for usability. Why do you think, why would you defend that this is what prototypes are for? And there is no right or wrong answer, there is just your answer and how you see prototyping. We have a gentleman in the middle over there. Where is our... oh my God, can I throw it? I mean, if I do throw it to you, please make sure that whoever's next to you actually tries to catch it and you don't just get hit. Thank you, almost there. Wait a second, I think it's maybe on now, can you hear me?

[00:06:35] Audience: Yes. Oh, there we go. Yeah, my name is Terren, I'm a product manager. I think context is always dependent. I'm working with a pre-launch startup right now, and so we definitely use prototypes for all of the above, because we have to use it to visually evaluate some ideas, and do some usability testing before we can put it into development, because it's costly. And then aligning with co-founders, and then the developers need something visual to work off, so we just translate that directly off the high fidelity.

Fidelity, GenAI, and when to prototype

[00:07:08] Alex: Cool, I like that. So there are some differences in how people use prototypes depending on where they're at, whether they're a startup, a scaleup, a founder-led enterprise, a big company, like I'm talking about 50,000 employees plus. And there's also the other thing of, I was reading an article, I think yesterday, on how designers, even though they aren't really expected to code, with the new emergence of GenAI and the ability of people to be able to code prototypes using GenAI, as opposed to having to learn how to code, it might start to shift a bit more where you have a working prototype created with code, which is typically the last fidelity level. So you'd have paper prototypes, you'd have low-fi, high-fi, and then you'd have code prototypes. But because of the emergence of GenAI and the ease with which you can now create code prototypes, it's actually a little bit of a change in perspective, where potentially in a year's time designers might actually be expected to code, in the sense that they might need to use a GenAI bot to be able to create a code prototype, as opposed to just doing Figma.

[00:08:27] Alex: So the answer, kind of generically, beyond it depends, is that you should use the prototype whenever you need to test customer and product assumptions. And that is whether you're testing a new information architecture for your knowledge base, whether you're designing a new feature, whether you're working on a new requirement as a developer and you want to get feedback from your team, whether you're a designer or researcher and you're trying to inform what your team are building. You should be using it to test stuff before it's too late and before you have already committed.

Challenges: time, resources and stakeholders

[00:09:05] Alex: So it's all well and good, but a lot of people will come and say, okay, but how do we make time, or how do we include this in our development process? Because if we prototype everything, how are we actually going to be able to do anything else beyond prototyping? And that's why we want to ask you, what are the biggest challenges you face with prototyping? So for the people that answered, most of the time, or never: why don't you do it, why can't you do it, who's preventing you, or what's preventing you from actually being able to do it?

[00:09:38] Alex: Okay, so we have, I hope someone's okay behind the stage. No time and resources, challenge finding participants, setting stakeholder expectations too early. I do like the stakeholder expectations. Would anyone like to talk about that? Okay, we're going back. Oh look at that, it's in your row, perfect timing.

[00:10:07] Audience: Thank you. I answered the stakeholder expectation. I think when you are early on in the product development you should be doing storytelling and more sketches, not prototypes, because you don't want to commit too much at that time, and have a much more open viewpoint, so you can get the stakeholders and everyone in line on the goal, what you're trying to achieve, not exactly how to achieve it.

[00:10:34] Alex: Yep, I like that. And how about the no time and resources? Would anyone like to share why no time or resources? And then I'm going to challenge you and ask, if you don't have time and resources now, then what do you do when it doesn't really work and you have to go back and fix it? How do you find time and resources then? I'm trying to see, any hands raised? Any hands raised? I know it's a hard question, but you can give me your perspective. We have the lady over there, you were smiling from the beginning but you didn't want to raise your hand.

[00:11:09] Audience: Hello. Yeah, I think for no time and resources, maybe it's more what the people who decide think about it, because they might want you to do other things, and then you have to kind of challenge that. And then sometimes it's not so easy to do small tests either. For us, we work in a big organization and we have to find the right people, and this takes time. So if we could just test a person on the street it wouldn't take that much time, but since we have to have the right participants it takes much more resources than other times.

[00:12:00] Alex: Yeah. And what is your role?

[00:12:02] Audience: UX designer.

[00:12:03] Alex: Okay. And do you have access to your end users or your customers? Is it that you're far removed from them, or do you have access if you did want to test?

[00:12:14] Audience: We have resources in the project that we can talk with, but they know the project so well that we don't really want to test on them, because they can do anything because they know it. So what we want to do is to test it on the people who don't know it, but then we have to go through many teams to get to the end. And then we have to plan, and then someone has to say that it's okay that we use the time.

[00:12:46] Alex: Yeah, okay. Does anyone have any suggestions? Have you encountered that in your companies? Have you had to do some of that, have you had to change hearts and minds to invest time in prototyping? Anyone have any insights? There's a gentleman in the back. You can throw, see how far it gets. Awesome.

[00:13:06] Audience: Hello, hello, can you guys hear me? Yeah, I am a product manager also. The biggest challenge, we work in a large organization, very large organization, so the challenge was always the stakeholders. They almost never find any value in prototyping, for example. And we do lots of decision science kind of products, so mostly the stakeholders, the leadership, they say, oh, why do we need prototyping at all? So what happened is, recently, one of the projects, when it came out, the outcome of the project didn't satisfy the requirements of the end users. And so that's when I could actually tell leadership the value, I could show for the next project the value of prototyping. Because we started with the prototype itself, even if it was a very low fidelity sketch, but we started with prototyping and then we evolved, we made it into a high fidelity prototype and things like that. And so the project of course went smoothly and the customer expectation could be met.

[00:14:16] Alex: Yes, so you went with their way of, we don't have time, we don't have resources for prototyping, okay, let's see what we get out, what's the outcome. And then after that you were like, well, let's try it the other way now, since we did this and we failed, let's see if this actually works. And then it did, so now you have kind of social proof to be able to say, well, next time when we start a project let's do it this way, because obviously it's a better outcome.

[00:14:40] Audience: Yes, and actually they are the proponents now, they come to you and they're like, so we should prototype this before we actually get to do it.

Where to start: low or high fidelity

[00:14:48] Alex: Thank you, thank you very much. All right, so we were actually talking about fidelity just now, of your prototypes. What fidelity do you mostly start at? And I know the designers here will be like, wow, Figma's my best friend. Does anyone here do paper prototyping? Do people do low fidelity anymore, or is it like, Figma is my best friend and that's the vibe? Okay, we have low fidelity, high fidelity.

[00:15:19] Alex: All right, I would like two volunteers, one that picked low and one that picked high, to help us out with why they start there and how this helps their prototyping and design process. Anyone for low? Anyone for low? I feel like an auction. Anyone for low? Oh, we have a person in the back over there. Where did it end up? Oh, you can throw it to the lady over there. Yes, okay, microphone, getting there, getting there.

[00:15:53] Audience: Hey, hello. So low fidelity helps you literally just copy whatever is in your brain, rather than just moving to high fidelity where you'll have to find the tools, the options, to just put it on the screen. But in low fidelity you can literally draw anything that's in your mind. It doesn't matter how it looks, and it doesn't matter if it's good or not, it's just, it's on the paper, whatever is in your brain.

[00:16:23] Alex: So it's like a starting point that you can iterate on, as opposed to, hey, here's my final design, what do you think? Oh, there's a gentleman next to you that has an opinion. Yes please.

[00:16:38] Audience: Hi, hello. Of course, I'm an engineer, but I work a lot with the designers. So I think if you have an established design system, that could also be a way. You can just start with it, the components are already premade, just put them together and bring the idea to life with that as well. So having a design system also could just jump you up straight away.

[00:16:59] Alex: So I have a question on that. When you first start working with your counterparts, so whether they're design or product, do you start with the high fidelity, so basically your design system based Figma designs, or do you have a previous stage where you go low fidelity before you go and make the actual high fidelity?

[00:17:21] Audience: Most of the time the high fidelity.

[00:17:24] Alex: Okay, okay. There's no room for iteration, it's like, we know what's in the customer's mind, we're good on that. Cool. Anyone else that wants to converse about why it might not be great to do high fidelity first? We have two people in the front. Oh, we have someone coming with the microphone, so you don't have to catch it. The first row lady from Italy, and then the second row gentleman. She was here before, that's why I know she's from Italy, I didn't just randomly guess.

[00:17:53] Audience: Yeah, so for example, I work in an automotive company and I work in UX/UI for displays for automotive, and for us it's absolutely high fidelity, because we must be sure that also the colors and style that you choose are really safe. I mean, people have not to have doubt about it. So for us low fidelity, but just for us, doesn't work, because of safety. I mean, if the product is really safe you can say just when it's finished, when you have the final look and feel that the customer wants to see. So that's why we go high fidelity.

[00:18:31] Alex: Yeah, no, that makes total sense, I totally understand that. And the gentleman at the back, are you low fidelity or high fidelity?

[00:18:39] Audience: Oh, I'm high fidelity. So yeah, the only disadvantage is that because we have a design system and we just jump straight into high fidelity, we're actually only presenting ideas, but the initial problem when I started was that everyone thought we jump straight to the final solution. And it took a little while for teams to get used to the fact that it was slowing the process down for us by sort of sketching everything out, when we could just go, right, we'll put it straight there and start to talk about it.

[00:19:09] Alex: So you had to do some education and change the culture of, even though this looks like it's polished it is not the final design, this is us just using the design system to quickly iterate, but don't assume that this is ready and this is it, this is a starting point. Cool, I like that, I like that. So you're educating your stakeholders to come to that same understanding.

[00:19:29] Audience: Yeah, but when I hadn't had them educated I actually turned projects around very, very quickly.

Measuring success and iterating

[00:19:35] Alex: Look at that, it's like reverse psychology of, I'm not going to tell you this is not ready so you can agree and then we can move forward. Cool. So, how do you measure the success of a prototype? And this is our final question before we move on to our next session. What do you think is good for you?

[00:20:01] Alex: Sorry, I'm looking at my next speaker to make sure that they're there. Right, how do you measure? But see, the last gentleman, sorry, I didn't get your name. William. So William said that when he didn't tell people that this is a prototype, they would deliver very fast. And I feel like that last bit there with rarely iterate, that's like, you have a high fidelity prototype, you show them, they're like, okay, yeah, this seems to meet the requirements, looks like what the customer wants, and then you rarely have to iterate, because it's like, oh well, this is kind of good. But do you iterate after you get feedback from your customer? And sometimes I want to know personally, why, and how did it go when you did ship faster? Did you have more negative feedback, and now that you spend more time iterating, has it improved?

[00:20:58] Audience: So with the team I run, all the designers would actually see all the prototypes and they knew how we worked, so they'd act as almost my user testers[?]. So if I'd gone off on a tangent and something seemed to be a bit too far out for the customers, they'd push back, and so I'd actually just tweak it here, and then I'd go back to the product manager and say we've made a little change, and yeah, they'd be happy.

[00:21:27] Alex: Okay, so the customer feedback isn't very different whether you shipped it faster or whether you take more time to review.

[00:21:36] Audience: Yeah, someone would see it at some point, I'd get feedback from somewhere.

[00:21:39] Alex: So you'd fix it before it got broken, pretty much. Yeah, cool. So I want to ask someone, what are some of the metrics that you use for user feedback? How do you quantify that success? Because I see 65% of you have said user feedback. So, okay, we use user feedback for success. What do you do? Do you do, I don't know, feature usage, feature feedback on usage, do you do actions tracked, events tracked, survey results, what kind of stuff? Oh, there's the gentleman at the back over there. We need to throw it now. William, you can do it, he stood up over there. Yes, oh, over-reach, but got there. Thank you, thank you.

[00:22:33] Audience: Yeah, I'm standing now. So we do it in a couple of ways. Survey is one of them, feature surveys basically, and then also doing an analytics of the usage of the features. So we do that, and then of course we have a way to do value analysis after the product is out. So we do it in these different ways.

[00:23:01] Alex: Okay, yeah, I like that, thank you. Anyone else on how they do it? It could also be internal team feedback, because I'm really curious about that. What do you mean by that, is it that a prototype is successful if you do get people from your internal team saying this is useful? How do you quantify that, because again, 20% of you have said yes to that. So what kind of things do you do? I'm trying to see if there are any hands raised. Any hands raised? No hands. Okay, a hand back where, sorry, I am blinded by the light. You have to get the lady in the back. Oh, Fernanda. Hello. She is one of my colleagues in Dublin, so that's how I know her too. Thank you.

[00:23:53] Audience: Hi again, hello. For internal feedback, what happens is everybody already worked on similar projects. They already got users' responses and feedback regarding previous prototyping and previous deliveries. So when you have a new feature, or a feature that we are just adding to an existing product, they already kind of know where to go, because we already had previous feedback. So when everybody engages on the team and says, okay, this is the right direction, we have everything we need here, and we're pretty sure that it's going to be a successful prototype as soon as we implement it, then I feel like it's a prototype that is actually successful, because the whole team already had previous experiences with that.

[00:24:36] Alex: Cool, awesome, thank you. All right, and how many times do you iterate? And again, maybe some of you don't iterate, maybe some of you iterate sometimes, but I'm curious to hear how many times you have to go back to your prototype design and go through it. All right, some people do it once or twice, three times or four times seems to be the general range, five or more. Again, maybe it is about the complexity of the prototype as well, it might be on how broad you're going, the scope of it. It might mean that you have to iterate more if it has many more components than if you're actually doing one small change or one small feature prototype.

[00:25:23] Alex: Okay, some people rarely iterate, and again, maybe you just really listen to your customers and you get it right the first time, so you don't have to iterate that much.

[00:25:37] Alex: All right, and I promise I won't grill you anymore, because we will be welcoming Petra shortly. So thank you so much everyone for your time. I hope you enjoyed the discussion on prototyping. Please don't forget to leave some feedback, and also please don't forget to leave your headset on the seats so we can sanitize them if you're moving to the Vision stage for the next session. If you're staying here, keep your headsets and we'll get started in a couple of minutes. Cool, all right, thank you so much.