OK Cool, But What Do We Actually Do? Making Product Design Work in the Real World

Evan Cannarozzi at UXDX Community: AI Speed, Real World Design. Video: https://youtu.be/E1xvuaCXReQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and the plan for the session

[00:00:07] Rory: I'd like to introduce our first speaker for this evening, who's Evan Cannarozzi, who is the product design lead at Ernst & Young. Evan will be talking about making product design work in the real world. So I'd like to welcome Evan to the stage. Over to you.

[00:00:22] Evan: Thank you very much, Rory. Great to be here. Thanks for having us, UXDX. I am Evan Cannarozzi. I'm a product designer, and honestly the only thing you need to know about me right now is that I love what I do. Because I love what I do, I'm really excited to get through a ton of information today. For the next 25 minutes I want to talk about the realities of what it means to be a designer today going into 2026, not just with AI but also with real-life team constraints. My session is called "OK cool, what do we actually do?" because it's focused around real-life scenarios, day to day, within my design team, on how we operate.

[00:01:07] The first half, I want to talk about building an operating system behind your decisions. What this is, is how we can think about all of these parameters of being a designer and narrow that down to what we're good at and what we love. Designers are getting pulled in so many different directions right now. We're being told to master Figma, understand code, adopt AI, think like the business, move faster, ship cleaner, all these things that can be overwhelming. How do we narrow that down to a working path that works for you? In the second half, I want to show these methods and put them into practice: how my team operates day to day, how we're using this operating system in our mindset to make sure we can achieve these goals, and what reality looks like within design teams at big companies or small companies.

What kind of designer do you want to be

[00:01:57] Let's get into it. Let's build our operating system going into 2026. As I said earlier, there's an explosion of AI tools out there. I think everybody's up to speed in terms of what we can do and the capacity we have to build full-on applications, back-end support, AI integrated in minutes. It's incredible. But is that for everybody? No, it's not necessarily for every workstream. Whether you're fluent in Cursor, you're a vibe coder, you're using tools like v0 or Replit, that is great for stakeholder prototypes, and that might even work for back-end support like we mentioned. But does it really work for every person? The answer is not necessarily.

[00:02:42] What I want is for everyone to think about what designer they want to be going into 2026, and I have these five categories to try to help that thinking. These five categories describe all of the parameters of the design spectrum and where you might fit. There could be someone called a builder. This is someone that stays close to engineering, understands how products are actually built, with technical feasibility in mind. There's the translator, which focuses on business alignment, speaks with the leadership teams, works closely with the PMs. There's the orchestrator: that's someone that maybe weaves AI into every daily workflow. They can help reduce friction, speed things up with vibe coding, maybe speed up user research.

[00:03:25] Craftsperson: this is someone who obsesses over quality and execution. I'm personally in this bucket, because I'm still a traditional Figma user, and I'm still using that as my latest design tactic to understand how we're building from a product perspective, but also still understanding user needs. And then there's the generalist. This is somebody that does everything. Not ideal; it's a lot of processing power. It's someone that may try to juggle all of these areas across the design spectrum. But we're not looking to be a generalist. What we want to do is narrow this down to maybe three or fewer categories. Think about what works best for you and your team. Think about what you love within the design space.

[00:04:07] Because at the end of the day, we have to remember we're in this field for a reason. In the creative industry, you should love a good amount of the stuff that you do. Even when we're overwhelmed and we're busy at work, it's still fun. So focus on what you're good at and what you love. In my case, with our team, I consider myself a builder. I know a little bit about code; I like to keep engineering in mind. I also love speaking with the business, so I stay close to our PMs and I help frame design decisions around outcomes. And I'm a craftsperson, as I said: huge Figma guy. I understand vibe coding tools out there do a lot of the same things, but right now what works for my team, realistically, is we still use Figma day to day.

The death of the delivery guy

[00:04:52] Another part of our operating system is confirming the death of the delivery guy. As you can remember, when we started, early in the UX/UI days, a lot of the time we were just screen pushers. We would just throw work over the wall and wouldn't question anything. Fortunately, I feel like that era is finally done. We are now thought leaders. We are taught to question things. The types of questions that we should be asking now are things like: who is this actually for, if it's feature-based? How many people does this really impact? And what changes if we don't build this?

[00:05:28] Real examples: if we're in stakeholder meetings, a lot of the time some of these feature-based questions come up that may not merit an actual feature. It's something that we could think about, and that only comes up if we question or push back a little bit. That alleviates some of our backlog: maybe this isn't dependent on being a feature. It's also good to frame this in the sense that not every problem deserves a feature. So it's always good to question things.

Speak the language of the room

[00:05:58] Another part of our operating system is speak the language of the room. Getting caught up with all this explosion of AI, and seeing all these LinkedIn posts about cool ways our designers are now building things with back-end support and everything speeding up, there are some things that don't change, and language is one of them. Language really matters. Even in today's climate, with all the advances we have with technology, it's really important to skip the UX term talk sometimes, especially when talking with end users or stakeholders, because we want to be transparent, we want to be honest, and we're looking for better outcomes for them.

[00:06:35] Stakeholders don't really care about how smart we sound. They care about outcomes. They care about revenue, and how our teams can work faster, meeting their standards, not our own. We actually want users to spend less time in our application, not more. We want quicker happy paths. You want to get them in and out, understand the context, understand the goals, and that helps when we frame language in a way that they can understand, that's not considered too technical.

[00:07:04] An example of this: a very typical term in UX is conducting usability tests. In our frame, we're thinking about not testing but having conversations, because at the end of the day that's what we're really doing. We're just having conversations with these people. It fails when we turn it into a performance. So this is about dropping the scripts, dropping the checkbox mindset, and just listening to people. That doesn't change. Even with the advancements in technology, the way that we talk to other humans doesn't change. Real insight comes from honest conversations, not perfectly worded tasks.

[00:07:42] An example of that: think about if you're a UXer and you need to get some validation on a feature that you're working on. Sometimes we'll send out calendar invites for something that we need to test, and we might put in the subject line "Usability testing session, Q1 feature validation. Hi Erica, we're conducting a usability test to evaluate..." whatever. They may not even get to the body of the email before they brush it off. So have the empathy to understand that these people are busy. Especially depending on the quarter, they may not have the time to give us what we need for feedback and meet our own UX deadlines. Real talk could be something like "Tell me how you really get your work done, 30-minute chat," just as an example subject line, and then in the body writing something like "We're working on improving part of the product. We'd love to hear how you really use it today." Something that resonates more with them, not just our scripted UX talk.

Partnering with product managers

[00:08:42] Let me go back a second. Another part is PM partnership. This is a big one. What we want to do is make friends with our product managers. What we actually do is we want to make our wins their wins. We really want to help them shine. To do that, we want to prove our output, meaning we want to prove to our PMs that, with the advances in Figma and the latest design tools out there, we can provide quicker turnaround times. We can do prototypes that used to take days in minutes. We have the technology. We can say yes more often. That's investing in our PMs, to let them know that we're able to provide better outputs, meet them where they are, deliver consistently, and rebuild that trust a bit. This isn't just theory. This is how we get our PMs to invest in us so that we can invest in them.

[00:09:45] An example, and this is realistic to how our team operates: before we had strong relationships with our PMs, the UX teams would gather requirements led by the PM, but we would do all the work, as expected; that's our job. In the beginning of these meetings, UX would sometimes be less involved in feature requests from the jump, and we'd also sometimes fall into the category of the delivery guy. If you remember, I mentioned that pixel pushers throw work over the wall. Sometimes, because of time constraints, that would still happen.

[00:10:15] But after that strong relationship is made, PMs and BAs gather requirements for us, by their choice, proactively. We didn't ask them to do this. They actually take the initiative to do Mural boards, doing all the groundwork, problem statements. They'll do meeting notes, they'll gather requirements for us, and they'll deliver it to us so we can review it and correct it in real time, collaboratively. That's a huge win for us, and it's because we have that stronger relationship now. We're now getting roped in earlier in UX calls, and the PM looks at UX more as a valuable source for the business, not just design.

[00:11:00] There's debate, there's pushback, but there's also respect, and that's important. That middle layer of respect is always there. It's also because we are showing, not just telling. We are giving them some design control so they can give us some business control.

Recap of the operating system

[00:11:18] As I said, the first half is to talk about building this operating system. We've defined the type of designer we want to be. We've narrowed down that list of five to maybe three or fewer. Who are you? Are you a builder? Are you an orchestrator? Are you more involved on the dev side? Are you a vibe coder? Are you a leadership person, where you want to get closer to the PM and the business to understand things like ROI and how we can make a bigger impact reaching business goals? That's up to you. But the point is we want to avoid being a generalist. We want to really narrow down on what you love, what you're good at, and where that sits within the design spectrum.

[00:11:56] Next, we confirmed the death of the delivery guy. That thinking is out. Thankfully, we are not pixel pushers. We are thought leaders. We are taught to question things, and we push context. We've defined language. We don't always have to speak in UX tech terms. It's okay to be transparent and have honest language, and our users would thank us for that, and the business as well. And the last piece is the PM is your friend. We've convinced our PMs that we trust them and they should trust us, and we're able to provide quicker outputs, quicker prototypes, meet them where they are, and they'll in turn give us things like requirement gathering, and set up digital whiteboards with all that work mapped out for us, because of that investment.

The design process in theory and in reality

[00:12:45] Now the second half starts. This is the part where I'm going to show real examples of how our team operates with team constraints. A lot of the stuff I'm showing is fictitious data, but the methods are real, and it's about how our team works together. The design process may only work in theory, not on your calendar. What does that mean? The perfect landscape, left to right, from discovery through production, in theory should have staffed team members, something like this, where we have a research team, a design team and a launch team. In this landscape, from the left side, something that should be very familiar to us is discovery, research, interviews. Then the design phase could be prototype, test, iterations. Launch could be dev, UAT and production. Everyone might have their own interpretation, but essentially it does the same thing.

[00:13:44] Now, with the integration of AI, AI impacts every phase of the life cycle. This is ideal, but it's also curated. In an ideal world, and this is particularly in perfectly written case studies or LinkedIn posts, it's crafted around the fully staffed product team. There are few hiccups, ideal timelines, UX life cycle steps are covered and validated, products are shipped, everybody's happy. In a lot of cases that's true. But what about when it's not true? In this case it's true because we cherry-pick the success stories. We tell what's smooth, what's clean, what happens with strong outcomes. We don't really include the bumps or the detours. And then we post. We post the perfect case study, highlight the outcomes that worked out really well and the wins, but we omit the constraints and the trade-offs.

[00:14:38] In reality, it might look something more like this, where the design team is narrowed down to covering all of these steps themselves, discovery through iterations, and then launch is still its own team. But AI isn't necessarily integrated in every step. It's cherry-picked in terms of what works best for us. So how do we handle this workload when we're maybe not staffed with full researchers and full design, when we're maybe down to a smaller team for whatever reason? How do we handle that workload?

[00:15:13] In this unfiltered approach, before I move on to the next step, what's important to note here is that this is realistic, because a lot of teams might be smaller than you think. How do they handle scope change and tighter deadlines? If I were a hiring manager and I was looking for a candidate, what I would want to see in a case study is what actually happened. I want to see the real story: the shifting scope, the tight deadlines, the trade-offs. And I want to see how that UX team came out of it. A lot of the time it's no one's fault of their own. Sometimes the reality is we don't have a fully staffed product team, but we still have to meet our requirements. So I want to see how that's actually crafted. What I would like to see posted is an honest version of the work: what changed, what broke, what got cut, and how we still came out of it and met our deadlines. That's where the methods kick in, which I'm about to show here.

Focused, deep and lean phases

[00:16:08] Left to right, ideally, everything like this should look the same: discovery through launch, equal effort, equal time, everything should be evil... I'm sorry, not evil, even. The reality is it's not. They're measured differently. What I'm showing here is each UX stage is actually measured a little bit differently, and I can show how our team operates to meet these goals. There are three categories here. If we look at this beige category, focused, this is a measured phase where our team uses a mix of AI and humans to get that work done. There's also this other category called deep. This is for something that's more human-led. In our case it's for requirements: little AI, mostly humans, crafted with the larger team. This is where we spend a lot of our time. And then this third box, this narrow one, which is this orange color, is lean. This is for something like refinement, an AI-accelerated approach to get things done quickly and efficiently, based on how we built that process.

Design and test with AI agents

[00:17:20] Let's talk about this. Let's talk about design and test first. This is a hybrid approach. As I said, this is measured as part humans, part AI, in how we can actually get this stuff done from a design and test perspective. If I go to an example here, again this is fictitious data, but the methods are real. The first thing that we do, which I hope resonates with a lot of you on this call, is how we're testing designs. It's a no-brainer. We set up time with our end users. Step one would be maybe we schedule a 30-minute call. We typically state the problem statement and we show our designs. In this case we have a UI, we're talking them through it, with our typical scripts, making sure we don't get any biased feedback, and then we capture it. How do we capture? We usually use Copilot, but very minimal usage of AI. How you capture feedback is up to you; that's how we do it.

[00:18:21] The second step is we're now actually leveraging AI agents, but in a little bit of a different way. We're using them to help build personas. What my team does is, after that call with the stakeholders or the end users, to get that first human response for feedback, we build AI agents with the minds of five, six, seven, eight business users, however many we can capture. We configure these agents with the minds of the business so that we can leverage our own design work to test with those agents. As I said, you can configure them however you want. You can supply documentation. This can be used in Copilot or ChatGPT; any LLM can do this. Try to stay as close as you can to how the business actually operates. The context really matters. But this is how we set up our agents. It takes about five minutes, maybe less.

[00:19:17] The next step is we test with the agent. In this case we have Grant. He's one of our AI agents. He's a business user; we've configured him to be, and this is just an example, a financial manager aligning production schedules and quarterly reporting. He's our guy, and we test our designs with him through Copilot. The way we test is twofold. The first part is we make sure our prompts always start off with something like "unbiased, direct feedback," to make sure we get the transparency from the agent. The second part is you can take it further and actually ask the agent to output it where it highlights everything in green that was clear and provided clarity, and highlights in red any friction points, so we can easily understand what went wrong with the design.

[00:20:06] Here's an example of an output. The AI agent tested it and gave us a response on the screen, on the screenshot of the UI that we uploaded, and we can see in green here the things that went well, and in red here the things that were friction points. Maybe two minutes for that response. The last part is we compare the AI to the human response. This is something you can do with the same agent. You upload the responses you got from the human and say, "Now compare between what you reviewed and what the human reviewed, and find commonalities," and for the most part it does.

[00:20:45] All of those steps, from two to five, for AI, can be done in under 10 minutes. This used to take multiple meetings, multiple calls, multiple people. Now we're able to consolidate that down to where we have one or two humans in the loop, if time permits, because we know our stakeholders are busy, and AI does the rest of the grunt work for us. Now, this isn't bulletproof. We do try to test as much as we can with humans, and we fortunately have a lot of really good stakeholders that participate. But if we need anything in between or during sprints, that helps accelerate this process, and this is where our AI agents kick in.

Human-led requirements

[00:21:33] We've got two more here. Requirements: as I said, the one before, design and test, is more AI. This one is more human-led. Our requirements are actually shown to us like this, which is really cool, because our product manager would provide something like this for our UX team. They'll give us the problem statement. They'll give us imported table data with certain things that they find are critical for us to review. They'll have meeting notes. They'll even go as far as doing affinity mapping. It's important to note here: we don't ask them to do this. They proactively did this because of that partnership, as I said earlier, between us and the PMs, which is awesome.

[00:22:17] They come to us with affinity mapping because they're starting to understand the UX process, and they do this work for us that we can help review. We then in turn do journey mapping, whatever we have to do before we start designing. But I love the fact they take the initiative to do this work for us. That's the requirements part. I don't have to go through the rest of it; I just wanted to show the process of how those PMs work with the business and the BAs to get this to the UX team.

Refining in Figma

[00:22:44] The last part is refine. Refine means an AI-accelerated approach, but also a traditional method, with how we operate in Figma. When we're refining design, you can go two ways. One way is vibe coding, which a lot of people are doing now, and I think that's great. But does it work for everybody? No, it doesn't. In our case, we are not at the vibe coding stage yet. We may get there one day, but we still have a lot of control with how we use Figma. Within Figma we have fully loaded Figma canvases: everything from using 100% auto layout to variables, variants, stage components, interactive components, you name it. It's fully loaded. Our auto layout is used everywhere. Even the outermost container, as you can see in this screenshot, is auto layout, so that our screens are completely responsive.

[00:23:42] Here's an example of a screen. Again, this is fictitious data; I designed this just for the demo, but this is a real-life method for how we operate live on calls with PMs, where these screens are easily adjustable so we can adhere to feedback in real time. These are set up with our stage components so that everything is set for us and easily fetchable. We even have meeting notes from the last call that we had. We have our problem statement. Everything is included. Our AI agents are here. For example, if we have a grammatical issue, or we need a content writer, we have our agent that we can link out to, to ask additional questions as needed.

[00:24:23] But the point is we're able to pivot. If we need to move any of this stuff around, if we need to duplicate tables, our tables are flexible. If we need to pivot anything around, all of this is at our fingertips, so that we can show our PMs we can adjust and adapt in real time and get them their answers quicker. All of this maps to benefits for the PM, the dev and the designers.

Summary

[00:24:52] So what did we learn here? We built an operating system. What kind of designer do we want to be in 2026? Hint: it's not a pixel pusher. We've confirmed the death of the delivery guy. That era is done. We are not screen designers; we are thought leaders. We provide context. Language matters. We are trying to avoid UX jargon. We want to have honest conversations, not tests. PM partnership: we invest in them, they invest in us. Fantasy versus reality: that's the real examples that I just walked us through, on what works for potentially smaller teams at large companies, and realistic case studies on what actually went wrong and how you fixed it. And then the sixth tier here is using AI selectively. No hype; use as needed. Let me leave you with this one question: what's one thing you're doing today to build trust with your PMs, your devs or your design team? That's it. Rory?

Q&A

[00:25:55] Rory: Thanks very much. That was great to see. You've obviously put a lot of effort in, and you've iterated a lot on your process of how and when you bring AI into the equation. I'm going to go there with probably the most controversial one, which is your agents, and what people are calling synthetic users. What has been your experience there? A lot of teams are saying, why don't we start running before we even talk to people: we run a few iterations through the synthetic users, refine it, and then start talking to people. What's your take there? Do you trust them enough? Is the feedback you're getting real and valid feedback, or is it hallucinations?

[00:26:42] Evan: I would think that's the most controversial, so I agree with you on that, and I would say no, it's not completely trusted. That's why we always have a human in the loop. It's a great question, Rory, because I personally don't feel like it's ever going to be left unattended. I think it's an enhancer. It's augmenting; it's not replacing. We always start with a real person. Like I said in my step-by-step process, we start with a real person just to understand how they perceive it, because we want to still offer that real human empathy. What we do with the agents is comparing. We use the agents to compare to the human responses to see if they line up, and a lot of the time they do. That's why we use it as an enhancer, but in my mind it never replaces.

[00:27:30] Rory: But would you trust it for a first pass before you go out to users? Because the way you mentioned it, it was in parallel.

[00:27:37] Evan: Not a first pass, to be honest. It could just be my hesitancy with AI. I think I would always want to start with the real person, not AI. Maybe I'll change my mind one day, but right now my order is person first, AI second.

[00:27:54] Rory: As you said, you've got that builder part, which was one of your profiles, and you're still at the Figma stage, as you mentioned, through the flow. So maybe, who knows, there's a 20-year-old out there who's doing everything with synthetic users. We're the old guys now.

[00:28:11] Evan: Could be. We'll see how that changes, but we are still operating using some traditional methods.

[00:28:16] Rory: Excellent. I love the example of how you're merging the new and the old, because I do think if you go completely AI, you're opening up a lot of risk.

[00:28:24] Evan: Yeah, agreed.

[00:28:24] Rory: Excellent. Well, thank you very much.