If AI Can Build Products, What Are The Humans For?

Rory Madden at UXDX USA. Video: https://youtu.be/V4jL6JcQsH0

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Three categories of product development

[00:00:08] I want to give a talk about: if AI can build products, and it's getting scarily good at building products, should we be worried? Where do the humans still stay in it? And how I'm going to do that is I'm going to go through four parts of my talk. The first one is the state of AI development, so we'll talk about where it's at, where the gaps are, where the research is, and where they're trying to fill it in.

[00:00:33] Then we're going to talk about how roles are already changing. I've started to notice a lot of job ads and things are changing, and people are talking about how the specifications are changing. We'll ask the question that is on everybody's mind: are we going to see job losses? It's a bit of navel-gazing, so you'll bear with me, but I think it's backed by research. And then summary and takeaways.

[00:00:56] Let's jump into the state of AI development. I want to talk about it because product development is too broad of a term, so I just want to classify it into three distinct areas. One would be like a simple marketing site or a blog site or a brochure site, where there's very low interactivity. It's more of a one-way communication. The second is maybe a typical SaaS platform, where you have some forms, maybe a dashboard, some interactivity. And then the third level is very high interactivity. Think of that like a Figma canvas or a Miro board or Google Docs, where there's lots of things happening, real-time chat applications. So we've got those three categories.

[00:01:45] When we look at the first one, I actually think this is solved now. I don't think there's much, obviously there's levels of quality, but I want to give one example, and it's my brother. I probably should have asked him, can I put our texts up on a screen to thousands of people? But he was having a problem. He's not very technical, and he just needed a brochure-type site for his company. He had been burned by Fiverr. He'd been trying for literally weeks to hire somebody and was not happy with the output.

[00:02:16] What you'll notice here is I tell him, just try Lovable. And the most important thing I want you to see here is it was 34 minutes between his two messages that said "I'll give it a try", having never used the product ever, until he was happy with the quality of the output. So when you're looking at those low interactivity sites, I think it's a solved problem. But most of the people here today aren't really working in that space, and if you are, I do apologize. Most of us here today are working more in the high interactivity to really high interactivity area. So we're going to focus this talk more on those areas, because I think there's a lot more complexity there.

Walking the software delivery life cycle

[00:03:02] What we're going to do is go through a software delivery life cycle and talk about each step of that, and see what the current state of AI is as we're going through this. We'll talk about architecture, so that's the technical architecture: what databases are we using, what communication patterns, that type of thing. The UI design, then the actual coding of the product, QA, the testing of the product, deploying it to servers so that users can use it, and then the ongoing maintenance.

[00:03:39] I'm going to skip over QA and deployment, because for the last 20 years, 15 years, the DevOps movement has been focusing on automating this. So it's very highly automated. AWS: you can spin up servers, tear them down, you can get your code to production very quickly. So I don't think AI is going to improve that. I think we're already in a very good state here.

Can AI write code?

[00:04:04] But the question I want to answer first, I'm going to jump into code, so I'm going to do them in a slightly out of sync order. For me, I write code, and November was a game-changer, last November. If you talk to any developers who have been actively using these AI systems, the Claude Code release in November moved it from being "this is pretty good" to "this is better at programming than I am". And then they did another release in February that bumped it even higher again.

[00:04:39] This is the guy who created Claude Code, and he said in December, "In the last 30 days, 100% of my code has been written by Claude Code." And he's not alone. Obviously he's going to push it because he's creating the product. But this guy, I don't know if you've ever heard of Linux, but it was created by a, how should I put it, prickly personality? If you've ever heard about Linus, he's not known for his hospitality and his warm-heartedness. He gets very critical about bad quality code, so a lot of his critiques are very harsh to people who submit patches to Linux if they don't have high-quality code.

[00:05:19] And he is even saying, this was from January 2025, "Is this much better than I could do by hand? Sure is." So when you're talking about the guy who wrote Linux and he's saying it's a better programmer than him, that's showing you the quality that these AIs are doing at writing code.

[00:05:40] And the other thing, with code there's two dimensions. Is it writing good code? But also, to write code, okay, yeah, it can do something small and then I tell it do this five-minute task, do this five-minute task. Last year at this conference I was chatting to somebody and I think I said it was like two minutes I could leave Claude run before it would go haywire and I'd have to jump in and stop it. Whereas now it's in the hours. And that's my experience as well. You can give it a very detailed feature and it will one-shot it. It won't get it right the first time, you'll have to go back with a few prompts to fix the problems that it did, but it's pretty effective. What we're seeing is that exponential curve of it being able to take on longer and longer tasks. They're trying to do things like write compilers and really complex systems autonomously.

[00:06:34] So to our question, can it write code? The answer is yes. It clearly can write code, and it can write better code than most humans out there.

Can AI design good architecture?

[00:06:44] But can AI design good architecture? The difference between coding and architecture is like drawing a picture. I guess anybody can draw a picture of some flowers, but unless you have good architecture, you've got good practices and patterns up front, it's not going to be a high-quality output at the end. So from a coding perspective: is it maintainable, is it secure, is it performant, all of these things that developers think about when they're writing code. Is AI good at doing that?

[00:07:17] The honest answer is right now it's hit and miss. Sometimes it will produce the best architectural design, and then you ask it to make a change and it will do something absolutely crazy. So there isn't a consistency yet at that architectural level. The other reason for that is that AI needs context and it doesn't know your business. It doesn't know why you made a decision last week. It has the sum of all internet knowledge, but it doesn't know the ins and outs of your business.

[00:07:52] Here's just an example of somebody saying that they try different things and it's very frustrating because they're non-deterministic. One day it's amazing and you think, oh, they must have fixed all those bugs, and then the next day it's doing crazily stupid things.

[00:08:10] And the problem is, if you have bad architecture, bad things are going to happen. This poor guy was publishing on Reddit that "I vibe coded this app and it's amazing", and within five minutes of his post the app was down because he got hacked. Because he hadn't thought at all about the security of his system, and the AI hadn't thought about the security of his system. So it was very easy for people, with not much effort at all, to hack into the system and take it down.

[00:08:39] There's a saying in development: foot guns. You can shoot yourself in the foot very easily if you design things badly, and you can code yourself into corners. That's what we need to think about with architecture. Architecture is thinking about the plans up front so that you make sure that you're maintainable, scalable, secure, all of that.

[00:09:02] So the saying is that AI is an amplifier to knowledge you already have. An experienced developer will know the questions to ask and the things to check to make sure that it is doing good architectural design. And that's one of the challenges right now. It multiplies the discipline that you bring to it, but at the moment you still need that discipline. You still need somebody who's able to direct it in the right way.

The learning problem

[00:09:34] But then, this is the quote from the CEO of Anthropic, and he says we're six to 12 months away, and this was back in January. We're six to 12 months away from models doing all of what software engineers do. And what he means by that is architecture as well. I don't know. All I know is last year he said we're six to 12 months from AI writing 100% of code, and he got that right. So I don't know whether he's going to be right again.

[00:10:06] But the challenge is, right now AIs have a design problem. The problem is they can't learn. It might sound stupid saying they can't learn, but they can learn everything in their training data until the point that they're released, and once they're released they never learn after that. There's a problem called catastrophic forgetting: when it learns something new, it accidentally forgets something random. That's a problem that they're trying to do research on, to figure out how to solve it.

[00:10:37] So it knows everything from the internet up to a point, but you know your business. You know why you made certain decisions, why you didn't make certain decisions, and that context is really hard, because it just lives in your brain and it's not written down often, it's not very clear. It's just the accumulated knowledge of people in the building about what's happening.

[00:11:00] So how are they trying to solve this? Because it's not a hidden problem. Every AI company knows about this problem. The first way they're doing it is with bigger context. Every time you spin up an agent, every new session, it starts from scratch. It has zero knowledge of your business. But there are patterns of, okay, let's create an agents.md or an architecture.md file, so I'm going to write up all the context so that every single time I turn it on it can read every bit of context about my company, and then it'll make smarter decisions.

[00:11:35] The challenge there is that, first off, you have to write everything properly, which can be very difficult; but second, it starts chewing away at the context window, so it leaves less space for it to do some actual work. And they're trying to improve that by making bigger context windows, and that's causing problems, because they're starting to forget parts of their context.

[00:11:57] They're dabbling with persistent memory, so having a separate system to store memory. I don't know if you're using ChatGPT, it's talking about its memory function. So they're trying to save that, automatically creating this documentation in the background instead of you manually creating it. It's coming up with some improvements, but there's still some space left.

[00:12:19] And then there's fine-tuning, which is where you intentionally change the model with your own data, and you fine-tune the models to be specific to your business and your context. That's effective, but it's incredibly expensive, so it's limited to the bigger companies.

[00:12:37] Just on the prediction, I am going to throw up another CEO who's very fond of making outlandish predictions: that a Tesla will drive from LA to New York by the end of 2017, almost a decade ago. They're almost there. The problem is that last mile is the hardest bit. So we'll see whether the prediction about software engineering will come true.

[00:13:02] So, can AI design good architecture? The answer is, kind of. It still needs people to be able to prompt it in the right way to avoid it making stupid mistakes and decisions.

Maintenance and UI design

[00:13:19] Okay, here's a big one. I'm jumping to the end of the flow now, to maintenance, because a lot of people say, okay, yeah, AI can write code, but writing code is only like 10% of the cost of the code. The other 90% is that you have to maintain that code in life, as your product goes over time. So if this is writing crazy code over here, we're going to pay for it in maintenance over here.

[00:13:42] And honestly, it's not true. They've done some studies on this. AI writes cleaner code than most people. And if you prompt it in the right way, so part of architecture is your organization methodology for how you store files and how you do separation, if you prompt it in the right way it actually improves maintenance, because you can jump into a code base you haven't touched in a while and go, "Remind me exactly how this works", and it will tell you instantly. So these studies are showing that maintenance is not a problem.

[00:14:16] And here's one I guess might be more interesting to a lot of people in the audience today. Our last step is UI design. Excuse me. This is something that came out of a session that I produced. I asked for a form, and I think we can all agree there's some scope for improvements in terms of UI design. So it's definitely the most lagging area.

[00:14:43] But there's a huge amount of interest and investment going in specifically to this problem right now. I'm just going to name a few of these companies out there who are trying to solve this. Google Stitch, Figma Make, there's Claude Design, Lovable. And then Magic Patterns, UX Pilot, Canva, Workflow, Webflow, Replit. There's a lot of people investing a lot of money to try to solve this problem.

[00:15:11] And I have noticed it getting better every month. So is it able to do good designs? No, not right now. But it is very obviously getting better every month. In my scoring, I would say it's an orange for architecture. It's still a red, but it's moving towards orange, for UI design. And we'll see next year where this is going to be.

Build the right product, and build the product right

[00:15:39] To summarize: as I said at the start, it's never been easier to ship something, and your confidence in what you're shipping, because it's high-quality code. That's completely wrong. It's not helping us to decide, is this a good feature that customers actually really care about? Is this going to solve their problems? Because the temptation is going to be, oh well, we can do that in a week, oh well, let's just do it. And we're going to have these Frankenstein products that are just growing and having extra features.

[00:16:10] So there's two dimensions: build the right product, and build the product right. It can build the product right. It's really good at coding, but it's not great at building the right product yet.

[00:16:23] I love Marty Cagan's four dimensions, or four risks, for a product. You've got, is it desirable, do customers really want it? Is it usable, is the UI pattern able to be understood and used? Is it feasible, do we have the technical skills to do this? And is it viable, can we make money out of this? For feasibility, that's where AI has really accelerated, because people who didn't have the skill sets to do certain complex things now can do those complex things. And viability, because the cost is going down, it makes things a lot more viable.

[00:17:01] But desirability and usability: it can help you with some research, generic research, it can help you with things like that, but when you get down to your individual customers in your context, that's where it's lacking right now. So humans are still needed. To answer the question at the start of it, I do think humans are still needed, both in prompting and in understanding the real value.

Roles are changing

[00:17:27] But I do think that roles are changing. I think that how we used to build products, if AI is able to do so many of these tasks that used to take us weeks, then expectations are going to change. And I'm going to talk about this in three areas as well.

[00:17:45] The first is metrics, because AI is going to shift the output. It's going to get increasingly performant at, you tell it a feature and it will run for hours, and when the design and architecture is there it'll be able to output that in a matter of hours, maybe a day or so. So it can produce the output. So the value is going to shift to the outcome. Did it achieve what we wanted it to achieve?

[00:18:12] Tasks are changing. You're no longer going to be doing all of the tasks that you used to do in the past, because AI is able to do those faster, cheaper, and often at a higher quality. Now, as I said, there are areas for improvement, but if I were putting some money on it, I would say it's going to improve in both architecture and design over the coming months. So the responsibilities that you're going to have are going to change.

[00:18:38] And then, shorter iterations. I believe that we're going to go back to prototyping, but prototyping probably in production, more so than prototyping with prototypes. Because I was talking to somebody yesterday and they said, oh, I want a prototype because it's throwaway. If I make a bad decision, if I do something, I throw it away and I move on and I iterate and I do something else. But if the AI can produce the actual production code just as quick as you producing a prototype, then it becomes throwaway as well.

[00:19:10] So if AI shifts outputs, I believe the teams are going to become much more accountable for the outcomes. Did you ask it to produce the right features?

[00:19:22] We used to build teams around the skills needed to deliver. We used to have large teams, eight, 10 people. Going back to waterfall, you'd have your business analysts and your project managers and all of those people. But you might have your scrum masters, your product owners, your front-end developers, back-end developers, interaction designers, all of these different roles, because each one required somebody to be really skilled in a particular area. But AI is taking on a lot of those tasks.

[00:19:48] So my prediction, and one of the problems with teams is we had this balance: the more people you put in a team, the harder it is to move, because there's more coordination required. But we had to just handle that balance because we needed these skill sets to build our products. And I don't believe we do anymore.

[00:20:07] I think what we're going to start seeing is a merging of expectations. The design engineer: if you look up job openings for a design engineer, that is probably one of the most booming job positions. It's somebody who can design but also code. So the expectation is you design it, but I want to see it in HTML and CSS on my screen versus just purely in a mock-up separately. And the same is happening with devs as well. They're being merged as well. People are expected to take on more.

[00:20:40] So what I see is that we're going to have smaller teams. Our two options are the same size teams but taking on more scope, because you have more bandwidth to do more work, or smaller teams with the same scope that you used to have. And I believe we're going to end up with smaller teams, because of the overhead of coordination when you have more people on a team.

[00:21:05] And then teams will have to move faster. Take away all of the miscommunication in the team, take away all the handovers, and you're able to move a lot quicker. So that's why I believe it'll move faster. We're back to probe, sense and respond, where you try something, release it, see whether it works, and come back. So I talked to somebody[?] earlier.

Are we going to see job losses?

[00:21:34] Are we going to see job losses? I just realized I spent a bit too much time on the other stuff, so we're getting to the fun question. If teams do get smaller, if you need fewer people to do work that happened in the past, does that mean you need fewer people?

[00:21:49] And this is where we come back to economics, and there's a thing called Jevons paradox: when you make things cheaper, people use more of it. So I believe that we're going to have a lot more people working in software product development over the coming years, because companies that could not afford to do it are now going to be able to build things that they have put off, because it just wasn't possible before.

[00:22:12] And why I believe that's going to happen: software is exploding. GitHub, if you've heard about GitHub, it's where developers store code, it has gone down a lot in the past year because their load is exploding, as software is up 14-fold this year alone.

[00:22:32] And AI can do tasks, but it can't do a job yet. I think there was a study in, where is it, where did I list this study? Okay, Wharton business school. They did a study that estimated how many tasks of each job AI can perform, and it's in the 60% for a lot of product development jobs, but that's still 40%. And that last bit, that's the Tesla driving to New York. That bit is hard to automate, and it's going to take a very long time, if ever, to be able to get rid of that last bit. So we still need people to be directing the AIs to build these products. Full automation has a very long tail.

[00:23:16] And here's a graph about job listings. Yes, job listings have been going down for the last few years, but there's been a change. Software development is going up. The rest of the economy is still going down, but software development has started to buck that trend, because we're starting to see the change now where people are saying, yes, I can start doing things that I used to not be able to afford to do.

Takeaways

[00:23:44] So, to summarize. AI can build products, but not necessarily the right products. That's where you need the humans to be asking the questions: should we have built this? And as AI takes on more and more tasks, and it's going to keep chipping away at more and more of those tasks for our roles, you will be expected to do more. I think that's going to be the shift of merging multiple job roles currently into a single job role.

[00:24:16] You will become accountable for outcomes. Today, a lot of times it's about how much output did you create, how much code or screens or whatever. It's all about the output. But in the future, output's going to be so cheap that it's the outcomes that are really going to matter. And I believe it's going to be smaller teams doing both discovery and delivery together, prototyping in production with code, with real customers, and then iterating at higher speeds to get products that actually work for customers.

[00:24:50] So that's my talk. I hope you enjoyed it. The other thing I'd just mention, if you do want to know more about how smaller teams work better, I'll be publishing a book this summer. You can get an advance copy there, but otherwise, thank you very much.

Q&A

[00:25:12] Host: Thank you, Rory. That was a great talk. We have time here for some questions, so let's just jump right into all of your questions here. You've talked about UI, but what about UX?

[00:25:23] Rory: Yeah, I did a little bit merge those two when I was saying that it can do good UI. I think actually, with the wealth of knowledge that the tools have now, if you prompt it well and say I want some better UX patterns, I want to make this flow smoother, I want these kinds of things, it can actually produce pretty good breakdowns[?]. I've tried it myself and it's given me really good options for how I can improve user journeys and things like that. But again, you have to bring that knowledge to ask the question, because it's not going to answer it. If you just asked it to build a flow, it's going to build a terrible flow. So you need to know to ask it, with your context, what's the right way to build it.

[00:26:05] Host: Great. And the secret sauce is no longer the code, but UI is converging to the same bland look for everyone. Are we approaching a bleak void of design?

[00:26:18] Rory: It's navel-gazing, so I'm going to put my opinion: yes, I think we are. But I think a lot of design is about usability, and a lot of usability is about familiarity. So I think for a lot of sites, bland doesn't necessarily mean bad. From a business perspective, it's more user-friendly, because customers are aware of the patterns, they know how to use it. But then, I think what it does leave is that it's going to leave space for individuality, and some companies are going to be able to make a mark as being more unique.

[00:26:57] Host: I guess one thing I would think about there: as things start to move faster and roles start to blend, what should product leaders protect within their teams?

[00:27:07] Rory: Yeah, it changes every month. I do think right now there's a lot of talk about AI layoffs. I think a lot of that is AI masking. They're going to do the layoffs anyway, and they're just saying it's AI, versus it actually being caused by AI, because as I said, AI can't replace a job yet. It can replace tasks, but it can't replace a job. So I think as a leader you need to be saying, look, okay, you can cut my team, but we're not going to be able to deliver, because AI is not there to replace people just yet.

[00:27:41] Host: Great. And is AI getting cheaper, or is this cost simply offset by subsidized investment? The most advanced models cost two, three, four times the token cost of what the models that can do much less cost. So you're also seeing token inflation as well. It's a great question.

[00:27:58] Rory: It is a great question. I think the answer is China is actually doing a great job at helping keep costs under control, because they're releasing open-source models that are about eight months behind, maybe nine months behind where the US models are, and that is keeping cost controls on, because if costs start inflating too much then people will start shifting.

[00:28:23] From a coding perspective, you get what you pay for with these models. The leading-edge ones are much better, but I think the competition is going to help manage it. Who knows when the bubble bursts and people actually have to earn a profit, what will happen. But I think that there are constantly advances in technology, and if there's one thing with technology, costs drive towards zero over time.

[00:28:45] Host: And one big blocker for speed is our alignment meetings, stakeholder sign-off, etc. What's your take on improving alignment processes, and how can AI support that?

[00:28:56] Rory: Okay, I need about three weeks and I'll explain this exactly.

[00:29:00] Host: You have a minute and a half.

[00:29:02] Rory: That question is literally why I wrote the book that I've written. It's all about the alignment, it's not about the technology. The technology is an important part of it, but it's more about the alignment of how you get people to agree what to build, how to change teams. Do you empower teams? Oh, that's scary from a management perspective. How can you do that without risking teams going crazy? So yeah, I can't remember the exact wording, but it's a very complicated problem. It involves trust with boundaries. You can't just trust teams, you have to ensure that you have the right boundaries in place.

[00:29:36] Host: And where should a designer start to learn about code, to evolve into a design engineer quickly? And do all designers have to think about becoming design engineers at this point?

[00:29:46] Rory: My personal opinion? I would be nervous not being one. That's just my personal opinion, because I think you're going to be competing against people who can. So it's just my personal opinion that you should learn how to. And when I say code, I mean CSS. I don't mean that you can write the back end, it's the front end. There was a great, I had a slide and I took it out, there was a coder or designer and he said, designers used to code and used to write in CSS, and React came along and everything became a single-page application, it became too complex. Now AI is going back to basics. It's making these simplistic sites that aren't overly engineered, and designers can just get back in there at the CSS and making the styling. So I think it's back to coding, but coding from a design perspective, not the JavaScript, not the interactivity. And the AIs are going to be doing most of that, so you're prompting it. You just need to know the right knowledge to prompt it in the right way.

[00:30:48] Host: Do you have time for maybe 20 seconds?

[00:30:50] Rory: Oh, 20 seconds. Okay.

[00:30:51] Host: Fewer handoffs between different roles on a team. Can you quickly elaborate?

[00:30:55] Rory: In 15 seconds: engineers do pairing. Two people sit at the same computer with one keyboard and they work on a problem together. Some companies are experimenting with design and engineer pairing. You work on the designs together, you work on the code together. So the engineers know the design problems, the designers know the code problems, and I think that's a great pattern to take away.

[00:31:20] Host: Is there any chance at this point that AI really is a bubble, meaning not just economically, but that it simply can't ever reach the potential promised?

[00:31:30] Rory: I'll answer that in two ways. Financially, it is a bubble. Everybody has kind of said that. There's too much money going in. Startups are being valued at hundreds of billions with barely a product. So that's the textbook definition of a bubble. I don't know when it's going to pop, but that's the textbook definition.

[00:31:49] Will it ever reach the potential? There's two trains of thought here, and it comes back to what I talked about with the learning approach. AIs are really, really, really inefficient at learning, and that's why the training runs cost so much money. So you have some experts in the field. Yann LeCun is probably the most famous, and he believes that LLMs are a dead end, and he thinks that they will never reach the potential that people think they will, and he is investigating a completely new paradigm for how to solve that, because he thinks they're too expensive and the learning problem is never going to be solved.

[00:32:24] Other people believe that you just give it bigger context windows and it can relearn all of your context on every run, and then that will solve that problem. We have a million context tokens on most things now. It'll probably be 10 million next year. So maybe that solves the problem. But I'm a little swayed by Yann's argument that LLMs, because of the learning problem, aren't the best way. So I think over time, because his business is also valued in the billions and he has so much research money, I think the cat's out of the bag. It will keep hitting the milestones it's expected to, and we'll probably see a shift of technology over time.