Building the Next Generation of Software

28 May09:50 – 10:25Stage: Main StageTalk
Slides

Checking session availability…

Hang tight while we load the latest updates.

To survive and thrive in an AI-first world, Intercom made the deliberate decision to abandon every aspect of how it built software, something the company is well known for. When organisations talk about implementing AI, there’s a temptation to look for the minimum necessary adaptations or to focus on the changes that are the most fun, but that is not even table stakes.

Building real AI products demands rewriting the entire approach to software at a fundamental level. At Intercom, that meant rethinking everything including its processes, speed and roles. For example, software development operated less like a factory and more like a lab, the pace moved faster with timelines and even team sizes cut in half, and roles transformed across the company as designers and product managers became AI coders and builders.

In this talk, Intercom’s Chief Strategy Officer, Des Traynor, details the necessary messy chaos of rebuilding the plane while it’s flying. He’ll explore how this radical shift ultimately led to the creation of Fin, Intercom’s AI Agent for customer service, and what it taught Intercom about building products that can actually win in the AI era.

Building the Next Generation of Software

Des Traynor at UXDX EMEA. Video: https://youtu.be/eAmph-w1GhU

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

How Intercom became Fin

[00:00:09] Good morning. Thank you all for being here. I'm going to talk about the next generation of software that we're building. A thing we say internally a lot is that AI will change almost everything in some way, and it'll change some things in every way. And we really believe software is definitely in the latter category. It's all changing.

[00:00:30] I'll give you some quick background about us and how it's changed us. August 2022 was not a fun time in Intercom, or the company you might have heard of known as Intercom. This is roughly where our revenue growth was. We had been declining for quite a long time and we were actually underperforming the average company on the market. We were substantially underperforming. We had just changed CEO and we were like, something has to change pretty quick or this company will cease to exist. That was our choice.

[00:01:00] ChatGPT had just launched. We all remember where we were on November 29th, 2022. And it was just insult to injury for us in a sense, because it was like yet another thing we have to deal with, along with all the other problems we were going through. The problem we faced was that we sold a human support product for humans to sit in and do customer service, and then we all started playing with this ChatGPT thing and realized, this is going to do customer service. And our choice was, build a way to make this thing do customer service, or die.

[00:01:36] So we decided to not die, or at least fight as hard as we could, and we built and launched the product. Things started to turn slowly when we launched Fin. We started to see some sign of life, some sign of people being like, well, this thing is at least useful. And we invested harder and harder and we went kind of ultimately all in. And today we are substantially outperforming the average company on the market. Our revenue is 400 million. Fin itself is a $100 million business. And yeah, it's kind of worked out in a lot of ways, but it was not at all easy, nor was it straightforward, nor was it an obvious choice. And most people have looked at the Intercom story and assumed that one of those three things is true.

[00:02:21] So the only thing that changed in our business was literally everything. We changed our strategy, our company mission, our culture, our values, how we build, what we build, how we ship, how fast we ship, how we price, how we package, what makes our product different. We changed how we sell, who we sell to, how we pay our sales reps, what our brand was, and then right up to and including two weeks ago, as he just said, we changed the name of the company. Intercom is now the name of a help desk product, and Fin is the name of a business. That was all the stuff we had to do. And it's taken us about two and a half years to do that, but it has ultimately worked out.

[00:02:54] It was not easy, it was not straightforward. And if you're trying to apply this to your own case or your own company, I would just say, bear in mind, some of your customers will hate this. Some of your team will hate this. Some of your followers or fans might hate this. Your shareholders will hate this. You'll get a lot of tough feedback, you'll get a lot of "what about," a lot of second-guessing. You'll have no evidence on this. We were going all in on a product that was like 1% of our revenue, and everyone was like, you look like an idiot doing that. And we had to do it.

[00:03:26] But you must persist through all of this. And whenever I explain this to any startup who's going through a similar struggle, they basically always say, "Hmm, is there an easier way?" And that's a very natural reaction. People try to minimize the amount of change they have to get through. The key idea here is, you have to go too far to know you've gone far enough. Most people adapting to AI look for the minimum set of changes that are kind of fun and cool to make, and then they tweet about them and they say, "Job done." And they haven't really done anything. They've just kind of taken some AI kudos, but they will still die.

[00:03:59] So this talk is about what it changed for us, but I'm writing it from a point of view of what will hopefully be useful to you if you find yourself in a similar state. Specifically if you're in a SaaS business, as you know, the world, the stock market, investors, they do not like SaaS businesses right now. They like AI businesses, because they believe that AI will eat SaaS. And I think that's largely proving to be true at the moment. So I'll talk to you about what it looks like to go on an AI transition.

AI changes what you build

[00:04:24] Firstly, AI changes what you build. So we talk a lot internally about the inconvenient truths. Which is this idea of things that are true and no one wants to say it out loud. An example of an inconvenient truth we recently shared was, one of our engineers basically had Claude entirely clone one of our competitors' products in the space of about four days. And we're like, huh? Because obviously that is just as applicable to us. And we're like, I wish that wasn't true, but now it is true, and now we have to deal with that.

[00:04:57] There are lots of inconvenient truths that we all have to face, and I think silence, burying your head in the sand, pretending that the world isn't happening outside, are not strategies. Hope is not a strategy. Survival is a strategy. Your product will become irrelevant fast. New competitors will blindside you, and you basically have to change everything to be fit to fight and survive. This isn't an option. You don't get to decide whether or not you want to do this. It's going to come for you whether you want it or not.

[00:05:27] AI will change problems and product domains entirely. There are some products that I don't even know what the future of them looks like, because it's built on so many predicates that are no longer true. And I'm not going to pick on any specific named product here, because I don't like doing that, because people do that to me all the time. But I'll show you, here's a mocked up product of a project and program management tool. The idea of this tool is everyone should stay in touch with everyone else: status updates, routine inquiries, progress tracking and all that sort of stuff.

[00:06:01] And my question is simply, when AI connects everything to everything else, and can ultimately read GitHub, and it can follow Slack channels, and it can update itself with Google Docs, what's the purpose of yet another tool to just aggregate information? I'm not saying it doesn't exist, but I'm saying it's unlikely the same purpose it had in 2021. It's quite likely the entire problem set has changed.

[00:06:23] And AI changes the shape of a solution too. Products will merge and converge in an AI world. In the old world we used to see this kind of, you'd have lots of different screens and lots of different UI. And on the left you'd see, hey, this is a big complicated SaaS app, and because we have to have all these features and screens and dashboards, we could say that there were obvious bounding boxes of what this product could do. How many tabs did you have? Six. You probably had six core areas.

[00:06:54] On the right is what AI products typically look like today. It's not obvious what you can't ask it to do. If you log into your HR tool and you say, "book me a holiday," it's like, that's a stupid thing to ask an HR tool to do. Okay, take the time off. Tell my colleagues that I'm taking the time off. Update my calendar. It's not obvious why it can do some of those things and not all of those things.

[00:07:20] And what you're going to see is these tools will expand to do all of the things that they can possibly have a right to do. They will integrate into other technologies. And as a result, every single tool in your kit will grow to kind of consume its peripheral software. We're seeing this already with companies like Notion. They're just growing and growing in terms of their strategic scope. And it becomes very hard therefore to work out, where do you draw the line? Where does the scope of your product stop?

[00:07:52] So if you take a typical flow of, hey, a meeting tracker and notetaker and maybe project management, we think that it's cool that these are all different tools, and that's great. So you might have six or eight different solutions. The problem is all these guys are expanding. They're all trying to take over each other. Granola is now taking off, going from meeting notes to account updates. Gong is going from account updates all the way back into meeting notes and beyond, and hydrating Salesforce and all that sort of stuff as well. So it's really hard to work out where your product starts or stops.

[00:08:24] So where does a product stop? Well, the only obvious places to me are when there's a different buyer, or if you're bumping into a really solid, hard to displace tool. If people are just really happy with the other product and you don't want to compete and play that game. But it's not obvious where we draw the line on the shape of our solutions anymore. And that has real implications for the products that we build.

AI changes UI

[00:08:48] AI also changes UI. So we all remember the command line. It was extremely fast and extremely hard to learn, and typically was known only by the arcane Linux system administrators. And meanwhile, GUI was this beautiful thing where we all had our trade. We all got really good at designing drag and drop buttons, UI toolkits, shadows, shades, UX, usability, all that stuff, wireframes, etc.

[00:09:13] We now have something that's in between these two. Something that lets you have all the speed and efficiency of the command line and all the learnability and ease of use of the GUI, where you just say the thing you want. And this is going to change how software works for us. In the old world, if you said, "Hey, I'm using a spreadsheet and I want to make all the negative cells red," well, the instruction set I would have given you was the following: this is how you make the negative cells red. And people like us spent a lot of time being like, is that easy enough? Is that intuitive? Can we make the buttons more obvious? Is there a better word than conditional formatting?

[00:09:51] And none of that matters anymore. You just say, "Make the negative cells red," and the negative cells turn red. That's it, game over. And what that does is, this isn't just about faster UI, it's about who needs to learn your UI. In the old world there were loads of people who wanted to use spreadsheets really powerfully, and then we all had to have one or two friends who were actually good at spreadsheets. In the new world we're all genius spreadsheet operatives, because we just say the thing we want and we get it.

[00:10:26] So when you look at all of this type of UI that we've spent 15 years working on, I don't know what role makes sense anymore, because it feels like for a long time we're going to be working on this. This is possibly the future kind of primary product UI. And we can lie to ourselves if you want, and we could say, no, no, people are always going to love these artisanal handcrafted buttons and shaders and dropdowns. But we know that that's bollocks. We know that ultimately there's a reason why ChatGPT is winning, there's a reason why all these new AI agents have so many hundreds of millions or billions of active users. And it's not just about faster, it's also a better strategic thought partner.

[00:11:07] One of the things Fin does, our product Fin, is it tells you, hey, here's the bits of your help center that are out of date, and because of that you need to update it, because otherwise I will give out-of-date answers. And we used to build great, lovely, easy ways to surface topics that need to be updated. But with our most recent release, Operator, we were like, well, let's see, can the AI just do all of this end to end?

[00:11:33] And here's Operator taking a very simple thing, where you drag and drop in a file and say, hey, here's the new product we just shipped, can you update the help center please? And it says, yes, no problem. And you just go and approve all the updates. Like, think of all the UI we didn't build. It's just, here's the stuff you have to do, go do it, and it's like, yes boss, can you approve these changes? And we say yes or no. So many steps skipped, so much UI irrelevant.

[00:11:59] Similarly, a slightly different one is, here is when we updated a policy. So we said, hey, in this case, I remember this example. Oh yeah, we're basically saying, can you analyze all of our current reports from the last two weeks and tell me if there's anything going wrong? And it'll do all this and it'll render bespoke UI ad hoc for you. These aren't hard-coded screens, it's just like, oh, I'm showing something over time, I better use some charts or line charts or whatever, and it makes those decisions.

[00:12:26] Designing in the AI era is just a different beast. When the problem occurs, you have to ask yourself, what's the least amount of user input we need? What's the most we can do for them? How deeply can we be integrated to all the peripheral tools? How reliably can we nail this solution? And how automated can we make this whole thing? And they're the sort of questions you have to ask repeatedly.

[00:12:50] And the way people get UI wrong in the era of AI is, they think, oh well, if we start off with, again, a fake project management tool, they're like, oh well, you know what, we need a co-pilot. So they build a co-pilot. It's not about co-pilots. Oh well, we need one of those big text areas. And it might not even be about one of those big text areas. It might be about, if the problem in this case is "file a bug when everyone has reported," maybe we just create the rules that say file a bug when everyone's reported. Or maybe you don't even have UI for that. Maybe you just say, hey, here's our bug policy, you work it out. And maybe it just works it out. And in all of these cases, we're bypassing so much UI and so much human grunt work to actually just get the outcome that we want.

[00:13:38] So it's a different world. It's truly a different world. The pattern I would say is, when a problem occurs, generally speaking our approach has been, all right, what are the inputs, what are the outputs, what's the right amount of UI components to solve it. That's no longer it. It's now a case of, when a problem occurs, how do we handle all problems that look like that, and how do we automate that? And because of that, I think the future products that we build will be maximally agentic. They'll be self-improving. They'll let you interact whatever way you want: CLIs, Claude, whatever. You'll be able to interact with this software any way that makes sense. And the biggest thing that matters is actually the depth and quality of your AI. That's the differentiator here.

AI changes how you build

[00:14:27] So, AI changes what you build. AI changes how you build. You have to understand the problem you're solving, and that means decomposition. This, by the way, is where UX people are still essential, in my opinion. You have to understand the job to be done and be able to break it down into all the necessary steps. So if it's meeting or project management or expense management or anything like that, you have to be able to break it down and understand, what are the decision points, what are the inputs, what are the outputs, what could be automated, what can't be automated, where do we need a human in the loop, where do we not need a human in the loop?

[00:14:54] And then lastly, you have to ask yourself, can we reliably do this really, really well? The panel earlier spoke about the reliability and the trust building sort of aspects of AI here. You have to look at the chain of events and be like, how reliable is AI at every single step? So if you take one of these tasks and you can assign them, in all cases here we're like 99% reliable, 95, 98% reliable. That sounds really damn reliable. The problem is you have to multiply them all together, because that's how you do an end-to-end workflow. Multiply them all together and you're down to 88% reliable. Now it's no longer good as an expense tracker. No one's going to use a tool that makes a mistake with 1 in 10 of expenses. It's now a human in the loop solution, not an AI solution, and that's just different.

[00:15:34] So do not underestimate the complexity that you need here. People do this all the damn time. You have to be really good at AI. You have to decompose really well. You have to have really high reliability. And just beware this marketing overhang. Everyone's had this pattern of, they have a brilliant idea after two hours of playing with Claude, they think they're nearly there, and the progress is going so fast. And what happens is progress stalls. The new idea is far away and it can't ship. And if you don't believe me, have a look at Apple Intelligence. We heard about this product two years ago. Apparently, I've just read this morning, it's coming in two weeks' time. That's two years and two weeks late.

[00:16:14] So how you build software is different. We're building software through empirical evaluation, and we're building primarily AI software. It is not a game where design... design has always been about minimizing the risk, but the risk is not in the design now. The risk is in the AI, in my opinion. And the biggest change we're going to see here is that we have to care a lot, as UX people, about what's going on underneath the surface. In the old world, if you're building software, everything that mattered was visible. You could see all the buttons, the screens, the flows, the user journeys, whatever. In the new world the UI, as we've said, is reduced, and actually what really matters is the AI, and that's where you have to focus your attention.

New muscles: abandoning the old product principles

[00:16:57] And what that means is your previous process of building software is almost certainly out of date. And if you think it's not, you're just lying to yourself. You have to solve for AI reliability, you have to solve for design exploration, and you have to grow new muscles within your product work.

[00:17:13] So to quickly cover some of that. Intercom was famous for all of these, our old product design principles, that said this is exactly how we should build software. And we were so damn proud of them, and you can still find them online and all that, but ultimately we had to abandon all of this. And yeah, it was painful, and people didn't like it, and it was disempowering. But this is what the old world looked like: pick a job, have the designers work on it, pick a solution, ship it, test it, iterate, etc. That was the old world. We could do that because we knew what we were doing.

[00:17:41] In the new world, we have to start with AI. We say, what does AI make possible in our solution space? And given that, what must we build? And given that, can we do it reliably? And if we can't, how do we reduce the scope to something that will work reliably? And that's an entire change. But, just for folks taking photos, I will share the slides after, if you want. But yeah, there's a lot that changes in how you build software because of this new style, where reliability matters more than anything else. It doesn't matter how good your UI is if it doesn't work all the time.

[00:18:18] New muscles you have to grow: you all need to be fluent in everything from scientific rigor to prompt analysis to prompt engineering and causality, and understanding what changes, what breaks, what goes where.

[00:18:32] From a point of view of what changes in design, I think the biggest thing we're going to have to see is we're going to have to get really good at exploring the solution space. AI is not a solo designer. It needs our input, but it is capable and useful. You've probably seen some of this work from Figma, where you can literally say, "Hey, give me a few variations of this." It's actually worth doing. It will actually expand the solution space.

[00:18:56] You have to bear in mind, for the longest time in design it was an expensive task, so as a result you would tend to pick a solution pretty early. The old world was, you'd pick one and you'd iterate and refine, and you try and get the best version of that idea. But when exploration is a lot cheaper, you can actually explore a solution space a lot better. And this is where AI can be great. It's not the best final designer, but it's a great tool for exploration, and that's an essential skill in this era. You can try five different things. You can prototype them, vibe code them, show them to users, do lightweight user testing. Remember, we used to prick around with paper prototypes and all that card sorting. We don't need to do any of that anymore. We can actually vibe code things that are quite close to production software and see real users interact with them and get that type of feedback. It's very rewarding. So I do think that changes.

AI changes the momentum of your team

[00:19:50] And then lastly, AI necessarily changes the momentum of your team. We're spending like tens of millions on tokens at this stage. It better be doing something, right? So you have to actually see some tangible outputs here. There'll be changes within product design and engineering. These are the types of changes we've gone from. Remember, our industry used to spend probably the best part of 10 years discussing, should designers code? Designers should code now, because we can all code. Now, whether or not you should ship to production is a decision for your company, I'll get to what we do. You should also be expecting significant changes to engineering velocity. They should be significant, otherwise you're just pissing away money on tokens for no reason.

[00:20:29] Some changes we've made. Every designer in Fin ships to production. To do this, we set a date. We said, by August 1st of 2025 every designer must have shipped one change. It can be a typo, it doesn't really matter, but we need you to ship to production because we need to get that skill going. Because if we don't, we're going to have a load of designers who are clinging to Figma while the rest of the world is moving on.

[00:20:54] Another thing we did was this three step framework, where we said, right, step one was, designers are not allowed to file tickets. They must actually ship the fix themselves, and that has been really useful. Designers vibe code the roadmap, so that that's what our sales teams bring to our customers. They bring vibe-coded prototypes of different things they've been working on. And lastly, designers now own the entire front end. And again, a combination of a good pattern library, which we already had, and Claude Code has made this extremely productive. Extremely, extremely productive.

[00:21:28] And then lastly, on the engineering side, we have gone extremely hard on making sure that we're getting maximal output from all of this token spend. So what we've been looking at has been, we initially set a goal to double productivity by June, by like two weeks' time. We have actually tripled it in that time. That's not us being great, it's also that Claude Code has gotten better along the way too.

[00:21:52] But the way to think about this is something like the following. You can dig with a trowel or a shovel. You can have a combine harvester. You can get great at running a fleet of them. Or you can run a swarm of swarms. In the top right is what we call an agent swarm, which is when you deploy a set of agents, each of which will deploy more agents, all to tackle different decomposed problems that you fed it. This is what you should be thinking about when you think about how are you building software. The up-and-coming hot cool startups, they're way in the top right here, and that's ultimately what your competition will look like. Maybe not today, but next year and the year after.

[00:22:27] The fact that you have a million lines of code written from the last 10 years, irrelevant. These things will write a million lines of code very, very, very fast. "Oh well, ours is much more battle hardened." Yes, and that will buy you one, two, three years of a stay of execution, but you have to move with the times. A lot of people, they play around with hackathons, lunchtime learnings and all that. They're just doing this. It's not the same thing. They need to be doing this.

The numbers

[00:22:56] Now, here's just some artifacts. These are real numbers, real metrics, real data that we've been sharing, that I'll walk you through very quickly. We have successfully tripled productivity. We are basically building faster. We are... sorry, I need a step to actually see this. We've halved our product defect backlog. We're shipping more product changes. Product changes go live to production faster. Our code quality has actually improved in this period. We have done it with less downtime.

[00:23:26] We are spending a lot of money on tokens, a lot of money. But the relevant piece here is the totality of salaries plus token cost per feature shipped, if you like. Is that going up or down? If it's going down, it's a totally acceptable spend. We're spending more, but we're getting a lot more relative to what we're spending, so it's worth it.

[00:23:46] We, you know, 94, probably 94 and a half percent at this point, of our entire ships to production are handled entirely by Claude Code. We've added a lot of skills to make this happen. We've given Claude a lot of access, a lot of ways to look things up, ways to run simulations, ways to run tests before it ships.

[00:24:06] We are pushing people for performance. We want to make everyone aware of where the new standard is. If you're just as productive as you were in 2022, and now you're spending an extra $10,000 a month on tokens, what's in it? Like, what's happening? Probably nothing. Productivity should be increasing if agents can do this that quickly.

[00:24:29] We're replacing the tools that get in the way. And ultimately, this is kind of the journey we've been going on. We've been basically swapping out what was a pretty messy process to get real software live to production, with something that's a lot smoother and a lot more accessible to all of our R&D people. So designers can be productive, product managers can be productive, engineers, technical program managers, everyone can make meaningful impacts to our product momentum now, which has been really good.

[00:24:53] But speed is the actual essence here. And people don't like to hear that, because driving fast isn't comfortable, running isn't comfortable, moving really fast isn't comfortable. But this is AI. The reality is, fast gets good before good gets fast. And that's just the nature of iteration: the more you do something, the faster you do it, the better you get. The era of, we're going to spend two years in a waterfall process to come up with a beautiful thing, that made sense if you're building a desktop app in 2005. It probably didn't make sense in the web era. It definitely doesn't make sense in the AI era.

There are no moats

[00:25:26] So AI changes what you build, how you build, and it changes the momentum of your team. The last thing I'd say is, there are really no product moats anymore. If you're out there thinking, don't worry, we've got a monopoly on this industry, or don't worry, we've got five years of a head start, there are no actual moats. Everything that you think is a killer feature, stuff that we've spent so long perfecting the JavaScript, the CSS of, to make it so simple, so sexy, so cool, it's all Claude-clonable. It's all "see this feature, build it for me," done.

[00:25:58] Every single differentiator we used to have... these are uncomfortable truths. I'm not here to tell you everything's fine. But everything that made you different or special in the old world is probably one prompt away from everyone having it. And we see this happening. All our competitors have all of our most beautiful reports now. They didn't have it for the longest time, but they all have them overnight now. There are no moats. Speed is the only thing that matters, and I encourage you to act accordingly.

[00:26:21] I'm Des from Fin. If you want to find more material like this, it's on our ideas blog. If you want AI research, it's on our AI research blog. And if you want these slides, you can just mail me. Hope this has been useful. Thanks so much.

Q&A

[00:26:39] Host: Thank you very much, Des. I think that kind of followed on from our panel of who should be most nervous, and I think everyone. So we've got some questions already coming in. Great talk. How do you push through this change? I guess maybe in your perspective, because you had the senior leadership bought in, it might be easier, but the people in the room here, how can they do it? Was it just a top down, you were able to do it, or something different?

[00:27:08] Des: I'd say, companies only change through inspiration or desperation. We had both. We were desperate, as I said at the start, and then we also saw the opportunity. You need both. Don't wait for the desperation. Unfortunately we didn't get that choice, but I would encourage you not to wait until it's too late and then scramble. So try to hopefully get there with inspiration.

[00:27:29] Des: The other thing is, and this is where we're into the slightly uncomfortable aspect of this, you have to motivate your people to change their habits, to change their ways of working. We did some useful things. We said, "Hey, for one whole week, no one is allowed to open an IDE. You have to just use Claude." We're just like, no one's allowed to do it. And that was like, oh my god, what the hell are you doing? We're like, it's okay, if you're unproductive, that's okay, but for one week no one's doing that. Those were the little sort of things that pushed people over the edge and got the "oh, this thing really works."

[00:27:56] Des: But there is an aspect of, if you can't change your people, you have to change your people. We gave everyone a year. We set the 2x thing in June 2025. We're two weeks away from hitting it and we've already more than hit it, but I think we gave a healthy time frame to do it. I don't know if, if we started today, would we give them a year, because I think we're late in the game now. But yeah, there's a few aspects to it.

[00:28:15] Host: And were there some people, because it is deeply uncomfortable, were there some people who decided it wasn't for them?

[00:28:23] Des: I actually don't know the figures, but I would posit somewhere between five and 10% of people opted out. Either designers who didn't want to code, engineers who didn't want to use AI, people who didn't want to have productivity measured, which is the next question. Some people just opted out. That's totally fine. They're great people, they'll find a great company for them. But it was not where we were going.

[00:28:41] Host: Okay, brilliant. Okay, we've got 17 upvotes for this one. How do you measure productivity? Is it lines of code, or what's your metric?

[00:28:51] Des: So the deeper answer to this is on that ideas blog, ideas.fin.ai. But the short answer is, it is meaningful ships to production of peer approved code to represent a substantial or meaningful feature. It has to do something, and it has to be something that actually executed. So that's the actual unit of productivity. In this case, we did some calibrating to make sure that it wasn't just a lot smaller, like people shipping 100 small features instead of one big one. It's not that. You can actually see the size of the pull requests didn't change meaningfully in the period. But yeah, so it's basically how much stuff went live per capita, if you like, right across all of R&D.

[00:29:30] Host: And you said there, it's being used. So are you tracking customers actually...

[00:29:34] Des: You can't just ship shadow features that aren't part of the primary UI. The one thing I would say is, measurement is a cultural problem. It's not really a product problem. There's no measure I could give you, there's no answer I could give you to that question that isn't in some way hackable. The reality is, for it to be hacked, let's say we both work together, you'd have to approve my thing, and possibly your boss or someone else would have to approve it too. It would have to have gone live. Other engineers would have seen it. Someone's going to be like, yo, this thing doesn't do anything. So we lean a bit on that.

[00:30:09] Des: And so to give you the counter example people might make: oh well, what about somebody spent all week helping junior engineers? I'm like, that's grand. If someone comes and is like, hey, I noticed you shipped nothing this week. Yeah, I was helping junior engineers. Cool, on with it. Like, we're not idiots. But I think we've spent so long getting this kind of allergy to understanding how much we're producing, and I think it's useful for our industry to sort of stress test that a little bit. And again, it's not like if our designer turns up and goes, "Yeah, I mean, I have 10 designs here, I can't get my head straight, I haven't worked it out yet, give me another week," we're like, cool. We're not idiots. But it's okay to actually talk about productivity.

[00:30:50] Host: Brilliant. Okay, next one is, how do you make sure customer is still the focus, and is there a risk that you trade off research findings for speed and efficiency if your starting point is what can AI do?

[00:31:04] Des: This is a really deep question and I like it. The reason we have to start off with what can AI really do is because, if you go the other direction, there's a classic, you never start with the technology, you always start with the customer, work backwards. That's cool. But your customers will say things that might not be possible. Later on, when all this tech is a lot more stabilized, I think you can return to that style. But right now the capability of AI is still evolving, so we don't know what is possible. So we have to start with, hey, what can actually work here? Because customers want, rightfully, the sun, moon, the stars, every possible thing.

[00:31:39] Des: Our challenge is to actually work out, what can we give them that will actually help them? We sell to customer support leaders. What we need to do is give them things that will definitely help them do support. So they can say, "Uh, it'd be great if you could handle all my super advanced complicated support requests." I'm like, I'm sure it would be, we can't. Well, here's something we can do, we can handle tier one.

[00:32:00] Des: So you really have to start with understanding the portfolio of capabilities that our AI group have made. To give you an example, something we're working on right now: can we ingest source code and produce documentation, and can Fin use that documentation to build knowledge of how the product works? We don't know if we can do that reliably yet. We do not know. If we can, it's a big advantage. No customer has asked us for that, give or take, pretty much, very small, but it's a huge boost for them.

[00:32:27] Des: And I worry that the two risks with only going customer first is, you'll either end up chasing illusions of things that will never really be possible reliably, or you'll be unambitious in your application of AI. So you really have to... I think more than any sort of time... and by the way, I'm a UX guy by heart, I was one of the kings of jobs to be done and all that sort of stuff. So it's not in my nature to say this, but you should start with understanding what the technology makes possible.

[00:32:53] Host: Okay, brilliant. We're out of time. So if everyone can give a big round of applause to Des.

[00:32:58] Des: Thanks very much.

[00:33:00] Host: Thank you very much.

Speaker

Des Traynor

Des Traynor

Co-Founder

Fin