From Board AI Pressure to Shipped Product Value

Cristina Fuser at UXDX USA. Video: https://youtu.be/w5A3jBvhXOM

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

The question AI never asks

[00:00:16] Cristina: I hope this can be as useful to you as possible. I'll talk through what I've learned in the last couple of years working at BuzzFeed and working with AI, and then we'll have a Q&A at the end. The headline of this talk was the board pressuring teams, from executive teams and CEOs down to the interns, to integrate AI into their strategy. The question of how you are doing that, what's the roadmap, how are we going to be at the forefront of this technology, is a question that everyone at this point has been hearing too much. And it's an important question, because every company needs a narrative around AI at this point. You can't avoid it. If you don't have a good, convincing answer to that question, something feels off.

[00:01:09] Narratives are important because that's what pushes companies' stock prices up. It helps you hire great talent. It gives confidence that your business is at the forefront of this huge technological shift. What I've learned is that it's actually not that difficult to find ways to use AI and integrate AI. It's very easy. You just look at your business, look at some issues, and think, could I use AI for this? Could I use AI for that? So that's not really the hard part. Also, AI will build anything that you ask it to. The only downside is that it will also help you build the wrong thing with such enthusiasm that it would be almost rude not to.

[00:02:01] We know it's very encouraging. It's relentless, tireless. It will always support your ideas. But the one question that, so far at least, I've never been asked by my many AIs is: why do you want to build this product? Since I've worked in product, to me that's the only question that matters. Why should I invest my time, the company's money, the team's resources into building this thing that we want to build? And most importantly, who will ever care about it?

[00:02:40] Sometimes I have very creative people on my teams, and they come back on a Monday after their weekend and they're like, "Oh, can I show you what I built over the weekend?", especially now that AI is a thing. And they show me these really fun but really weird products. Then I can see the silence in the room: should we maybe prototype it? Should we ship it? And in the back of my mind I'm always thinking, who is going to care about this product? Not because the product isn't good. Maybe it's a good idea, but they haven't really thought about it.

[00:03:18] And it's not just about asking whether anyone will care, because that's distribution. If you don't have a strong sense of who you're building for and what problem you're solving, your entire distribution strategy is non-existent. This is the question that AI doesn't ask. In fact, it makes it easier to skip this question, because it's so easy to get a prototype that is so convincing and exciting that it's very tempting to just move forward really quickly.

Three things we built without asking

[00:03:51] We found this out especially at the beginning. These are three things that were not failures, because you always learn something, but I do think of them as a bit of a waste of time. In the beginning we thought, okay, we have Tasty, a cooking app, and we integrated a chatbot. Everyone was doing chatbots in the beginning, and so did we. We thought, we're going to have an AI chatbot, and you can ask any questions about cooking. How do I convert this measure to this measure? What if I'm vegan? It really made sense in our heads. So we made this really cute tiny robot called Botatouille [?] and we slid it into the app. Suddenly users were going into the app to browse recipes as always, and Botatouille would come up saying, "What would you like to cook today?"

[00:04:50] It turns out, because we didn't engage with the users while building this feature, that the discovery of a recipe is fun when you're browsing and casually looking at recipes. But if you have to start having a conversation with a robot mouse, it's not really seamless. It costs effort. You have to think, okay, now I have to think of a question. We also found that people were asking questions that were completely irrelevant to the cooking app, like "My wife has been cheating on me, do you think I should run?" That was not the intended goal. We took out Botatouille. It's not there anymore.

[00:05:31] Another example where we should have involved the audience, especially because it's in our newsroom, but we didn't, because we were so excited to surprise them with a new tool, was something that we called Time Hop. Let's say we had an author who had been writing for three years, and we could see that every April they'd been writing this type of content, and they performed really well for this type of story. So come April '26, here are some stories that you should be writing, based on all the data that we have, because we think these stories will perform really well. We really thought they would love it, and they never used it.

[00:06:14] Part of the reason they never used it, which is so basic, is that we didn't integrate it into the CMS. The simple fact that they had to go to a different tool, that they had to remember where the tool was, was enough that they wouldn't adopt it. Again, why didn't we ask them? Why didn't we engage them? It's not just about asking. It's also about having them feel responsible for the outcome, so that they will want to use it at the end. These are the oldest lessons. This is nothing new, but somehow when there's a new tool, you forget everything you've always known, you start with great enthusiasm, and then you have to relearn the lessons of the last decade all over again.

[00:07:00] The last example was an app that we've been working on for quite some time, where initially we wanted to build, well, our CEO wanted to build, a social media platform based on AI. There was a long silence when he said that. We weren't sure what it even meant. Then we did see what it was when Sora came out, and it was exactly that. On paper it sounds amazing: social media powered by AI, what can go wrong? Except that people don't care. In fact, when they hear AI, they're like, "Oh my God, no way, I hate it."

Learning faster, not just building faster

[00:07:51] So those were problems that we tackled. But AI really is a force for change in so many ways, and the question is, what is it good for? How do you use it so that you actually ship something valuable? To me, the biggest thing it's good for is not really building faster, which it does, no doubt. We've never been able to ship things as fast as we have in the last maybe nine months. But speed in and of itself doesn't really mean that you're succeeding. You're just moving faster, but in which direction? The question is, what are you learning faster, and how do you change course based on that learning? For that learning to really come to you, you have to set things up differently.

[00:08:47] But isn't it nice that we can fail faster? Yes, it is, because having spent one year on that one app that we haven't launched yet is painful. It's very expensive. So think about the learning, and think about how you want to use AI and how you want to structure your process so that you can actually learn much faster.

Prototypes as a decision tool

[00:09:12] One thing I often hear is that now, because everybody can build a prototype, even I can build a prototype and I've never coded in my life, the design team is not really needed anymore. It's just a design shortcut. That couldn't be further from the truth. But the prototypes that everyone is building now are so useful in specific ways. The laziest PMs are the ones who will describe a feature with words on a video call: "And then I'm thinking the user will click on this button, and then this thing will come up," and they go on and on. "Do you get it? Do you see it?" And people are like, "Maybe, but some visuals would help." Then they start visualizing on the whiteboard on the spot. Then there's the napkin mockup. Then there's the really diligent PM who will create a mockup, and then you ask, "Can you turn it into an interactive prototype?" and it used to be a week or two before you could have that.

[00:10:22] Now a PM can use this tool to even understand what it is that they actually want to build. Some of the PMs on my team use it as a brainstorming tool. They brainstorm with the AI. They have a vision, they have an idea, but as they start seeing it in action, they realize, "Oh, that doesn't work," or "It would be much more fun if we did it this way." So by the time they bring their idea to the table, they've really worked on it in the most practical possible way, and then it becomes easier to align with the engineers, the designers and the executive team, because there's nothing like experiencing an idea to make a decision faster. So it is a decision tool.

[00:11:10] One of our teams built Quiz Party, which is a synchronous and asynchronous app for taking quizzes with your friends. They went from the Lovable prototype to an actual app on TestFlight in one month, which was something that, especially in app development, we'd never been able to do before.

[00:11:34] We had another example where we were building that one app, the social media for people who love AI. As part of that, someone came up with an idea for a feature and said, "Actually, that would be really cool as a standalone app." Normally you'd be like, "Shut up, we're not going to do that." But now you can actually experiment with what it would look like. So in a week they built a prototype that really gave us the answer: could this be a standalone app? And we felt like yes, it could. It's much stronger as a standalone, simple feature. This is the power of prototypes: to make decisions faster and feel more confident in the decisions you're making.

Agree the success thresholds up front

[00:12:21] I think the other learning is what happens after. How are you going to decide whether you should continue building the app, or the website feature, or whatever you're building? When should you kill it, and why? When should you upgrade it to something more robust? When should you scale it? Because AI allows you to work so much faster, people get into the craziness of this development cycle, and they don't do the diligent work of setting up a framework for measuring success up front. That means that once you start getting feedback, you don't know what to do with it, because you don't know if it's good or bad, or whether it could be better. It's unclear.

[00:13:12] So clarify what you want to see as a result of something new you're building from the get-go, and then decide: if this happens, we're going to do this; if this doesn't happen, we're going to do that. It's going to save you so much time and so much politics. Everyone will use AI to come to the meeting with a really strong argument as to why we should keep it or not. It's useless. You just have to look at the numbers, and you've shaken hands on those numbers at the beginning. So no one is trying to confuse the cards on the table. That's the story; now, what's happening? Part of it is also making sure that you take the time to integrate your analytics into whatever product you're building. AI can do it for you, but I've found that sometimes it makes it a little harder to trust the data you're seeing. So it's worth taking the time to hook whatever you're building up to your proper analytics system.

[00:14:18] We did this experiment where we allowed a number of editorial team people to publish whatever they wanted on the site themselves, literally shipping code into production, which we've never done before, for obvious reasons. But creativity can be anywhere in the company, and in fact our editorial team is the most creative. So we empowered them to build things, because now you don't need to code to do that, and to ship them. The only way we felt comfortable doing that was to have this graduation system, to know when to remove things from the site and when we should upgrade them.

[00:15:04] We found that users were giving us feedback in real time: "We love this, you should turn this into a standalone product," or "What is this? Why do you keep wasting our time with AI?" They're very honest, very straightforward, but that's the best type of feedback you want: the truth. And then we had thresholds. If it reached a certain retention threshold, we put it in our permanent games on the platform. So being clear on what those thresholds and metrics are, and agreeing on that at the beginning of the process, is super useful.

Smaller teams, shorter timelines, new rituals

[00:15:51] The other thing, obviously, is that if you just use a new technology but keep everything the old way, it doesn't quite allow you to make the most of it. We changed two things. One was timelines. We gave our teams challenges to build things in one month. It was completely arbitrary; there's no science behind it. But a month felt like the kind of timeline we were willing to waste if whatever they were building was not good. We lost a month, no big deal. We lost a year last year, so we can afford to lose a month.

[00:16:37] The other thing is that if you only have a month, you have to have a much smaller team, because even just your daily standups, the alignment meetings, the kickoff, would normally take a month. So three people. Of course, this is our experience with specific entertainment products; I can't say it applies to everything. But at least in the beginning, the fluidity you gain when you only have three people who make decisions by themselves, without having to explain themselves to anyone else, is fantastic. They really built an entire app with a lot of features in one month.

[00:17:16] The other thing we saw was the blurring of their roles. The designer was acting as a product manager at times. The product person was shipping code. The developer hated AI in the beginning and was so reluctant to use it, because preserving the processes they'd been working with for years was so important. But then they had no choice. When they saw that the product person was shipping the same thing so much faster, they had to really embrace it. So it was a great way to get traction, even with people who are very talented but didn't want, or didn't feel the need, to change their way of working.

[00:18:03] Then the rituals change too. At this point there's no real need for a daily standup where you say, "I closed this Jira ticket yesterday and now I'm working on this Jira ticket." Those are meaningless updates, because the reality is that if you're getting feedback so quickly, you can make changes. Making changes is so much cheaper now, if you have a good reason to do it. When you sync every day, the sync is not about what to work on. It's about what you want to change compared to what you were building the day before. If your team sees feedback as a disruption, that is a great signal that the process they're working with hasn't adjusted to make the most of this new technology.

Four simple things with a big impact

[00:19:03] I couldn't help adding a little listicle to this presentation, because the temptation is always very strong. Plus, I didn't know how to integrate these four points into the rest of the presentation. So now for something completely different: here are four surprisingly simple things that we've done that had a big impact across the board.

[00:19:27] Number one: spend two hours. I say a day, but even two hours a week is going to save you 20 hours a week forever. It's so tempting to give up when something isn't working. You're frustrated, there's never enough time, so you revert to how you used to do things. But if you really make a commitment to figure out a way to do it with AI for two hours, that's a huge benefit. In a month you're going to be a different person. You're going to be so much more autonomous and independent, and the amount of things you can do in the same time is so much higher. That's a whole other problem, but at least you can do more with the time you have.

[00:20:23] The second thing is working on a personal project, and this is for people who are more at the beginning. Everyone has individuals across the org who are more reluctant to integrate AI into their workflow. Also, sometimes the stakes are so high that you don't want to make a mistake that's going to cost the org. So starting with personal projects, where you have a strong personal interest in what you're building, is a great way to then come up with ideas for what you can build at work.

[00:20:55] We have someone on the team who has twins, and he's in charge of training the entire running track team of their kids' school. He decided to put all the performance data of these kids into a Monte Carlo simulation, and he would see, okay, based on this performance data you have a 65% chance of winning the race next month. Then he asked, what is the custom training program I need to give each one of these kids so that our chances go from 65 to 95? And Sharina [?] got custom training sheets for each one of the kids, he proceeded to distribute them, and they actually won the race, which I couldn't believe, because it sounded like sci-fi. Because of that, he came to work and said, "What if we apply this to our editorial team? We put in all the performance data for every author, and every day when they open the CMS there's a suggestion: you should write about this, but with a slightly different angle, for this audience." That idea came from a personal project. So that's another way to have a spark and be excited about using the tool.

[00:22:16] The third thing is: don't get too hung up on the old processes that are so dear to everyone. Focus on the input, because you're the expert on the product. Focus on what you want to get out of it, and then let AI figure out what happens in between, even if it's something you feel slightly uncomfortable with.

[00:22:43] And finally, start with something that you yourself have already mastered, because it's easier to evaluate the quality of the output when you've been doing that yourself for a long time. When you build something that's brand new, at that point you really have no clue. So in the beginning, of course, we started with CMS feature implementation and how to build a quiz maker that used AI, but that's because we had already mastered those things, and it was easier to tell good from bad and acceptable from not.

Poll: what is your bottleneck?

[00:23:26] This is the last slide, and it leads to an interactive poll where I ask you: what is the bottleneck in your organization for integrating AI successfully? I can't see this slide myself, but you should be able to see it with the QR code.

[00:23:51] Host: All right, everybody grab your phone and answer this question. This has been an amazing conversation. There's no experiment graveyard.

[00:24:13] Cristina: Yeah. What's the point of an experiment when you could just launch, right?

[00:24:16] Host: Right. There's no graveyard anymore.

[00:24:24] Cristina: It's interesting. "Forgot to ask why" is a bottleneck, maybe a bottleneck for success. If you answered "I don't even know", that means you're the bottleneck. Keep that in mind.

[00:24:35] Host: Oh, that's so good.

[00:24:37] Cristina: It's true. So, what do you think of your answers? I'm surprised there isn't more "forgot to ask why", because I find that that is the biggest issue. But I'm not surprised about process, because changing people's habits is oh so hard.

Q&A

[00:25:05] Host: Thank you for the poll. Okay, let's move into Q&A, and first, a round of applause for Cristina. It's going to be hard to take Botatouille out of my brain. Did anybody here sketch Botatouille in their notebook? I was hoping a designer was sketching.

[00:25:27] Cristina: It was very cute, I'll say that. The designer did amazing.

[00:25:30] Host: Like robotic, or...?

[00:25:32] Cristina: It was robotic, but it had a really cute face. He's in the graveyard.

[00:25:40] Host: I bet. When you move fast like BuzzFeed, I'm sure you have a lot of that, right? A lot of experiments. I think speed is your thing, responding within minutes or seconds.

[00:25:53] Cristina: Yeah. That's why at the beginning you should really think about who you're going to ask for feedback, because this is a question that always comes at the end, when you already have something, and then you're going to waste so much time even gathering that feedback. No one has time for that, so you skip that part. So you have to ask at the beginning: when, what for, and who am I going to ask for feedback? And then it will happen.

[00:26:18] Host: Yeah. I make teams guess the metrics beforehand, and I saw that you do that. People will put a button on a page, and I'll say, "How many people coming to this page will click it?" In a group of five I'll get 80%, 60%, 20%, and in my brain I'm thinking four and a half percent, three and a half. And for the 80% person, I'm just thinking to myself, maybe you've never done this, or you just have a very short memory. Okay, let's take a question here. What does documentation look like in these new systems? Do PRDs still exist? What's your source of truth? Is anything written down at BuzzFeed? You were talking about how people just like to talk.

[00:27:01] Cristina: Yeah. I used to be a Nazi when it comes to PRDs. If there was a roadmap, I would ask, "That's cool, where's the project brief for that?" And of course it was, "Here's a project brief," and it's a Word document with one paragraph. That was my nightmare. Now we don't necessarily have a process for everyone, because different people are adjusting in different ways. But if there's a prototype available, which most of the time there is at this point, that becomes the core of the PRD. The document still exists, but it's so much shorter than before, because you don't have to explain everything in every edge case. You have most of it available to experience. So the goal of the document is more about coordinating how we're going to ship it, what the milestones are, and less about...

[00:28:04] Host: Right, when it's in the prototype, why write it down? Of course, the question is, if it's in that person's personal Lovable account, how long is it going to stick around? So I think this documentation is amazing, but yeah.

[00:28:17] Cristina: It's a miracle already if you have documents from two years ago, so that's not my worry, to be honest. My worry is that the number one thing we still have to write down is what the goal is, how we're going to measure it, and who's going to give us the feedback. That has to be there.

[00:28:34] Host: The group wants to know: did you have a candlelight vigil for Botatouille? When Botatouille went away, was there any seance or ceremony, or was it just BuzzFeed, "Yeah, gone"?

[00:28:45] Cristina: No, but it was suspicious. We kept saying, "Can you remove Botatouille from the app?" and the team was like, "Yep, yep, yep," and then it was never going away. Every time I opened the app, it was still there, and it was taking up such premium real estate on the main screen. So it was a very slow death.

[00:29:06] Host: That sounds like the exact opposite of a candlelight vigil. Torture, slow death. Okay. What's your favorite prompt that produced quick and decent results?

[00:29:15] Cristina: Quick and decent. Oh: make it short. Especially if you use ChatGPT, you know that that AI does not know how to be succinct or concise. It never stops writing, never, so I don't use it anymore just for that. But sometimes I don't want it to be quick, sometimes I want it to be thorough, so I always specify what I want to see as a result. And I still try so hard to say, "Don't sugarcoat it, don't agree with me on everything," and that's the part it still does. It really does. People pleaser.

[00:29:58] Host: Let's see here. There's something in here, I forgot the question. These have been answered. Will you go ahead and pin these or dismiss these? Click up to the top questions; we got clicked down to the answered questions.

[00:30:22] Cristina: Who are you talking to? Me? Okay, I was like, I only have one clicker.

[00:30:30] Host: You should just take over. This is good.

[00:30:34] Cristina: I'm really curious to see what's happening here. Based on these words, I'm afraid of seeing the real question.

[00:30:41] Host: Joseph or Raleigh, would you click up the tab that says top questions instead of answered questions? It's top, pinned, newest, oldest. Just go up to that other one. Perfect, thank you. Okay. There was something about moving so fast: do you do retrospectives on this? You're clearly thinking about this, you're clearly changing process. Is it unilateral, as you see things, or is it a group effort?

[00:31:13] Cristina: We definitely do retros, and it's very interesting. I used one of these retros to put together this presentation, from the team that built an app in a month, and it was very interesting to see. The number one thing I loved to see in that retro was that every one of those three people had more fun. They were more excited, because they could do more than they could before. So it's good to see from their perspective how things went and what they've learned.

[00:31:42] Host: How are you balancing Figma and those types of tools with vibe-coded tools?

[00:31:48] Cristina: Again, everyone is adopting things at a slightly different pace, depending on their preferences. All we want is for people to try, and of course some people will be more familiar with Figma, others with Claude. So I don't think we have a one-size-fits-all format right now. People are just used to different things and they gravitate towards them.

[00:32:16] Host: Okay, thank you so much for your insights about all of this.

[00:32:22] Cristina: Thank you.