0 to 1 Product Orchestration: Building AI as it Changes the Way We Work

Haydyn Phillips at UXDX EMEA. Video: https://youtu.be/cV_2Ga3KwL0

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

First time in Berlin, and why I changed my slides

[00:00:07] Okay. Well, it's my first time in Germany and therefore my first time in Berlin. For those that work and live here, thank you for sharing your city with myself and my better half. We've had a fantastic time. I'm going to get this wrong. Volchixen[?]. Good. Yeah. Okay, pronunciation. We're staying by the river Spree, so we're going to experience downtown Berlin.

[00:00:32] Just a couple of things before I begin. I was really inspired by yesterday's two opening talks: the panel talk around who should be worried about user experience in AI, and then also Dez's talk. I don't know Dez, but his candid nature, the way he talked about user experience and building at SaaS, really resonated with me. So I went home and changed my slides, much to the dismay of Paula, who sent me about 3,000 emails saying, "Haydyn, can I have your slides? Can I have your slides?" Thank you for being eternally patient with me, Paula, and for the AV team as well.

[00:01:06] I'm here to talk about one thing that's very close to my heart. I've spent the last two and a half, three years building around large language models and other related types of models. But also along the way I've noticed a fundamental shift in how we as UX people design, research and orchestrate things. And that's what my talk's about. It's my own personal reflections and views of building a zero to one AI product platform at a very large company that's very regulated and that requires jumping through lots of hoops to deliver things, and then also the behavioral stuff along the way as well.

[00:01:46] I don't know whether it's just because I'm old school, but I do have an intro slide about me. When I go to conferences, I like to be able to know who's standing in front of me saying stuff. I've been working in user experience for about 18 years. I've had roles across interaction design, qual and quant research, delivery and leadership. I've built teams across various industries, and for the last three years I've been working at AstraZeneca. I come from computational psychology, and I decided, as I got towards the wrong side of 40, to do a PhD in two separate scientific disciplines, much to the bemusement of my wife. Don't do that. It's not a good recipe for a good life balance.

[00:02:31] AstraZeneca is an incredibly large company. It employs about 90,000 people, give or take, in about 85 different countries. And I'm telling you that for a particular reason, because of the complexities that exist in working in enterprise. Perhaps that's different to delivering products at scale at SaaS or fintech or telco or other verticals that you all may be part of. I like to run and eat. You can probably tell that the eating is winning at the moment.

AI is hard: building zero to one in a regulated enterprise

[00:03:06] So this is why I'm here. AI is hard. That could probably be my talk. Building zero to one is an incredibly difficult endeavor. As I mentioned before, AstraZeneca is a regulated company. That means that we're not only beholden to regulators externally, but also to our own internal regulations around GDPR, data privacy and the way that we build things.

[00:03:26] What stood out to me yesterday from Dez's talk was this transition from the command line, the CLI, perhaps in old school HCI, through interaction design to the GUI. And now we've gone back to this innocuous little chat box that has a multitude of complexity behind it, and that is currently how millions of people are interacting with knowledge and data. It looks harmless to design, it looks harmless to build, but beneath this I'm going to take you on a journey for the last two and a half, three years of the complexity that goes on in building enterprise systems around this.

[00:04:05] Now, to give you some specific context. Oh, by the way, I'm not a slides guy, so hopefully my content is good instead of my slides. We were tasked, there was an email that landed in my inbox about two and a half, three years ago, and it said, "Haydyn, can you and your team come do a piece of concept work with us to figure out how to build an AI PoC[?] around emerging LLMs?" We all know what's happened in the last three years. And that two-week[?] interaction from my product team scaled into a full-scale two and a half, three-year project, which means now we have an enterprise internal AI system that our colleagues can use safely and securely. They can experiment and learn from using AI. They can understand somewhat how large language models work and don't work at a behavioral level for doing their work, and they can understand all the tooling around it as well.

[00:04:57] So this is different. This is a different challenge when you work in enterprise. You are serving the needs of those that work with you, your colleagues. And I mentioned before the complexity of AstraZeneca. It goes right from clinical through to IT through to manufacturing through to some of the smartest scientists that you'll ever work with. So there's a lot of disparate behavioral knowledge. There's a lot of different ways of working and thinking and doing. The enormity of building a scalable AI product across a regulated enterprise business cannot really be understood without understanding how much complexity goes in.

What you are actually building: the orchestration layer

[00:05:34] Behind all of this I thought, well, what really are we doing? If I put my behavioral science hat on, my psychology hat, and I think about the behavioral aspects of AI and what we're actually building, it's unlikely, although there may be people in this room that train large language models, that have built this stuff from scratch, that have trained it on billions of data points, this isn't that. This is building the orchestration layer around these language models, with RAG, with other tooling and stuff like that. And it's about connecting all the pipelines and the gray areas that really define the experience when you're building with large language models.

[00:06:10] I kind of split this into three levels at the technical and product level. This is for engineers, this is perhaps for the product teams that work in the design tools. You're not really designing with large language models. You're designing for retrieval. You're designing for understanding what happens between the language model, your data sources, and perhaps external information, if you have access to the worldwide web or other sources like that.

[00:06:35] You're also thinking about accuracy and personalization. I'm not going to get into the mechanics of large language models and hallucinations or how they work, but a lot of this is around grounding data and figuring out what is good and what people expect when they get this information back from the prompts that they're given. So you hear a lot about prompt engineering, but the other side, where we're designing system prompts and the structure around this and what you can ground and what you can tweak, that doesn't often get talked about.

[00:07:04] Because this third point in my technical and product stack, LLMs as the material, I feel is a vastly under-represented topic in user experience. At the end of the slide deck I'll talk about what I think needs to happen next in our industry. But I was always taught that if you understand the material that you're working with, you will be a better designer. You will be a better researcher. You will be a better product strategist. So understanding large language models and related models as a material is really important when you're building AI systems.

The operational and behavioral level, and "so what?"

[00:07:36] What about the operational and behavioral level as well? This is about evaluation as continuous improvement. This isn't just about the system you build around it, how someone accesses, let's call it a chatbot, or the interface to communicate with the LLM. This is also what happens with the technical evaluation, things like red teaming, working with engineers to understand what is technically feasible with what you're building and how you're building.

[00:08:02] And then also designing for failure as well. This isn't about designing for delight and state machines and things like that. This is literally: can you design the thing? Is it physically possible? Is it technically possible? Can you get so far as we've done and build a part of an AI feature and go, we just can't do this. It's just really, really difficult. We don't know how to do it. It's not a failure in the sense that you've not achieved it. It's just you don't know how to do it yet. And that's par for the course. And then feedback loops as well, which I'll talk about in a second.

[00:08:34] All of that sort of looks at the technical side of things for you user experience people. But if I take a massive step back and I think, I'm in a leadership position, I manage design and research teams, I also have responsibility going upwards as well, we're building these systems. So what? That's what an old manager used to say to me. So what, Haydyn, what's the end result?

[00:08:56] I think really when you look at the outputs of large language models and the impact that they're having on all of us, these are the types of things that I'm thinking about as someone working in this space. Trust calibration, for example. Do the people that you work with, and who interface and interact with the product that you're building, do they trust it? Do they really trust it? Because you're spending time, effort, money and energy, and you're also getting buy-in from C-suite to build this stuff. Does the end result validate what you're doing? And of course all the AI governance that goes around this, that's a separate topic we're going to have to talk about today.

[00:09:32] But what's value? If you're building an AI product and you're just measuring usability, and I'm going to say some stuff that some people won't like today, if you're just doing small sample usability sessions and you're not leveraging data science for building AI products, you're going to struggle to realize both the product value, the technical value, and also the organizational value of things as well.

The orchestra metaphor, and what happens when the score keeps changing

[00:09:57] This is a picture of an orchestra. The reason I put this in is because I thought this was an apt metaphor for how things work: both how large language models and the orchestration of building and designing around them work, but also the behavioral layer, how we operate as well. You have a conductor, you have a person. This is a big band orchestra. I'm not a music person, but as far as I understand, you have the woodwind section, brass, strings, percussion, and things like that. And it's this person's responsibility to orchestrate the tempo, to help the sections hear. If one section's too loud, they can't hear. What if something happens? What if a string snaps? What if someone can't play their piece?

[00:10:37] And it made me think about the responsibility that we have when we're orchestrating AI systems, because of the speed that this is moving. So what if this doesn't work anymore? Because orchestras are deterministic in the sense that you have a score. You follow a sheet of music and stuff can go wrong and it will go wrong, but if all the parts of the orchestra play their piece, you're going to get to the end of the score, everyone's going to clap and they're going to leave.

[00:11:02] This doesn't work in the new world, because models are coming out at the pace of, I don't know how to describe it. They're coming out fast. We think about MCP. We think about the new protocols for agentic. This stuff is moving so fast. What do we do when the change is n times faster than we can keep up with? How do we orchestrate that? And this is what I've experienced firsthand as someone who's running a team that's being held accountable for the experience of enterprise AI, and also working with the engineering and product teams as well.

[00:11:35] So there's three areas that I think that we need to think about when we're designing with and for AI. This is about orchestration and stability. How do you design for continued stability when stuff is changing all the time? How do you align around a shared context? And I'll give you some of my personal thoughts around this.

[00:11:53] How do you experiment and think about risk? This isn't about, I'm in Figma, should I choose this button over that button? This is about material enterprise risk if we ship this feature. Can engineering build it? Can we support it? Can it go into BAU? Can we sustain it over two, three, five years, etc.? And what about measurement and impact? As someone who's got a behavioral science background, the economic impact of what we're building is huge. What's the net result for all of this, when companies invest lots of money or we spend lots of time building these products?

What really matters, and doing it first

[00:12:33] But then I thought, let's ask a fundamental question. The reason I mentioned the panel yesterday morning, I can't remember the lady's name, but she talked about designers having a seat at the table, which has been a conversation forever. But this is the fundamental question to me as someone building AI: what really matters? Like really, what really matters? And when we started this project we asked some of these questions to ourselves. What matters? Is it what we build? Is it how we build it, the performance of the team? Is it how we measure it? Is it the technical capability? Do we need different types of AI or ML engineers? How important is AI governance? Very, very important. And how can we scale this and keep this going?

[00:13:17] Going back to Dez, and I keep referencing this because it just stuck in my head overnight, he talked about AI first. I'm more of a just do it first. I've been there and done it in my 18 years of experience of product discovery, big design up front, agile, waterfall, you name it, whatever. And it still absolutely has its place. But from my experience, delivering something at scale and speed requires a different mindset and it requires us to think differently.

[00:13:43] No, I'm not asking you to drop craft, because I know I'll get pushback on that. I'm not asking you to lower your standards. But as things are moving so quickly, this is, in my book, the new playbook: deliver the AI thing. Because by the time you've delivered the AI thing, the foundational model or the diffusion model or whatever you're building around is likely to have changed, the parameters will have changed, the quality will have changed, and you have to manage all of that again.

[00:14:11] So this is what we thought about. What did we know? Did we have the right skill set? What was going on in the industry? And then the dynamics of our business as well. And I think one of the key things that UX people probably need to get better at is understanding the politics and the dynamics of businesses in relation to the impact of design. This is what we thought about quite heavily when we were building this. A tongue in cheek quote: good job everyone, we've solved zero to one enterprise AI. We really haven't. But this little model got us from a PoC to a business as usual product which is used by a substantial amount of our workforce today.

The operating model, and role collapse

[00:14:51] I put this in just to show, for the next slide, why this is different. So this is the current world. You have silos. This is the bit I owned on the left: product experience and research. I called it product experience instead of user experience because it's easier to sell to the VP and C-suite. We're designing an AI product. We're responsible for the experience of the product and we're doing research. It's that simple. There's no complexity to this. I know there's lots of debate about should it be user experience, product experience. I don't care. In our enterprise, this worked for getting us where we needed to be. Now, you can choose to adopt this or not. It's up to you, but I thought it was worth mentioning how we framed this. And then you have engineering and technical. We had product management, comms and change.

[00:15:37] This is what our operating model actually looked like: this amorphous blob. I have data science reporting to me, data analytics. She's a fantastic lady who works in Barcelona. She then works with project engineering and develops ETL pipelines for our data analytics. I don't care. I have zero issue with that. She goes off and owns that piece of work even though she reports into my reporting line. We have our principal engineer who leads UX work, a brilliant guy out of Barcelona, PhD. I have no problem with him sitting down with my principal designer and saying, person, we need to do this. In fact I actively encourage it, and that's what I'll talk about at the end as well.

[00:16:18] And the biggest impact that I've seen with our lead qual researcher is the interaction between project management, comms and change. Comms and change basically acted like a user research function, because they were attached to the AI governance and plugged into the business, and they helped and supported us. So these roles are changing. Role collapse is real, absolutely real. And maybe not in the ways that it's been described in some industries, but from what I've seen and the examples I've just given you, these protective silos, these boundaries, they should not exist, especially because things are moving so quickly.

[00:16:51] So perhaps if you run a team, or you're part of a team and you're pushing for change, I would fundamentally encourage you to embrace new boundaries. If you work in Figma and you want to do some data science work and that's your background, find a way to do some data science work. If you're a PM and you want to do UX work, do it. Role collapse is real. My function owned the product-centric ways of working. We own the PRDs. And at the end, I'll say something about PRDs. This wasn't product. This was product experience or user experience. So we had completely different ways of thinking about things.

[00:17:27] You are a PM. Now you might not be a PM, but when you're working with highly complex, highly regulated products like language models, and you're building AI systems, you are, in my experience, responsible for product and project managing what you do. And that's a new model as well. So if you are a designer, a researcher, my lot have complete autonomy mostly in what they do and I trust them to deliver it, which means their role is mostly disambiguation, in the sense that it's their role to make things crystal clear for our stakeholders and for the product teams. And then instead of doing long-term planning, we planned for evaluations, bi-weekly demos and tighter cycles, because we learned stuff just got compressed way too quickly.

Experimentation: complexity versus stakes

[00:18:10] So what about experimentation? The way I think about this is complexity versus stakes. In the top right, high stakes, high complexity, we know this exists. There's a new AI feature, which I'll talk about. Versus low complexity and low stakes: we can add value quickly and efficiently. This might be, for example, my principal designer says, I want to do quality of life improvements on the system. I can do this in Figma really quickly, we can ship it to front end, into Storybook, whatever, and we can get it done.

[00:18:43] That's different to perhaps when we need to think about the harnesses around the language models and the systems and RAG and things like that. So we need to make time. It's high complexity, but the stakes are a little bit different. Whereas governance related things, which have a material impact in the product, how long you keep someone's data, what you do with it, are high enterprise stakes across everywhere. So this is the way that we conceptualized experimentation and risk.

[00:19:07] And here's another amorphous blob of the types of experimentation and risk that we worked with, just to give you an example of the concrete types of AI things that we were thinking about as we were building this around our models and our technical pipeline. For example, deep research, voice chat, frontier personalization. But when you take a massive step back and you put your HCI hat on and you think about voice, for argument's sake, voice is a fantastic modality for lots of different interactions. We built a labs for all of this stuff and we usability tested it.

[00:19:42] Can you imagine perhaps someone from the senior executive team on a flight, or someone on a bus, giving away confidential information in a public space? Or perhaps the engineering effort required to build deep research as an AI feature, if only 10 people use it, what's the value there? Or perhaps frontier personalization, as new models come in and we look at the system prompts and things like that, what's the value in that? So when we're talking about highly complex, high stakes stuff, this is the type of stuff that you probably likely interact with in your personal subscriptions with AI. But when you're building this stuff, or you're building around this stuff, it has a completely different way of thinking about it.

[00:20:24] Some thoughts around this. If you are building AI, and again it was mentioned yesterday, these types of plugins that are bolted onto software is, in my opinion, materially different to when you build a, and I won't use the term foundational, a new platform that is built around language models or other models. That is a different type of experience, and that was mentioned quite critically. So if you're going to spend time doing this, will it add material impact? You need to make that decision with your product teams.

[00:21:00] The other thing that people do is they get really giddy and say, "Let's build a thing." Can you support that thing in six months? Is it going to cost you a million pound a month? Do you have the engineering, technical support to be able to do that? These are all realistic questions. And then finally, can I measure it? This is the biggest one for me. Again, putting my behavioral science hat on, my psychology hat.

Measurement: the most under-represented part

[00:21:25] To me, this is arguably the most under-represented part of building and measuring AI, is this part. So let me talk about this in a bit more detail, because I've zoomed through the other slides. Measurement is the act of doing the thing. You measure the thing, whether that's actively or passively. You speak to people, you spend time in a lab, or you collect analytics. I'm not going to talk about optimization today, because that's more sort of geared towards the models themselves and I'd value an engineering colleague to talk about that. But thinking about making better decisions.

[00:22:01] If I put 10,000 euros on the stage right now and you're a researcher, how many of you could come up here and take that £10,000 and say that I know how to accurately measure AI impact? That's a rhetorical question, but you need to answer it. I'm pretty sure, myself included, that I wouldn't be taking that $10,000, or dollars or pounds, because it's really difficult.

[00:22:28] So here's the types of things that we looked at, and this is not exhaustive by any stretch of the imagination. As well as building AI, we built a foundational analytics stack from scratch, because we could and it was the right thing to do. If we think about active data collection, or measurement, that is where you go out and ask someone some stuff. That's the simplest way in my head that this works. So perhaps you do some usability testing, you sit in a lab, or you do surveys. So we're thinking about metrics like UMUX-Lite, SUS, NPS.

[00:23:01] And for what it's worth, NPS gets absolutely battered, but again it's context. If you're in an enterprise environment and that metric holds value to the people that perceive it, then you should use that metric. This is all contextual. So it's all relevant to what you're building. And then I mentioned before about the economic value, measuring productivity and efficiency, training and learning, because enterprise AI requires also people to upskill, which is a fantastic part of this product.

[00:23:30] But then also the analytics. So, monthly active users. We treated this like an e-commerce product. For example, compound monthly growth rate shown yearly, active users, unique users, so we could see what's happening. Are people using the features, the part of the product? And retention: are they coming back month after month? And then we built a separate data science pipeline which was around topic modeling, regression, predicting variables, understanding what's happening.

[00:23:56] And that's really where I see the industry going. If you're in a research position and you're mixed methods or quant, then product data science is where it's at, especially when you're measuring hundreds of thousands and millions of data points. That's where you need to be. So there's a massive opportunity to upskill. I'm only going to say this once, because this is the critical point. Baseline data is the north star. If you have nothing to measure against and you're trying to measure something in the future, what are you measuring? Maybe that's your baseline data, I don't know. But my point is you need to have something to compare to, to show progress, whether that's good or bad.

What we learned from measuring it

[00:24:34] So what did I learn from this? Well, one of the things that we did was build a dashboard, and we got the metrics out to the C-suite, to the VP, pretty early so that they could see what was happening. And we saw this gradual adoption curve, and we could see the prompt usage, and we could see the adoption across the different parts of the business, and we had all our features dashboard. We could see the usage. That was critical for us, because it was a piece of comms and change, which was a behavioral change piece of work essentially saying, look, here's what we're doing, here's the metrics to support what we're building.

[00:25:06] Measure what makes sense. So don't measure what you can't understand or don't understand. That's sort of a foundational rule in statistics. If you create a bunch of stuff and you don't know what it means, don't do it, or get someone who does. And then finally, I've held mixed methods positions in my career, and I cannot stress this enough: if you're a qualitative researcher, at least in my industry, pharma, in the projects that I'm working on, you are in demand. Absolutely in demand. Because as much as the data science is important and it builds the business case, the felt experience of using AI within an enterprise organization, the way people describe what they're doing, is so valuable to people. So I would actively encourage you, if you're a qual researcher, to keep pushing in this space.

What I'd do differently, and what are you orchestrating?

[00:25:53] So, last two slides. This of course is an image I spent five days in Photoshop designing. Obviously not. This is what was spat out by one of my subscriptions. I asked it to visualize the complexity of zero to one AI and this is what it came back with. Some of it works, some of it doesn't. Hopefully this has been an insight into what goes on at enterprise level, the complexity involved, the responsibilities that we take seriously. And we're just scratching the surface, as are most of you. We're still trying to understand what's going on.

[00:26:28] I wanted to leave you with this. A bunch of people might not agree with this. Someone asked me the other day, "Haydyn, if you had x million pound and you were to do this again, what would you do?" And I said, everything. Probably everything. I'd change everything again. I'd focus on speed. I'd refocus how time is spent in design tools. I'd cut out low value research. I'd go straight to code. I'd flatten planning. I'd use roadmaps to position at exec level, but not use them for actual planning. I'd probably get rid of PRDs and a lot of technical documentation.

[00:26:59] Where I would focus is driving deeper L&D for my team around the engineering, technical and mathematical parts of large language models. Like I said before, know your materials. LLMs are really, really complex, maybe not so much now as they were two years ago, but understanding them and what they can do is really important. I'd also push further for engineering to own UX and for UX to own engineering.

[00:27:22] And then most of all, the question or statement that I'd like to leave you with is, I'd like all of us to take a step back and ask ourselves: if you're working in AI right now, what exactly are you orchestrating? Thank you.

Q&A

[00:27:42] Host: Thank you so much. This was really inspiring for me. I learned a lot of new things right now and it's just fascinating. We have a lot of questions, so we're going to ask a couple. When what you're building is new and has no baseline data, just what you mentioned, what's a good way to define whether the results of the new thing are good, bad or different?

[00:28:07] Haydyn: Obviously, when you think about certain usability metrics, there's percentiles and things like that you can look at. This is still emergent in the AI industry, so there's usually a reference point for what's good. You've seen the categorization of like A, B and C when you're working with SUS or UMUX data and things like that. NPS obviously has detractors and things like that, so it's worth looking at. We used industry standard benchmarks initially, because then we had a reference point. If you're making stuff up, which is also fine, you can make stuff up, make sure that you can then measure that again. So if you're building scales around AI behavior or impact or productivity efficiency, then it's worth saving them. I'd go industry first, build your own stuff, and then scale it out.

[00:28:51] Host: Scale up, yes. Did the usage of internal AI tools come organically, or was there a push from the top? So an organizational decision for employees to use it. If there's a push, does it taint the experiment results?

[00:29:07] Haydyn: I think that's a really good question. Well, of course, if you've got top down stuff and people are mandated to use AI tools, then there's probably friction. You get pushback. But I think we saw a huge amount of organic growth. It was a really high percentage year-on-year growth across our business. So yeah, it was mostly organic, and then what you'll get with distributions, you'll hit the top of the distribution and it'll stay there and then drop, and it's exactly what we saw. It followed a bell curve. So it was organic, and I think that came with AI education and people experimenting externally as well. Did it taint the experiment results? Yeah, probably. There's always bias in research, you can't get away from that. So probably. But what we're seeing is people are using it, and the net results are going to be way in the future.

[00:29:51] Host: Did you set up any specific targets that people would reach in terms of usage? What we've seen lately from a lot of companies, this is becoming a thing, so the curiosity is if this is in your case as well.

[00:30:05] Haydyn: Yeah. So we've just started thinking about what our active market looks like and then what the percentage of users looks like for that. And I would say, thinking carefully about AI adoption, it's only one metric. It only tells you how many people are using a system. It doesn't tell you about the quality of the experience.

[00:30:22] Host: Thank you. We have time for one last question. How would you research if, and which need for, AI a company has? How would you research if a company needs AI, I guess.

[00:30:40] Haydyn: As in like the foundational need for AI? Is that what's being asked?

[00:30:45] Host: I would reckon so, yes.

[00:30:47] Haydyn: Yeah. I think, well, we did a lot of external market research and we got caught in that initial wave when language models were becoming very, very popular. So a lot of this was organic growth, as mentioned. I think depending on your industry, what you're trying to achieve, is it FOMO or is there an actual business case for using AI, and can you set up a foundation to measure that? So I think it's completely contextual, what you're doing really.

[00:31:10] Host: Yes, totally. Thank you so much. Thank you very much. A round of applause for Haydyn.