Product Teams and AI

Marijke Jorritsma, David Hoang, Chris Reardon, Keyvan Azami at UXDX USA. Video: https://youtu.be/Z-abk6Y3E1o

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introducing the panel and scoping the conversation

[00:00:03] Marijke: Is this thing on? Oh, there it is, great. Hello, thank you, exciting to be here. I'm Marijke, and joining me today for this conversation on AI and product design is Chris Reardon, creative director, head of product design, responsible AI at Meta. We have David Hoang, co-host of today's UXDX sessions, design leader, co-founder, founder and angel investor. And Keyvan Azami, enterprise AI engineering lead at Google New York. So let's welcome everyone here.

[00:00:43] Marijke: We're excited to be here. We know that this is an exciting topic. Probably the most difficult part of preparing for this panel has just been keeping up with the news cycle, which seems to be changing every day. Just to scope this conversation and provide some context, we're going to be focusing specifically on how AI may transform some of the roles and opportunities that we have as product teams, and how in turn we may have new opportunities or even responsibilities in terms of how we consider the human component of these new workflows. That's the scope of the discussion today, though I'm sure it will go all over the place.

[00:01:25] Marijke: Just to get started, I'd like to start off with Keyvan. You've been working in enterprise AI for Google, so you're working behind the scenes developing tools for AI developers. I'm curious to know a little bit. I know a lot of people have become aware of AI through tech demos like ChatGPT, Midjourney, these kinds of things that have become available to the public for people to play around with and start thinking about how they might use them. But I'm curious if you could share a little bit about the kinds of other roles and maybe workflows that are happening behind the scenes in enterprise AI at Google.

[00:02:07] Keyvan: Absolutely. First of all, I think that the advent, or the acceleration, of this technology is only going to be additive to people's roles: designers, product managers, engineers. I think the kind of thing that we're seeing is an opportunity to really rethink the way businesses run. My job along with my team is to help Google run better and more efficiently and effectively, and to do that is looking at how our processes have organically developed and how we can introduce AI to the various business functions.

[00:02:49] Keyvan: But the interesting thing is that you can't just throw AI in there. You have to think about the implications of it. How do humans work with it? How do users interact with it? How do we explain and interpret what AI decisions are? AI comes with uncertainty and it works on probabilities, so how do you manifest those probabilities into user workflows, and deal with exceptions and the higher level functions that humans can provide? There are a lot of interesting and complex ideas that it could bring to the table, and I think it just creates new avenues for folks in these different functions.

What teams supporting model building will need to think about

[00:03:39] Marijke: I think this is something that we've connected on, because the work that I've done at JPL has been a lot in modeling and simulation. NASA has a long history of performing simulations, and so we seek to make building and augmenting models more accessible to our user base, who are scientists and engineers. This is something that you and I connected on, and I think it's something I feel like I don't hear people talk about a lot. I was wondering if you could share some of the interesting thoughts you had about the work and considerations around the future teams who are going to be supporting model building.

[00:04:23] Keyvan: Yes. The new models and the new model architectures that we're hearing about have a lot of capabilities, but do we understand their limitations and their risks as well? Because there are some. In this new world where we're trying to bring in engineering, product, design with AI, now we need to really preempt some of these gotchas that we haven't experienced in real life. What we're seeing is the cool side of it, the great things they can do, but we haven't actually developed a large body of data points in how these are used.

[00:05:15] Keyvan: The teams that are designing these things need to be thinking about these implications. What are the privacy implications? If I train a model on my data, a large language model, or if I give it prompts that use my data, is there danger of leakage of this data back into broader use cases that I may not want to see? How do you address bias remediation? These models have learned from human output, whether it's content on the web or whatever corpus of data that you've fed it. Do we understand that? Obviously there's a lot of research in this space and I think Chris is the expert here, I'm sure he'll talk about it, but we need to think about how we bring that into the fold.

[00:06:06] Marijke: Great, thank you.

[00:06:08] Chris: I'd love to build on what Keyvan was saying. I think the role of design is going to change a little bit. It's not necessarily about the UX and UI as much in the future as it will be about how an intelligent system actually makes decisions and recommendations. Really where design is going to move to is thinking about how the brain, the AI, thinks, and in terms of working with our engineering partners, using methods like service design to track back on seeing something that might be a negative outcome in a system and figuring out where in that overall process of building it that error or that missing piece of information might be housed. So trying to dive into that and understand that.

[00:06:57] Chris: I also think design is going to be more rounded out with other kinds of roles, like ethicists and philosophers and other kinds of roles that aren't typical, because when you're thinking about something that is intelligent, design systems in terms of UX and UI, where buttons go, typographic grids and so on, aren't going to really make much of a difference on those decisions.

Designing for AI when you don't write the code

[00:07:23] Marijke: David, you've recently announced that you've become head of design for Replit, which is developing tools to help people develop code and coordinate for AIs. Congratulations, by the way. You must have some insight as well about how this might change the roles of the developer.

[00:07:48] David: It's interesting, just to build on these points here. Going from Webflow to Replit was like no code to a lot of code, so it feels like a different change. But I think, to what Chris said, a lot's going to change but a lot's going to be the same too, because when I talk to designers they're so nervous about "I don't know how to write Python, how do I design for AI and make a space here?" It's systems thinking. In AI it's a lot of IA and a lot of systems thinking, and once you get a sense of the implications you can start thinking about interactions and manipulation too. I do think we're still in this very early moment of how we design and manipulate applications.

[00:08:38] David: There are two ways to approach it. You can go from tech to then the customer need, or go from the customer and understand what the tech can do. You want to play the best of both worlds too. You don't want to lean too hard in one area. What we've seen at Replit is people building AI applications because the robustness of the dev environments is pretty jarring. It's a place for people to get started and build their ideas. We have a feature called Ghostwriter that helps you write your code as well. All of a sudden something that was unapproachable for someone, like writing Python, they can start building applications using these underlying systems to really understand what these interfaces might look like.

[00:09:34] Marijke: And how do you think, in terms of developers or any member of the product team working alongside an AI that is helping them, how is that going to change? What kind of skills do we need, or what does that change if we're not necessarily needed to be the expert of developing the code itself? This question goes for anyone here.

[00:10:02] David: I can start with my thought. Someone told me, kind of as a joke, this is the time for the generalists, it's their time to really shine because now they have so much leverage. Or if you're like me, people who are just really good at Googling stuff, you feel like you can be really great with ChatGPT. For me this is like the dream I've always wanted, and I think this is what a lot of UXDX really embodies: it doesn't matter if you're a PM, designer, engineer, you're all builders and starting to work towards that. That world becomes more real based on the capabilities you have as a team.

[00:10:37] David: So there's a lot of role shifting. We may be doing different things. There's a world where brand designers might play more of a role as product designer because of having a lot more leverage in this space. Maybe product designers are leaning more towards the engineering side. Maybe it's now true that designers should code, because code compiles for them. There's a lot of shifting, but I'd be curious what the group thinks about some of those implications.

[00:11:08] Chris: I think it's an amazing time for creativity and allowing people access to building things that couldn't do it before. I think the focus then moves from just building things to why are we building things, and are we building the right things? Are these things that help society, are they helping people, and trying to be a bit more mindful about the future of how these things might impact us? I think design needs to move more into that space of considering the ethical implications of things, civil society and human rights, those kinds of things. So it's less about necessarily pixels and more about how the outcomes of these systems might impact people.

How to develop your skill set in this area

[00:12:00] Marijke: Specifically, I'm sure there are a lot of people who are wondering: how do I start to understand some of the terminology, which we've all discussed lacks definition in many ways and is still changing as we speak? How do I start participating in this conversation? Do I need to sign up for an AI/ML course? What are your thoughts about developing your skill set in this area?

[00:12:29] Keyvan: I'm happy to chime in for that. My team actually ran an internal training course that I think was attended by 17,000 people at Google on large language models. It was a non-technical talk and our goal was to educate folks on: hey, what are these, where did this come from? Language models have been around for a while now, but how did this become what it is today? What are the capabilities? What's the myth, what's the fact, what are the limitations? And show them how to access it, because this type of stuff exists now.

[00:13:10] Keyvan: I recommend folks look at some of the resources, whether it's offline or online resources, and develop a bit of intuition for what these models are. In general there are a lot of resources, books, that are non-technical and talk about the different facets of artificial intelligence and machine learning in a conceptual manner, so look for those. The key word that I want you to take away is developing a bit of intuition for the technology, because that's what we do. In my conversations with my team we talk about a new development, a new research paper, and we try to understand what they're trying to do and how to wrap our arms around it.

[00:14:08] David: For me, I can start with what I don't think you should do, and work backwards from there. I'll tell you why. If you play around with Midjourney all day, it's not going to help you improve your skills. It's fun, and I do that a lot, but the point I want to get to is: think about what the capabilities are. What are the heuristics and patterns that you see from playing with Midjourney? Understand what generative AI is and then get a sense of, okay, what are some of these other use cases like copilot, assistive, some of these other things?

[00:14:48] David: And then, in the spirit of dogfooding, look at your own product or your own company that you're working at and see what those use cases are, because I'm pretty confident that these patterns and heuristics are going to be different across the industry. Really have that critical thinking and try to break down what patterns work and what do not. A lot of my work's on the application layer, so I'm kind of biased towards that, but that's my thought.

[00:15:24] Marijke: I'm in here with my UX designer hat on. For me, I think familiarizing myself and doing the research to familiarize myself with key technologies and terminology within this domain, so developing my mental model. Some of the ways that I've done that are by joining, for instance, JPL has a data science reading group, and so we get together and we read articles on data science. It definitely takes me five times longer to read those articles than the other people in that group, who are data scientists, but it also gives me an opportunity to sit down and look at all of this terminology. What do we mean when we're talking about this Monte Carlo simulation, or this large language model? What is the history of that? So that as a designer going in, I have some semblance of what the bounds of this conversation and this technology are capable of. That's been my approach. I'm curious about you, Chris.

[00:16:28] Chris: I think everything that all of you have said is spot on. One of the things I talk about a lot, and we've talked about this, is transparency. Transparency, obviously you can use it as a technical term, like how easy is it to understand how a model makes recommendations, what's the documentation, all that kind of stuff. But there's transparency in being involved in this work and working in teams and being able to just comprehend what is going on. If everybody's not on the same playing field you're not going to get the best out of your team, so being able to make sure that everybody in the room is brought along and has the same understanding, shared values and so on, can really help accelerate this. And if you don't have certain expertise in the room, can you try and articulate those things in a way that everybody at least agrees to the same definition, so that when you're having an important decision-making moment you can all at least agree that you're on the same page?

Bringing ethics into the room

[00:17:27] Marijke: Yes, great, this actually leads to my next question, thank you. Following up on our conversations and your work on responsible AI and the role of ethics now in product teams: when I think about my experience and the experience of my team, I don't know if anyone has a background in ethics. What do we do when we want to invite a conversation like that? Do we hire an ethicist? How do we get started with that? I want to be inclusive of that question to everybody in the room. Not everybody has the same level of resources that a Google or a Meta has, so how do you bring ethics into the room, or ethical leaning thinking into the room, when you don't have those resources?

[00:18:18] Chris: It depends on the scenario, what the product is, who the audience is, how big the audience is, how you need to make those decisions. Ethics and ethical thinking is not a one-size-fits-all, so trying to come up with standardized guidelines for that is not really going to work well. It's very much a scenario-based discussion. But what you can do is try and pull together a diverse group of people with different backgrounds, different levels of education, different skill sets and functions and so on, and have them talk through a framework for values that you can hold yourself accountable to. I was very lucky to have access to the chief ethicist at Meta and worked with her pretty much every day. It was definitely an eye-opening experience.

[00:19:12] Chris: The thing with product design in general is that if you're a product designer, in general you're working on the UX and UI, and the decisions are already being made somewhere further up the food chain in terms of the AI/ML model. So you need to start to learn about those other aspects of what went into why a certain recommendation or an outcome is happening on the screen. Starting to become familiar with those other functions, who makes those decisions, how they make those decisions, is something that is a learning journey for everyone.

[00:19:42] Marijke: I don't know if you guys agree, but I certainly think that there's an opportunity for user research, UX research, to be part of the conversation a little bit earlier in the life cycle of software development for AI, because there are these very big implications for the user.

[00:19:59] Chris: That's right. We used to even bring in third-party outside expertise, thought leaders, and even have other companies host discussions and workshops together so that we could have unbiased discussions in the room. So bringing in lots of outside communities to make decisions where there's something significant that you're working on that might impact a large swath of society.

[00:20:28] Marijke: Oh, exciting, go ahead, Dave.

[00:20:30] David: I was going to say, I think it's the time for user research to shine in this area. The irony of all this is that we're talking a lot about AI and technology, but the most important thing is human factors right now. What's actionable for researchers right now is, so many times researchers will do this 20-page study and no one will read it, and it's frustrating. But there are just so many implications we don't know, and you hear a lot of these even experts in AI talk about how we don't know what's going to happen. Really understanding what human sentiment is and how we design around that is going to be crucial. My hope is that research gets a huge voice in really shaping not just the insights but also the strategy to this too.

Human in the loop, and designing for human ability

[00:21:29] Keyvan: Where I see the real opportunity is thinking about, in a world where the product is powered or complemented with AI, what does a user experience need to be? I don't like situations where we treat AI as this black box that's just going to automate a bunch of stuff on the side. It needs to feed off of the users, and the users provide direction and guidance to it. Having this human in the loop model, which is a term that you'll hear about, is really critical, but that requires a different way of operating.

[00:22:16] Keyvan: If you take an example, you find a process that has 73 steps and you find five of the steps that could be automated with AI. Well, that's not the right way to do this. You may want to take a step back and say, well, we need a new process that doesn't have 73 steps, and an AI could make recommendations along the way and the human is the one that makes the final decisions and interacts with those recommendations along the way. So I think there's an opportunity to really take a step back, and that's where we need the designers and the researchers to really give that input in the room in the first place.

[00:22:53] Marijke: I couldn't agree more. This is a topic that we've all talked about, which is this idea of designing for, I'm going to coin the term, human ability. Designing what the best part of the human and autonomous system or AI system, what the collaboration should be. As you were talking I was thinking about how this technology is changing so quickly that you could come up with a design that could quickly be out of date. However, the thing that is not changing that rapidly are human needs and goals and pain points, so that might be one area where you can nail down the design.

[00:23:39] Marijke: But this does bring me to my next question, which is: in a landscape such as this, where there is so much change happening so rapidly, are there methods that you all are employing, or frameworks for thinking about how to move forward in such a rapidly changing landscape?

[00:24:10] Keyvan: I think obviously staying up to date is a full-time job. We know that. But do we need to consume everything that is coming out? I think the opportunity is to look at how new developments could mitigate risks, make life easier, make the development process and the design process easier. But it doesn't necessarily mean that what you've already done is stale. One of the things that's critical is AI models live off of data. They need that data to be safe today, so as your data changes you do need to retrain, et cetera. But the fundamental design doesn't necessarily need to change, so you don't need to be trigger happy with new developments.

[00:25:01] Keyvan: I think it's important to be aware of what's coming, and especially if there are areas where research shows that maybe there are gaps in things that you've implemented in the past. So how do we do this? Reading groups are a great idea in organizations. Subscribing to condensed blogs, I don't know, tldr.ai or things like that, is always a good way to have a finger on the pulse. And then sharing amongst peers within the organization or outside the organization is always a good approach.

Sharing in the open instead of building in a silo

[00:25:34] David: I was just going to say, it'd be amazing, when we start thinking about building for AI, is there a W3Schools equivalent of it? Are there more open standards that we're talking about? A lot of it's still shaping up right now as we speak, and I think that's why you ask the question. It's so overwhelming to try and keep up with all this stuff. Stuff's probably changing right as we get off stage, and there's some technical breakthrough. I think the thing is just being able to take in as much of it as you can, and then through sharing things outwardly, the shared knowledge is really key.

[00:26:16] David: When we're doing stuff at Replit, we want to share that more outward, get feedback, get thoughts, and even with the design community too as we think about these areas. I think that's the pathway for it, otherwise it's a lot to read and get caught up on.

[00:26:34] Marijke: You mean instead of developing in secret, waiting to release only to find that you're outdated, sharing.

[00:26:42] David: Exactly, yes. And I think that plus being able to share how other companies and other products are thinking about it too. You're converging on an idea together when there is a convergent point, and that kind of helps, so you're not just building in a silo.

[00:27:03] Marijke: That makes sense. Chris, you have thoughts?

[00:27:06] Chris: I have a few thoughts. If you're at a team that's working on zero to one, I recommend that you try and invite as many AI forward-thinking people into the room, even if they're not part of that team, to be there and help guide you in those early brainstorms and ideation sessions. Especially policy and regulatory experts. I found that really effective because they can help creative teams and product teams think in a way that guides them but doesn't quash their creative ambitions.

[00:27:44] Chris: And then if you're in a team that has already got an existing product, I think things like service design and design strategy can help. Obviously ethics, even anthropology, things like that that aren't necessarily what you'd say are typical product design roles, can help as well with unpacking how a product works and figuring out all the steps that went into it and finding those places where things might have gone wrong or could be improved. That's something that all companies need to do, continuously improve their operations. So depending on the scenario it's a different kind of team.

[00:28:23] Chris: The last thing I'll say is trying to lean into the people who are really passionate about this, even if they don't know what they're talking about and they just are really interested in learning about AI. Build that community, have office hours, have open discussions, get them to participate, have them come to conferences like this so that they can start to be advocates in your own culture.

Q&A

[00:28:46] Marijke: Great advice. I want to check the time really quick. I think we're going to stop my questions here and open it up to the audience. Where's that? How does this work? Oh yeah, I don't want to throw that thing. Okay, thank you. So who has got the microphone?

[00:29:14] Audience: Can you hear me? Yes. Eva asks: yesterday I found out about a company that provides synthetic users based on AI to run user testing and interviews to validate ideas faster. I have my opinion on that, but I'd like to know if someone on the panel has some thoughts to share on this.

[00:29:39] Chris: I definitely have thoughts on this. That couldn't be worse. I couldn't imagine a worse idea, because it's bias on top of bias on top of bias. How was that original AI trained to represent those people? That's a terrible idea. You need to go direct to the source and represent your audiences around the world and use them directly. So yeah, I think that's problematic.

[00:30:15] Speaker [?]: I agree.

[00:30:21] Marijke: We have a question over here.

[00:30:32] Audience: Hi. Thanks so much. Chris, I really appreciate your focus on ethics, and I think this is what's weighing on me the most too, and the idea of bringing ethicists into the process. Do you think that that's really going to happen at most places, realistically speaking, or are they going to become like the lawyers, where everyone's like "I guess you have to consult with legal on that," and then they're just going to ignore them and do their own thing? So I guess my question is how realistic do you think it is that ethicists will be a real part of our processes as we're developing this technology, and what do you think will happen if we don't include them?

[00:31:16] Chris: It's a great question. I would say that for any stakeholder, any expert in a product team, it's always a tension of the shipping goals and the metrics versus what you hope to be the ideal solution. So it's not easy, and I'm not going to lie, it's very challenging to get ethical considerations into product roadmaps. How do you do that? You have to start to train everybody to care about ethics, so that it becomes everybody's job to be responsible at a certain level. That's where you start: building those frameworks for what should engineers care about, what should data scientists care about, what should product designers care about, and so on, and make everybody responsible slowly over time.

[00:32:09] Chris: If you don't do that, obviously there's going to be problems in the future with your product. I think trustworthiness will be a huge issue for your company and your brand. It'll be a matter of time before that becomes a public PR nightmare. So it's better to lean in, but there will always be a challenge between measuring that ship goal versus doing the right thing.

[00:32:36] David: Can I add a thought to that too? For me, I have this strong fundamental belief that designers and researchers should get immersed in the stuff that feels kind of gross too, because of these implications. It's just an important thing to understand the work too, and be able to advocate like that, because I hold the same opinion you have. If you have someone talking about ethics at our company, there are going to be bad players out there too, so how do you do that? I think that's why it's so important for designers and researchers, when there is emerging tech, to really understand what that means and have a strong voice for it too.

[00:33:16] Chris: Can I just add one more thing? Most product teams are used to designing for the core use case. Everybody's familiar with that and they understand why the product feature is going to deliver against that. If you look at the edge cases, sometimes at either end, that's where people haven't thought about, oh, the product could go awry in this way, or it could be used in a less awesome way. So grounding the team on a frequent basis in what those edge cases could be and how those might deliver harm is actually a good way to keep grounding people on: we're doing the right things going after the big use cases, but we also need to make sure that we start to bring these other use cases in. The only way to do this is to build empathy for those edge cases where people unfortunately might not be well represented in the group.

[00:34:12] Marijke: I don't remember if it's a conversation I had with one of you or all the reading I've done in the past week, but in terms of testing what you're coming up with, or the products that you're developing, I was reading something, or it came out of conversations with one of these guys, about thinking through worst case scenarios. I think that's an excellent idea. Where could this go terribly wrong? Bring in all the dystopian thinking.

[00:34:41] Chris: A tactic we use is called red teaming. Blue team is, let's think about the blue sky, happy path, everything is amazing. The red teams take what that product is on paper. Red teams are usually engineers who try and attack a system, penetration tests and things like that, but red teams in design can be: how could this go horribly wrong? How could this AI deviate from its intended purpose and hurt people in a different way? How could it learn something new or be saturated with data that might change its outcomes?

[00:35:18] Marijke: It's like Jeff Goldblum's character in Jurassic Park, right?

[00:35:23] Chris: Exactly, yes. And it's actually a really fun and creative process. Oftentimes what happens is that you'll think through those things and then actually come up with new features as well, or it'll at least help you prioritize the features that were in the original roadmap, because some of them might be weighted more to problematic issues.

[00:35:47] Marijke: Thank you.

[00:35:51] Audience: The question I have here is aligned to this topic about ethics. I've been listening a lot as well about how maybe regulations and laws might be adding to this topic in terms of setting the ground rules, and also guardrails to avoid these kinds of issues. What are your thoughts on that topic?

[00:36:20] Keyvan: Sure. Obviously most of us have heard about the Senate session yesterday, and the EU regulators are also thinking about this. I think some regulation is necessary to make sure that the technology is not abused and it's not misused. I think that will help. I don't necessarily think that that's a replacement for taking it upon yourself to develop things responsibly and bring that ethic and responsible AI mindset to your product development and design as well. But inevitably the world is waking up to the possibility that, hey, we've got to live with more and more of this technology and we need to have some safeguards.

[00:37:19] Marijke: Brilliant. I think that actually brings us to time, so if everybody could give a round of applause for our panel.