Evolving with User-Centered Design Practices at Bloomberg

Ash Brown at UXDX USA. Video: https://youtu.be/ZSrlIGGYthg

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

The Bloomberg product suite

[00:00:14] Host: Excellent. Thank you very much, Ash.

[00:00:16] Ash: Thank you very much for having me, and thank you everybody for being here. Hope we have a good time.

[00:00:19] Host: Excellent. Global head of user experience for Bloomberg. For those in the audience who might have heard of Bloomberg, they might know the brand name, but can you give a bit of context around the product suite? What exactly is the full landscape at Bloomberg?

[00:00:34] Ash: For everything that my team covers, it is the Bloomberg professional application. If you watch Billions or have seen Wolf of Wall Street, you will see the famous Bloomberg terminal screens. That is our primary product. But we also work on our mobile application. We work on different web products, and we work on internal tools, so engineering applications.

[00:01:02] Host: Just on those internal tools: most of those web and mobile, are those end client facing, or would those be more traditionally internal facing?

[00:01:11] Ash: The mobile is completely client facing. We also use it internally, and this is something at Bloomberg, we use a lot of the tools for clients for our own purposes. For example, all of our email we use is on the terminal, the same thing that is available for our clients. Web is also client facing, but then we'll reuse the technologies and the design system that we have for web for our internal purposes.

[00:01:35] Host: Excellent. So dogfooding your own.

[00:01:37] Ash: Yes, exactly.

Why Bloomberg started investing in UX

[00:01:40] Host: Brilliant. We want to talk about the UX journey in Bloomberg. What I always hear when I'm talking to people, and I'm sure there are people out there who might appreciate this as well, is when you have those successful, really large companies, their current products are doing great. Why rock the boat? Why not keep going with the traditional way of working? Why introduce something like UX? Can you give us a bit of context around when did Bloomberg start to make that shift, and how did that look?

[00:02:11] Ash: I was not there for the inception. I have been at Bloomberg for 10 years, so I joined in 2015. Really the transformation started in 2009. I don't have the full story, but the lore that is passed down is a senior leader looking out at the entire application, and maybe looking at the world and seeing quality changing. This is B2B software, maybe a little bit later to the UX and the design game than consumer facing, and said, what can we do? Got a team together that then was really just focused on, all right, if this is our terminal product suite, how are we going to transform it? And they really just focused on what that next generation was going to be.

[00:02:50] Ash: When I joined in 2015, we were in a very different place. We had started to make the move from being very focused on the terminal product environment to now working with the teams that published, we call them functions but you can really think of applications, to this environment. The team was originally focused on some things that were really widely used. Monitors, which were worksheets, which is where users can configure all of their data. That was being completely revamped, a technology migration and then a user experience migration. And then outside of that we started working in areas where people were interested, maybe there was a new market, maybe there was a recognition that the quality could improve.

[00:03:43] Host: Could you give a bit of context around the time? As you said, it was before your time when this was being supported by some of the senior executives. You joined 10 years ago. Was it 10 years before that? Five years?

[00:03:55] Ash: It was six years, I think. 2009.

[00:03:57] Host: 2009. And where do you think you were at at the point that you arrived?

[00:04:02] Ash: I think we were at the point of, we had some senior leaders who were really bought in, and then as we were trying to get into the culture, a bunch of people who were like, what is this? What is this UX thing? What do you mean, user research? This feels like it's going to slow us down. Are you just the team that's going to take my ideas and make them pretty? So very at the early stage of product and engineering understanding what this is, what we could deliver, and how we were all going to work differently together to increase the quality of the products.

The scale of the team, then and now

[00:04:37] Host: And then on a scale perspective, can you give people a bit of context of the scale of Bloomberg as a whole, and then let's talk about 2015 when you started. What would have been the UX, or whatever it was called, it might not have been called UX 10 years ago. Can you give people just an idea of the scale? Our current scale, so how many designers we have versus everybody else.

[00:05:00] Ash: Our entire team is 165 actually, I think now, but somewhere north of 150. And we have a couple of different roles. We've got product documentation actually within our team, and then designers, researchers, visual designers. That team compares to, I don't even know how many, but let's say more than 1,200 engineers and product managers. Now, not all of them build anything that is client facing, but that just gives you a very rough scale.

[00:05:36] Host: Okay. So one to 10.

[00:05:40] Ash: Nine.

[00:05:41] Host: Even higher. Okay.

[00:05:43] Host: And what was it in 2015? Because what I'm trying to understand is your journey. When you joined 10 years ago, how far along on that transition? Because I'm going to start asking you about the challenges you faced.

[00:05:52] Ash: When I joined, I don't think we were really more than 35. We could all fit in one room for the entire team design crits at the time. But I don't think the entire company has scaled as much as we've scaled in those 10 years.

The challenges: a shared language, and speed

[00:06:08] Host: Excellent. So there's obviously a buy-in that this is important and it's valuable. What were the challenges though? You mentioned one already, which is that hesitancy of people. Was that the biggest challenge that you faced, and has your challenge evolved over that last decade?

[00:06:27] Ash: I remember, I joined Bloomberg as an interaction designer, and I think my first project was redoing fixed income trading tickets. I remember sitting with the product manager and they were very much like, okay, cool, I get this is going to help me. So it wasn't a buy-in thing, but it was just a process thing. I would be asking questions that any designer would understand. Who's the user? What are they trying to do? Let's map out the workflow, what are their use cases? And that was just anathema. I think that was the biggest initial hurdle, just the I want to work with you, but I don't speak the same language yet. That just took a lot of patience and a lot of adapting.

[00:07:12] Ash: I think the second thing was probably just speed. You introduce something new, that's changed. People are going to have to take the time to figure out how to do things differently. And there is the, hey, if I have an idea and I can draw it, the engineer can build it, we can release it really quickly. Now, if I'm stopping, I'm asking and answering a lot of questions, I'm going through an iterative process, that's slowing things down. Initially that was really seen as a burden, until we had enough successes where it's, yeah, sure, it took two months more, but the result was the clients were a lot happier and the quality of the product was much higher.

[00:07:50] Host: And save time on rework.

[00:07:52] Ash: Yes, exactly.

Winning hearts and minds one by one

[00:07:54] Host: I'd love to talk about that process, because I'm sure, could we see a show of hands of how many people are facing that problem with process, that you're trying to do something but you're getting pushback from your organization because it doesn't quite fit with the flow? So we're seeing a lot of hands. How did you tackle that?

[00:08:16] Ash: I think being really adaptive and meeting people where they were. We definitely had some instances where we had the process in mind and we were like, this is it, everybody, this is the process we're going to follow, and then we will get to the perfect outcome. But if nobody's buying that, then you're not getting the engagement, you don't have any opportunity to deliver any value. And so it was scaling it back and going, kind of like the design journey, let's not build the bicycle first, let's build the small thing.

[00:08:52] Host: Yeah, let's go.

[00:08:52] Ash: Exactly. Let's build the cupcake before we get to the big cake. So it was like, okay, fine, we're not going to get to the bike. What's the cupcake that we can deliver?

[00:09:01] Ash: I remember being so passionate about this and taking so many opportunities. I remember a product manager, still in the trading space. He was just in the hallway, complaining about something. He was like, yeah, the client said that this didn't work for them. And we took a look at it. I wasn't even working on the project, but I just went, oh, I see how it could be better. And so I took a couple of hours and I just mocked up what could be better, and took the time to go back to him and say, hey, these are the changes that we can make, this is why I think it's going to be so much better. And taking any opportunity like that. And then I think you get the buy-in. Once somebody sees, okay, I can see what the value can be, now I can put in the time with a little bit more confidence.

[00:09:47] Host: So it was really winning hearts and minds one by one around the organization, and then it would snowball itself? Or how would you describe it?

[00:09:56] Ash: No, I think a good point. Definitely a lot of on-the-ground work, one by one, and then starting to amplify those successes. One of the things I focused on when I became head of UX, this was in January of 2022, was really making us more visible, and supporting some initiatives that were already going on, to get product and engineering to step up and say, hey, we worked in a different way and it was really, really good. So having that advocacy come from the people that we had success with, so that we were not the only ones going, hey, we're really trying to prove to you, we worked with these people over here. It was those people over there telling their colleagues that they should be working in this new way.

[00:10:42] Host: Excellent. The last one on the process, because what I found is maybe an organization might have a project management office or something like that, where you have very strict written-down processes. So while you might be winning the hearts and minds, I know in my past experience we've had to skirt some of the processes in order to do things that end up well for the company. Did you have that problem? And if you did, did you convince them to change, or was it more just working around the edges?

[00:11:11] Ash: I don't think we really had that problem, actually. And this is where everybody who is trying to achieve something is going to be so influenced by the context and the environment. If anything, Bloomberg is a little bit like process, process, a little bit lighter. So I think we were maybe the ones that were introducing that, and people were like, wait a minute, what do you mean I have to take this number of weeks and go out and talk to users first, and then we're going to have this other round? That's too many steps for me. But again, over time, people just go, all right, this really paid off.

How the team is structured

[00:11:47] Host: Great. I'd like to jump forward now to where we are today. Can you explain the structure, or maybe even the evolution of the structure if it has changed over time, of how you design and structure your team?

[00:12:03] Ash: Some of the structure has not changed. When I joined we had teams that were dedicated to particular business areas, and this has been really critical in the company because of the domain knowledge that you need. And that was another thing, actually, just like winning hearts and minds. Getting designers and team members who were passionate about, yeah, I'm going to really learn about bond pricing and options pricing and understand this entire thing. Because it's not only really critical for when we go out to users and we're interviewing them, but also for showing our partners that we're really in the game. We're not just here doing this level, we're here completely with the team. So we always have that and we still have that. We have interaction designers that are in teams that are dedicated to different areas.

Research: project, embedded, and back to centralized

[00:12:54] Ash: One of the things that's gone on a back and forth evolution though was research. When I joined we had a centralized research team and they were working on a project model. So they would, whatever the top priority project would be, they would go and they would work on that. And the feedback was kind of, all right, this is okay, we're covering maybe the top priority stuff, but that domain knowledge was just too hard to acquire every single time that they were working on a project. So then we did embedded, and when I was a team lead I had one researcher for the area that I covered. But recently went back to centralized with domain specificity. So we've got researchers on one team, but they will only take projects from a specific area.

[00:13:36] Ash: I really wanted to do that because we've reached a point of maturity with product and engineering being a huge part of our research. I'm sure like many, in Covid we had a big shift from visiting clients in person to now being on Zoom. And what that meant was we can invite everybody. You're not limited by, you can only have three people on a trading floor looking at somebody's screen. So there's been a really huge swell of engagement from all of our partners, and that has meant that they are seeing the value of being able to ask questions in the chat on the side. Hey researcher, I'm really interested in why did they do this, or did they see that thing? And that has meant they are more eager for that research knowledge. So I saw the opportunity for a more centralized team to really think about how their impact can be richer and deeper, and how they can partner with business leaders to forecast a little bit what are the research questions that we're going to have over the next year, maybe not just for this particular project, this particular product.

[00:14:42] Host: And can I ask what was the problem? You said you went project, to embedded, back to centralized. That's quite a trend in the industry at the moment, the distributed research. What was the problem you were facing with that model that made you decide to go back to a centralized with domains model?

[00:15:00] Ash: I'm not even sure it was necessarily, well, I think the problem was the power of the team being able to come together. We had communities of practice, and we have multiple of those for all of our roles within UX. But I think there's something to having just one person accountable for pushing forward research. And me as head of UX, wearing many different hats, that was not going to be possible. And so making one person accountable as a team lead, and getting that entire team to have specific objectives as a team for research only, versus let's say advancing a particular product area, was what I thought would be really effective. So far, so good. It's been about a year.

[00:15:52] Host: Okay. So to summarize or recap my understanding, is that there's the potential risk that if people are isolated in different teams that they mightn't have the support that you would have together working as a team, and the quality therefore might be impacted.

[00:16:04] Ash: Support, I think just focus. If you're on a team, so let's say we had one researcher doing all of our fixed income products, or fixed income trading, well, that's where they're going to really be focused, and they're going to be solving the problems that are specific to that team. And maybe they're a lower maturity team in terms of product and engineering being engaged, maybe they're a higher maturity team. But pulling everybody together helps that team then see, okay, across the entire company.

Is a centralized model a victim of its own success?

[00:16:32] Host: Okay, I'm going to go with a bit of a devil's advocate question now, because particularly in the DevOps world, there's the DevOps Research Association. They released a study and they showed in DevOps that having a centralized model, they call it an anti-pattern. And the reason they do, their explanation is you become a victim of your own success, because people go, oh, this is great, I want your services. You get overburdened with too many requests and then you're slow and unresponsive. So that was their rationale. Is that a problem that you're facing? And if not, how are you dealing with that kind of victim of your own success situation?

[00:17:09] Ash: Such a good question. And definitely something we've faced a lot, which is, it's a good one to have. You're doing really great work with teams, whether it's a centralized research team or centralized design team, whatever it is, and then everybody just wants to come to you. And so we have done an enormous amount of work on what are our prioritization points, prioritization metrics. And a big part of that has been making sure the leaders in the team are really clear on that, and everybody feels comfortable saying no. And we've got them really clear processes for that intake, and communicating out, we are so sorry, we would love to work with you, but we can't, in part because we've already said yes to something else.

[00:17:55] Ash: That took a lot of reiteration and a lot of coaching, I think, for team members to not feel bad about saying no. There's always so many opportunities, which is a fantastic problem to have. But the concern in our journey was, well, we put in all of this effort to convince people to want to work with us, and now they want to work with us and we're saying no. Is this going to damage our reputation? Is this going to hurt us? And my position was no. They're just going to want to come back later, after we do a fantastic job with the people we have said yes to, after we've been able to be really focused and show the impact and the results.

[00:18:36] Host: Excellent. So you found that even though you're saying no, it doesn't result in resentment or any kind of ill will towards the team.

[00:18:42] Ash: I don't think so. And I tried to really phrase it as, there's always so many things that everybody wants to do. People understand, okay, you can't do everything.

How the team decides what to say yes to

[00:18:55] Host: But the next question that comes to my mind is, if you are saying yes to some things and no to other things, how do you decide which ones you're going to accept and which ones you're not?

[00:19:07] Ash: First one is what is going to be the business impact. There are so many different products that we work on, and so much that we would love to evolve, and you always see so many opportunities for things to get better. But you have to be practical about, well, what is the business currently investing in? So that would be number one.

[00:19:28] Ash: And then second would be, how effective do we think we're going to be with this team? We're small. We don't have an embedded model with absolutely every single product team: product, engineering, UX. Some are just product and engineering. And so if we're coming into a space where we're starting to work with a group maybe for the first time, and we're comparing that to another group. So let's say two groups. If we worked with one already and we know they're at a certain level of maturity, they know how to engage with us, they're really going to be on board, that might win out over the team where maybe we've tried in the past, we don't have the buy-in, we're not sure we're going to be as successful. And the really big one being, are we going to be able to deliver the most impact?

[00:20:10] Ash: And then I think the third one is, where are they in their life cycle? We know we can have the biggest impact if we're coming in as close to the beginning as possible, just the start of that double diamond, in terms of discovery. And so if it's a project where they're almost at execution phase, versus a project where they're at the beginning, we know we can have more impact if we start at the beginning.

[00:20:34] Host: Is there a chicken and egg problem there, in that, just on the particular point of, you've worked with somebody, you know where they are in their life cycle, but they can't mature until they get working with you. Do you face that problem of, how do you start that chicken and egg, or solve that chicken and egg problem?

[00:20:54] Ash: That is a good one. I think probably the business impact one is going to be the main driver. So even if we've got a team that's less mature, we haven't worked with them as much, or we haven't been as successful, if they are the top business, we're just going to definitely work with them. I think we've then built up enough of a library of examples, supporting materials, coaching materials for teams. They've got enough of a network where then we can go, all right, we're going to make this work. What are the tools? What is our approach going to be in order to be successful, or as successful as we can be?

The people side: changing habits and hiring in

[00:21:38] Host: Awesome. I wanted to look at it from a person side now, because we've been talking very much on the process side and structure and things like that. And I can see in a lot of companies, I'm sure, because we had a lot of hands here earlier when we talked about it, you're going to have in my mind two sides of the challenge. You've got the people who are used to working in a particular way, so this is brand new to them, and you're advocating change, which is always hard. And then you've got your new staff that you're probably coming in and you're hiring because they have all of this experience, and they have to get used to the culture of that company. How do you deal with that? And is that a challenge for you?

[00:22:15] Ash: Definitely a challenge. And so I think on the people who we are asking to change, product, engineering partners, maybe even people outside, I think there's a matter of maybe me or leaders reaching out and engaging to begin with and setting expectations. Hey, we're going to be working with you, we're going to be able to get this or that, but it's going to require you to work in a different way. Then I think when they start working with the designers who are just expecting these things, you don't have that, wait a minute, I thought I was going to get this, I'm not getting this, this is not working at all, and now we're too slow.

[00:22:51] Ash: I think for designers coming in, it's being really honest about where we are as a company, how we work, and then what the opportunities are as a result. Bloomberg, I think, has a fantastic culture of a lot of autonomy, and a lot of spirit of, it doesn't really matter your title. If there is something that you see needs to get done, just do it. And so I really try to emphasize that. You're a designer, maybe you're used to, you get exactly this from product and this is exactly how you work with engineering. This is a little bit more open, but that means there's a lot more opportunity to shape it, and for you to grow your skills across a wider range of maybe project management, product ownership, technical skills, because there's just more opportunity to expand and fill whatever that particular moment needs.

[00:23:37] Host: And do you explicitly look at it? There are terms that I've come across in the industry of like a single focus, or more pie-shaped, more comb-shaped, where you're not expecting people to be very pigeonholed as a designer. You also need some research, you also need to understand a bit of how dev works, so that you can be that adaptable person.

[00:23:57] Ash: Yes, definitely. And research actually is one of the things we absolutely expect from all designers. Interaction designers, visual designers. We think that it's extremely valuable for getting to know clients, obviously. But then it really helps, I think it's another layer of championing what we do, that it is not design for design's sake. It is design because we're trying to solve a real user problem, and we are being very focused on, with what we understand now, this is what we think is going to be the best thing to take us to the next level.

What Ash is excited about

[00:24:31] Host: Brilliant. I'm just conscious we have time for one last question, then we're going to open it up to the audience. So I'm just going to ask you, what are you excited about for the future, and particularly with the industry and the state it's in right now and everything going on?

[00:24:43] Ash: I am really excited, and I know there's mixed concerns on this one, but for new tools and AI to really help us scale our impact and make everything that we do more accessible. I think particularly on the research side. That's where we're focusing a lot of our experimentation with the tools, and we're still in very early stages, so TBD exactly what the impact is going to be. But I've had a lot of product managers and engineers approach me and go, can you just hook me up to your research knowledge base and I can just query it and find things out? And that's obviously just an expression of intention or a desire, and we'll see how it plays out. But I want to capture that and go, yes, we are going to be able to do something. And then once we've got that information accessible to everybody, I think we can all be asking questions together as a team, and reasoning together, and making better products.

Q&A

[00:25:43] Host: Excellent. Well, thank you very much. That's concluded, I guess, our fireside, but let's put up some of the questions from the audience, because do you have any questions that you'd love to put to Ash? And we do have some here already. So I'll ask you the first one. What are the tools and materials that you provide for teams who have not worked with you before and need to be onboarded to the UX process?

[00:26:08] Ash: We have provided a number of different things over the years. One of them was just a, how do you work with us? This is who we are. These are our roles. This is when you should engage. Really just a workflow of the entire product development life cycle and what you can expect from us. We've got a couple of different roles. I mentioned interaction designers, visual designers, researchers. We also have a UX QA that can work with teams when they're more at the execution phase to make sure that they're doing it correctly. So materials like that would be number one.

[00:26:47] Ash: We've also invested in developing trainings for teams. So these are now videos that they can go ahead and watch. And we have UX tech talks that we do every single month. I do a lot of presentations. So we've tried to build up a library of materials that answer team members' questions, so that if a designer is working with a product manager for the first time, they can just be like, oh, here's the intro guide.

[00:27:17] Host: Awesome. And are those trainings on how to work with us, or is it more of that decentralization, how to self-service for yourself?

[00:27:25] Ash: More how to work with us. We have really focused on, the biggest value we can deliver is when we go for a full embedded model. We used to try to spread our influence and try to affect change that way, with a lot more coaching or consulting. And I really didn't think that that was paying off for us. It led to a lot of confusion. We would hear from people like, oh yeah, UX signed off on this, or yes, I did work with UX on this, when in fact they spent an hour with us chatting through what might be better. So for right now we are not focusing on self-service.

[00:28:05] Audience: Can you talk more about the documentation team? I'm curious about that.

[00:28:10] Ash: Sure. Our product documentation team, they produce all of the help content that you can see within primarily the terminal product, but also some of the web products and expanding into mobile. And they are also the product managers for our user assistance tools. So what's-new tools, help postcards, different applications where our users can see what are the new things that are being published. And in enterprise this is extremely common. You've got new releases, and our users really want to understand what is the newest thing, what is the latest thing, or if there's been a change, how is it going to affect me? So all of the content that they produce for that is really critical for feeding that pipeline to answer those questions for our users. It also plays a role in ensuring that our support and our sales also have these answers.

[00:29:02] Host: And are they a centralized team? So all of these teams who are building the features will feed into them, going, by the way, we're just about to launch this.

[00:29:10] Ash: Yeah. So they are a critical part of our go to market process.

[00:29:14] Host: Brilliant. You mentioned actually with people saying that UX had signed off, so I think this question might have come from this one. As head of design, do you personally have oversight and sign off for all designs before they go forward into product?

[00:29:30] Ash: We do not have a formal signoff process. I think a big thing at Bloomberg is wanting to deliver as much value as quickly as possible. And given that we are a very small team, there is still a lot that goes out that we don't work on at all. So number one, that would just completely be not feasible. I think what we've done is focus though on that prioritization element. What are the products that we are building that are highest priority for our business, highest priority for our user, and maybe need more oversight? We're trying out a new visual design, we're doing a new interaction that we've never done before, it's a completely new market and we're all learning about a domain together. That's where we'll get a lot more attention. But there's not an official document somewhere with my signature on it.

[00:30:20] Host: But for the ones that your teams do themselves, so as you said, you're not touching everything, but do you have a formal, or how do you do peer reviews within your team?

[00:30:29] Ash: So within let's say the domain specific teams, I think I mentioned previously there's a team that's just focused on designing for our buy side products, they will review together. So we have a big culture of design critiques, and so any design that somebody is working on will see weekly reviews by the entire team. In that way we get a lot of really good feedback from people who do understand the domain, and who maybe understand, okay, you're working on this thing here, but I see how it's going to impact a workflow step 10 steps later. And then the team lead ultimately has the final design direction.

[00:31:12] Host: Brilliant. Oh, the eternal question. How does your team measure project success?

[00:31:19] Ash: So many different ways. But we've really focused on, and I don't know if this question is implying it, but veering away from measuring UX's success on a project, towards the team should have very clear success metrics for that project or for that product. And we've done a lot of just internal education for ourselves and then built that into our processes. So we'll have a design template where page one is, why are we doing this? What do we think the value is going to be? And how do we know we're going to be successful? Going down to designers even understanding, well, what are the metrics that we are going to use? Do we have instrumentation for this or not? So that's really been where we've focused. And then it is a matter of, fantastic, the entire team came together, they established the success metrics, product, engineering, UX were bought into that. Along the way it was, are we building something that we think is going to get us there? And then of course just measurement after release to see if those were achieved.

[00:32:26] Host: So very much on the outcomes, not just building the product.

[00:32:29] Ash: Yes.

[00:32:31] Host: Excellent. That aligns perfectly here. Okay, we have time for one last question. Sounds like there's been a decent level up in the UX maturity of Bloomberg. What are some strategies you would recommend to do the same elsewhere?

[00:32:45] Ash: Oh, such a good question. And I think you brought this up earlier, we've really had an evolution. And even in the three and a half years now that I've been head of the department, I've really changed a lot of my strategies. So I think number one would be adapt. Just figure out where everybody is and make a strategy to just get to the next level, that next increment.

[00:33:11] Ash: And so thinking back to my first months in this role, a big thing that was still a problem was people not really understanding who we were and what we did. So I really doubled down on, we're going to create a lot of materials to explain that. We're going to have a lot of high profile events to be really visible. One of them that we created was UX week, where we invite all of our stakeholders to different sessions. The UX tech talks. So that was the strategy that I used at the time to make sure everybody had that baseline understanding. Now that's just established and we're seeing lots of other benefits, but I'm shifting my focus to, okay, we've taken what UX can control and do, well, not as far as it can go, but very far. And to get to the next level, I want product and engineering to be thinking about how do they need to change now. So engaging with senior leaders to get them thinking about how they can be contributing to the quality of the product.

[00:34:12] Host: Brilliant. Well, thank you very much, Ash.

[00:34:16] Ash: Thank you. Thank you everybody.