Efficient Teams Do Not Happen. They are Designed. It's called DesignOps

Patrizia Bertini at UXDX EMEA. Video: https://youtu.be/PIjTg0ku-VY

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and defining design operations

[00:00:00] Hi everyone. Thank you so much for having me today. I'm absolutely excited to be here and to be able to share my experience with you. I'm Patrizia Bertini, Pat. I work at a company called Babylon Health. I'm an Associate Director of Design Operations, which means I care for my UX team, I care about processes. If I had to define myself in three words, I'm an extremely curious person.

[00:00:33] I definitely have a non-linear but extremely fun career. I started with accessibility and then I moved into research, and then I moved into privacy, and then I moved into research, and now design operations. And this is because I love experimenting.

[00:00:55] Babylon Health is a fast-growing company and we are hiring a lot of people, both in the UK and in the US. So just to let you know, we'd be keen to talk to you if you are interested in a new job.

[00:01:11] But let's get into the topic of today. I would love to ask you to raise your hands if you know what Design Ops is. I can't, so I just assume that we all need to start from a common ground and to make sure that we all speak the same language. So let's try to understand, and let's define what design is.

[00:01:36] For the sake of time, and just because we need to move to the operations part of it, let's just assume that we all agree on the concept that design is not just what it looks like and feels like, but it's how it works. So it's much more a functional aspect. It's problem solving.

[00:02:00] If you agree on what design is, the question is, what is operations? Now operations have a lot of definitions, but the definition that I like the most is: operations transform the existence, the resources, the data, the context into a better design output and result, or outcome. And why do operations operate this transformation? To create value for the customer.

The three customers of design operations

[00:02:36] So if we start looking and thinking about design operations, we see that it's about transforming how design is, works, operates, to create value for the customers. But who are the customers? When we talk about design operations, the customers are not one, not two, but three, because every act of design operations is a balancing act to serve, empower and enable three key stakeholders within your organisation.

[00:03:11] One, of course, is the design team. Every design team, which includes content managers, content designers, researchers and design operations folks, design system people, designers, product designers, UI designers, you name it. Those are clearly the ones that design operations focuses on most, because we want to make sure they are enabled to do a great job, they have everything they need and they can do the best job they can, because they are empowered by the organisation and by the design operations team.

[00:03:46] The second stakeholder is the design leader. We operate with design leaders, and design operations work with them in conjunction. The big difference between design operations and design leaders is that the design leader is the person that has the vision and the strategy for the product and the service that you are designing. Design operations is way more focused on ensuring and making sure that this vision, this strategy that the design director has in mind and envisions with the product teams, can come to life.

[00:04:31] And then the third stakeholder is the business, because there's an increased need to prove the business value of design to the business. Design operations are exactly the function that actually connects the design and the business, and makes the most of the business resources to create the biggest opportunities for design to deliver the biggest value, the biggest outcome, the highest quality of product and experiences that they can. So everything in design operations is to try to serve and create efficiencies and create value for all these three stakeholders.

The three levels of design operations work

[00:05:19] But actually, what is it that design operations does? That's another triangle for you. There are many ways to look at design operations. This is one way. It's not the better, it's not the only one, it's my way. Design operations operate and work on three levels.

[00:05:44] As I was just saying, we need to prove the value of design to the business, so there's a part that is very, very strictly business operations. That's everything that has to do with business budget management, spending optimisation, overviewing and reviewing the policies. We're managing the procurement, contracting and negotiating agreements with vendors, onboarding and offboarding vendors, making sure the IT infrastructure is working and everything is assigned, managing licenses, assessing and calculating the return on investment of every tool, sequencing and deciding if we need contractors or full-time, managing all these aspects that are kind of basic around the business, but it's where a lot of efficiency happens.

[00:06:44] Then the second aspect is more about the workflow and the ways of working and how design operates. So it's literally end-to-end design processes. It's all about looking at how these tools are actually flowing within the ways of working, how they are integrated in the processes. How is the design team working with the cross-functional teams? How is the collaboration with engineers, with product, with marketing going? Do we have the right processes? Automations? Do we have the right data governance?

[00:07:24] Are we helping the teams with the right design system? Who is managing the design system? Are we quantifying the value of the design system? What are the design standards, the playbook, asset management and knowledge management? So the second big pillar is everything that actually has to do with hands-on design, and design as a collaborative, participative discipline that works with critical stakeholders and cross-functional partners.

[00:07:59] And then there's the fun bit, the people operations, which includes career progression as key metrics, assessing how we can help people to grow, assessing what the team culture is, how we can share more experiences, how we can grow, how we can onboard people and offboard people when they joined the company, how we can help the team to be happier. Because at the end of the day, design operations does all this to make sure that designers don't work long hours, they are efficient, and they can deliver great work and great value to the business and to their leaders.

Efficiency and efficacy

[00:08:41] So we have seen the design pillars. Let's get a little bit more into detail, because the title of my presentation is about tangible benefits and outcomes. So how can these be tangible? DesignOps can have a massive impact on you, your organisation and your team, and there are two elements that actually determine the impact of your design operations practice: efficiency and efficacy. Now they may sound the same, but they're slightly different, and that nuance is critical.

[00:09:20] So efficacy is all about doing the right things. The focus of efficacy is about behaviors. We want our design teams to do the right things, so do research, and make sure that we test hypotheses in a lean environment and we get proof and we know how to progress and we have a lot of ideas to test. When we talk about efficacy we start talking about metrics like empathy, user engagement, ideation and experimentation cycle times, the composition of the teams, having the right behavior, the learnings, the skills, the distribution, and designers' satisfaction and retention.

[00:10:10] When we talk about efficiency, though, we do talk about doing things in the right way to deliver a product. There are a lot of weights, there are a lot of tools, there are a lot of processes, iterations that can happen. So efficiency is very process focused, and it's all about measuring the return on investment of the tools. It's about measuring the time to deliver, it's about measuring the lead time, the productivity, the end-to-end time, the quality of the outcome, the number of iterations, the percentage of rework.

[00:10:57] So DesignOps really looks at these two elements because they can influence both. And it's important to note that efficiency is not a qualitative and subjective dimension. You don't measure efficiency by saying, hey, we did great. Great can't be quantified. Efficiency is always quantifiable and it always has objective measurements.

[00:11:29] So when you think about efficiency, about processes, you need to say, what are you gaining? Are you gaining quality of the outcome? Are you gaining speed to delivery? Or are you gaining an increase in the return on investment in the resources that you have? If you can't quantify, that means that you don't understand the problem properly yet.

The invisible inefficiencies

[00:11:57] Today what we know from a lot of research is that teams' efficiency is way under the optimal stage. There's this research from [?] from last year, and apparently 60% of knowledge workers' time is spent doing something that is not related to design. It's quite a lot. And on top of that, another research from the same, from last year, estimated that the productivity time is less than three hours.

[00:12:42] So that's really something. Why actually are knowledge workers, designers, not able to be productive more than three hours a day, more than 40% of their time? What is preventing them? What are the issues? There are a lot of invisible, endemic inefficiencies that have tangible effects. We may not notice, and we may even consider these things as part of the overall process, but generally we may start looking at the end-to-end ways of working, or the cross-functional collaboration is not optimized.

[00:13:21] Perhaps in the team there are people using different tools, because hey, I'm coming from a company and I really like X, Y, Z and I can't live without it. That creates delays in the process, in integration. Perhaps there's no clear way to get templates, or the templates have been modified so many times that they're inconsistent. There is a poor engagement model. The briefing process isn't efficient, or the collaboration is broken, or the QA is broken.

[00:13:54] But actually all these things — and I'm pretty sure I mentioned at least one thing you all experienced — will have consequences, like long working hours, because hey, if something is broken you will have more meetings, you will have rework to do. Delays, poor quality of the outcome, high team churn and low level of engagement. So it's all there. We have all this.

Case study one: fixing research recruitment

[00:14:25] So let me take you through a couple of case studies to tell you how you can do this, and hopefully this will inspire you to start tackling efficiencies and efficacy in your teams. Let's start with a story about improving designers' experience, improving efficiency. Remember, efficiency is all about processes.

[00:14:48] I don't know if it ever happened to you, but in my teams one of the things that always comes up as a problem is too much work, long hours, I can't finish everything, no one understands how busy I am, I'm so busy, I can't do everything, and blah, blah, blah. The thing is that a lot of the problems that people tell you, those are symptoms. They're not really the problem and the real cause of it.

[00:15:30] Because the real problem, at least in my case, was, why were people working long hours? Simply because a process was broken. In this case it was the recruitment process. So if you think about designers having to do iterative testing with users and customers every two weeks just to adapt to the sprints, we realised — and I measured — that actually to get five to six users every two weeks cost, in terms of time, 2.5 days of designers' time, which means every month there was the equivalent of roughly one week's worth of designer's work spent in non-design time. You can start seeing where these 40% or 60% of inefficiency is going.

[00:16:25] So that, of course, had causes, had the consequences of long hours, had consequences of people feeling overwhelmed. Actually, this is not the job of a designer, to recruit someone. This inefficiency had consequences both on the business and on the design team, because it was long working hours. Perhaps this caused people to not have so much time to analyse the results from the testing, perhaps they reduced the testing or the number of participants or the quality of the participants.

[00:17:06] And of course, if you look from a business point of view, what we were seeing was basically we were wasting the equivalent of 1.5 full-time employees, of 1.5 headcounts, so 350 working days. We also had impact on costs, because we were using agencies and the cost was too high, anywhere between $100 to $360 per participant. Because of that, of course, you go fast, but you can't really go fast, because the lead time with some agencies was up to 25 days. And of course this caused that we really didn't test as much[?], the number of tests. In some cases there may have been some creative and original ways of recruiting that were not necessarily GDPR compliant.

[00:18:05] This is what we actually mapped out. So we may have used agencies, friend of a friend, recruitment apps, internal lists. Don't do the internal list. And there was no real way to solve it.

[00:18:20] We started with a hypothesis, where we started saying, if we outsource the research participants to, let's say, an internal desk, let's create a new role, someone within the team — because if we were wasting the equivalent of 1.5 resources for a year and a half, wow, probably there's budget to get that resource in. So we may reduce designers' non-design work, we will reduce research lead time, increase participant engagement to do more research, reduce the spending.

[00:19:00] So for nine months we ran a test in the UK to see, hey, what is happening? Can we actually get someone to run an outsourced and managed end-to-end recruitment process? So we worked with this resource to optimise and redesign a platform that the company had. We reviewed the overall recruitment workflow. We worked with all the cross-functional partners, like legal[?], to understand how we can operate within GDPR. We recruited and trained this person, we rolled out the surveys, and we started measuring.

[00:19:47] Remember that I told you that efficiency can be measured and it's quantifiable. So what is the data? What is the cost per recruit? How much time are we saving of designers' time? And how many people are we talking to? Are we actually able to engage with more participants? Yes, we did.

[00:20:05] So after nine months what we learned was that actually we were having a massive impact. After the first year, when we rolled out the service with 65% of capacity, we gained the equivalent of two designers' time. And on the second year, on full capacity, so our recruiter fully operational, we gained the equivalent of four designers' time, and we have enabled what's called ambidexterity. So the organisation can invest equally on innovating and on delivering, because we had extra time without having extra headcount. It's about using your designers better and making sure they don't work more.

[00:20:59] We increased the number of participants in research by 300% and cut the cost by 55%, with some peaks in some regions and some markets of even 70 plus percent. And of course, attrition in the team went down, because if you are happy and if you are engaged, why would you leave? The engagement score increased, and the lead time for research again was faster, 65% faster compared to before.

Case study two: teaching designers to talk data

[00:21:39] But there's also the case when we want to improve efficacy, and when we want to improve it, because it's all about behaviors. So again, I don't know your organisations and your teams, but my teams used to have this kind of a Cinderella syndrome sometimes. Product really don't listen to us. We go to the meetings with all our wonderful personas. We know it all, we spoke to hundreds of customers and users, we know what the user wants, but they don't listen to us.

[00:22:13] And yes, that's absolutely true, but that's not the problem. It's not that design is not important. The real problem is that design and design thinking is wonderful, it's critical, and designers are empathy champions. The problem though is that in organisations, decisions are made with data, not with empathy.

[00:22:45] So it emerged that in this case designers were not really familiar with analytics. Going into your platform, looking at the journey from a user behavioral point of view, and being able to understand what the opportunities in product are. We needed to teach designers to talk data in order to be able to become stronger in their approach, and making sure that they could have an informed conversation with a product partner, so that they were all talking the same language and they were able to influence designers and product.

[00:23:25] This is because it's important to start this shift from T-shaped designers that focus only on empathy, are really great at the craft, and they excel in doing design and they execute the brief and they deliver for product teams, and add a systems quality[?] into pi-shaped designers. Designers that understand users' behavior, they make decisions based on data, they discuss as partners with product and they deliver with product and make decisions with product, and use not quality of deliverables as a metric, but impact of the solution on user behavior. That was the challenge.

[00:24:22] So the hypothesis was, you know what, if we expose designers and all the cross-functional partners to data-driven or data-informed decision-making training, to make sure designers can understand and speak the same data language that the PMs and POs talk, then probably we can help them to have a better relationship, better focus and better decisions that actually benefit both the user and the organisation.

[00:24:59] So for six months we organised weekly, 4.5 hours of training with both subject matter experts and vendors. This has been made with the buy-in of all the leaders to make sure that this time was blocked: no meetings, no nothing, to make sure we had people joining and taking advantage of it.

[00:25:23] We measured also efficacy. It's not just the efficiency, we can also measure behaviors. And to do that, what we did was a survey to start capturing where we are, and monitoring the awareness and the level of confidence in data, the level of confidence in the analytics tool, and then creating a whole program and running these sessions every week, 90 minutes, with someone that could inspire them on how to think with data, how to use data in your day and how to make the most of it.

[00:26:05] So in six months the results were more than I was expecting. We had an increase of over 110% in the engagement with the analytics tool. Lots of people didn't even have an account on the product analytics platform or on the web analytics platform. 84% of the designers started using analytics, and there's been an increase of over 70% of projects in analytics platforms and tools. We also measured the confidence in utilising data, and we had peaks of up to 27% increased confidence in talking and using and referring to data in analytics, and 29% confidence in using the tools. And the queries hit a massive spike, plus 1800%. I forgot to say that.

What you can start doing

[00:27:04] So just to close, I just want to tell you why design is important, then what you can start doing and how you can start thinking efficiency, efficacy in your organization to bring those tangible results and to bring this kind of story.

[00:27:24] So first of all, what I shared with you, if my story is not your story, which means every design team has inefficiencies. Acknowledging that your team, no matter how awesome it is, has room for improvement is the first step, because denying that we can all do better won't help anyone.

[00:27:52] And then you need to understand your team and your team's pain and issues. No two teams are the same, and what I did for my team was what was right at a moment in time, because all the inefficiencies change constantly. They will evolve together with the evolution of your team, of the organisation, of the product strategy. Inefficiencies don't disappear, they are there, but you need to start thinking, analysing where they are and how they can be tackled together with your partners in a very systemic approach. Because if you help your designers to work better, you will also help your product teams to work better, and your analytics team, and everyone. So it's all positive reinforcement.

[00:28:55] And measuring efficiencies: every project you start with design operations has to have metrics. And if you are not convinced, and you don't know how to measure impact, how to measure efficacy or how to quantify efficiency, keep trying it. You just haven't thought it through appropriately yet, and you haven't framed the problem right. The moment you have your metrics, you're ready to go, because you'll be able to prove the value of design to your business and the value of DesignOps to your design organisation.

[00:29:29] And the great thing about solving problems for a team is that there's a domino effect. You solve for your designers, but you help other teams. So this will also help to create better relationships with your cross-functional partners. It will help you to have better experiences that actually will have users have a better experience.

[00:29:58] And the final recommendation is, always remember, teams are living organisms[?]. They change, they evolve constantly. So the one single most impactful thing a design operations person should do is listening. Never stop listening, because what you hear is what will happen on the ground and what will influence not just the single designer, not the squad, not the team, but the organisation, the business and the user. Any questions?