From Output to Outcome: Driving Cultural Change in Product Teams

Jay Selvaraj at UXDX EMEA. Video: https://youtu.be/KAac22g-QL8

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

From the agile manifesto to the output trap

[00:00:08] Thank you, AJ. Thank you. Hello, everyone. A few decades ago, the software development landscape was very chaotic and complex. Very similar to today, I would say, but maybe a bit better today. Back then, the teams were slowly moving from the waterfall way of working in the 70s to trying to figure out new ways of building software better, quickly as well as reliably, because one of the challenges they were facing back then was that building software was very, very risky, unreliable, and took time.

[00:01:03] In the '90s, 1995 I should say, Scrum was introduced, and then there was something called Extreme Programming. I don't know if you guys came across that. But in February of 2001, 17 experts came together in the Snowbird ski resort in Utah. They got together and came up with the legendary Agile Manifesto. I know a lot of people talked about it, but I brought you the coolest picture and the trivia as well.

[00:01:52] Thus began the agile revolution. If you think about it, it is a revolution. It started as a grassroots movement initiated by software engineers and scientists, and it became very impactful, even adopted by top management. Today we have terms such as business agility, strategic agility and so on, thanks to these guys who came up with this. So it certainly had an impact on business, on developing products and so on.

[00:02:40] But the primary goal of the Agile Manifesto is to develop better software. If you look at the Agile Manifesto, one of the things is working software over documentation. It's all about how we build working software quickly, efficiently, reliably. I would like to share a quote with you. These days you can't have a product-related conference and not quote Marty Cagan. "The most successful companies don't just deliver software, they deliver value that changes customer behavior." So we do need an update to the Agile Manifesto.

[00:03:47] Back then, the challenge was that the teams couldn't ship features quickly and reliably. Over the years, teams managed and learned to build products more efficiently, build features more efficiently, but not necessarily valuable ones. And that's where we are today. Today's challenge is how we build features that are valuable. I think Rory mentioned this yesterday. I believe most features developed today fail to create any customer value, anywhere from 50% to 80%. I just want to have that big range because I don't want to be wrong. It differs from company to company.

[00:04:52] Along the way, agile became all about speed, all about delivering products and features on time, on budget, and product teams started to fall into this output trap. In most organizations or teams that I worked with, the primary goal, I should say, is to ship features. Once you do it, hey, it's a win. But that puts you in this output trap where you don't deliver any value. I would like to share another quote with you, by Jez Humble in his book Lean Enterprise: "In the digital age, winning is about delivering outcomes, not just outputs, solving real problems for real people."

Why culture, and where we started

[00:06:00] So why am I talking about this topic? Why is it important to me? I've been building digital products for nearly 20 years now, and over the last ten years or so I've been trying to integrate research into the agile development process and design into the agile product development process, and improve processes and best practices to create value-driven organizations, outcome-driven organizations. The short case that I'm about to present is one of my recent transformations that I worked on.

[00:06:49] Again, you might have seen this already, probably many times in this conference: "Culture eats strategy for breakfast." It's attributed to Peter Drucker, the management consultant guru. Nobody knows if he actually said it, but it's really true. You can have whatever organizational goals, team goals, strategy. If you don't have the right culture, if you don't provide the right environment and the right toolkits to your team, it's going to fail. It's bound to fail.

[00:07:35] In this case, what we did was we wanted to instill a culture that delivers outcomes. I'll tell you why. Maybe I will skip a few of these slides to give you some context. When I joined this business a year and a half ago, there was tech and there was business. The business came to the tech teams and asked, "Hey, build this feature, build that product," and the tech teams delivered, no questions asked. Their biggest metric was: are we shipping these features and products on time?

[00:08:22] That was the culture in the company, and to change that culture we looked at four areas. One of the most important areas is creating a shared purpose and values for the team. We asked these questions. Why do we exist as a team, as a product team? What do we stand for? Is it to deliver whatever the business team asks us to do? What is our purpose? Two, how can we organize ourselves in order to deliver outcomes? Third, how do we work together? How do we work cross-functionally? How do we interact with each other? And finally, how do we define success? How do we measure success?

A shared purpose and values

[00:09:28] Most of this was centered around this foundational, purpose-led approach, and there's a reason for that. As I mentioned, the culture was really about delivery, delivery, delivery, and the designers and engineers weren't part of the solution. They weren't asking questions like: why are we building this feature? Who are we building this for? What is the outcome we're looking at?

[00:09:57] So we came up with this purpose. This is not something that was forced top-down. These are purposes and values that the teams came up with; we did this with a co-creation process. As you can see, the purpose we came up with was: we want to solve people's problems through collaboration and a human-centered approach to create a sustainable world for us and for our future generations.

[00:10:35] A lot of companies have company-level purposes, missions and values, but many teams fail to connect with those. For example, in our organization, the company's mission was to be carbon neutral by 2035. But what does that mean for a product team that is working on customer service? They didn't have that connection. So it's good to have purpose and values at a team level. Also, if you look at the values, we had empathy, not only for our customers but also for co-workers; curiosity, to learn about the product, the competitors, the market, the customers; collaboration; trust; innovation; and impact.

Cross-functional teams with shared outcomes

[00:11:41] To reflect that, we went from a siloed team, where business had their own goals, product had their own metrics, and design and tech had their own metrics, to a cross-functional team. This is more like a cross-section of a value stream. At the core, based on the product operating model, you see the PM, the engineers and the designer for customer-facing products, and we customized it to our organization. We created a couple more layers on top of this core product team, with the extended team: data analysts, solution architects, other developers, DevOps. And the business sits on top, which includes marketing and sales. We also have something called CX, customer experience.

[00:12:51] All these teams have the same goal, the same outcome. We plan quarterly OKRs together, so we don't have separate product metrics and separate design metrics. Metrics such as CSAT, reducing churn rates, increasing conversion rates, improving operational efficiency, reducing the cost to serve customers: all these business metrics and goals became the team's metrics.

How we collaborate: continuous discovery and delivery

[00:13:30] The third area is how we collaborate, how we interact, and this is where the process comes in. The word process has a negative connotation, but as many people before me explained, it's necessary. I wouldn't call it evil, but it is necessary. To connect with the purpose that I mentioned: if you look at the org structure and the ways of working, everything goes back to the purpose. This is our own version of an opportunity solution tree.

[00:14:15] You saw the purpose of solving problems: a problem-solving mentality for engineers, designers, PMs and business stakeholders. So we started working with framing problems, framing opportunities, and quantifying problems and quantifying opportunities. How can we deliver the best value for the business? Not picking up the next ticket and building features.

[00:14:48] This is a continuous discovery and delivery process we implemented, and discovery and delivery are not separate from each other. As you can see, all these cross-functional teams work together. Within discovery we had a three-phase approach: insights, ideation and validation. I don't want to go too much into the process since we don't have enough time. We defined how researchers and data analysts work together to bring quantitative and qualitative data, and how those insights go to the wider stakeholders to come up with different ideas and hypotheses that can later be validated.

[00:15:38] Out of this process, what we really wanted to get was continuous learning. The outcome, as you can see, whether we are archiving, validating or invalidating a feature, or learning through experimentation: the end goal is learning, not shipping features. If you are an organization that wants to innovate, the most important thing is learning about your customers, and that is the biggest differentiator for any organization. If you know more about your customers, you're going to come up with better solutions than your competitors.

Redefining success

[00:16:29] And then finally, the final pillar is redefining success. Change takes time, and if you go and set these business goals and KPIs and expect outcomes in a short time, that is not going to happen, because people need time to adapt. One of the things we did, maybe before we go into that: this is from SpaceX. "Every rocket failure gets us closer to Mars." That's the mindset we wanted to adopt as well. The way SpaceX looks at any of these rocket failures is: okay, we learned something new that we didn't know before, and that's going to help us get to that north star, or Mars.

[00:17:33] So we redefined success. Learning equals success. What can we learn about our customers by shipping a new feature, conducting research, validating research, or experimentation? And also rewarding knowledge sharing. For example, we rewarded people for presenting their learnings in team all-hands. In experimentation, for example, the core value of experimentation is the learning. If you don't share that learning, and if you don't learn from the learnings, you're not extracting value from the experimentation. By sharing these wins and failures with other teams, they were able to learn from them and not make the same mistakes in their A/B tests or in their experimentation.

[00:18:43] One other thing is we rewarded the process. When we did performance management or set quarterly goals for teams, PMs, designers and engineers, we looked at, or we encouraged: how many discoveries did you initiate, how much research did you initiate, how many features did you validate? That reinforced the process, not just "okay, the business asked me to deliver this, so I just went with it." We wanted to avoid those sorts of issues by reinforcing and rewarding the process.

[00:19:31] It's also about celebrating wins. That's how you keep the teams motivated. As I mentioned, to get to the long-term goal, the outcome, you need to be able to motivate teams even if there is a failure, and that's one of the things that we did with this last pillar of redefining success.

Summary and the outcome

[00:20:00] I want to quickly summarize what we did in this transformation journey. We rooted our transformation in the shared values and purpose of the team, and we reorganized ourselves to reflect those purposes and values. Then we reinvented our ways of working, once again to reinforce that mindset and culture: how can we deliver value, how can we collaborate cross-functionally and get to the outcome that we want to? And finally, success and rewards: redefining success, and rewarding even failures, as long as people learn something from them.

[00:21:09] I wanted to show or share some numbers, but unfortunately I can't do it. But I will talk about a medium-term outcome, I should say. When we started this transformation, we had this financial goal. We wanted to reduce the operational cost; we wanted to bring down the cost to serve customers. So we had a target. After implementing this process, the customer service team especially, which was one of the pilot teams, was able to use the opportunity solution tree and the continuous discovery and delivery process and so on, and live true to the purpose and values that we came up with. By doing so, we were able to exceed the set target. We went 28% beyond the set target for reducing the cost to serve a customer.

[00:22:25] That's how you get to the long-term outcome, as I mentioned earlier: give the team a purpose, the right toolkits, the right environment, the right culture in order to drive outcomes. And then setting the right KPIs and the right goals, those all matter. But without the culture, it's very hard to get to the final outcome.

[00:23:01] I hope from these insights you can use some of this within your organization, whether it's a purpose, the process and ways of working, or rewarding, encouraging and celebrating wins. I hope you'll also be able to make those small transformations within your organization. With that, I would like to end my talk. Thanks so much for being here. Thank you.

Q&A

[00:23:45] Host: Amazing. Lovely stuff, Jay. Really, really nice. For me, the term continuous learning has been used quite a lot, but I think it's one that I often forget to try and champion as the only researcher in my company. So for me, a nice reminder, back to basics, of continuous learning, not just continuous delivery. Amazing. We've got a good chunk of questions that have come through already. You've talked about rewarding and changing what success is. Great question from Anonymous here: what does rewarding failure look like for you?

[00:24:22] Jay: One of the things that we did was offering a small reward to present a failure. Before, in the all-hands, people weren't really sure if they wanted to come and present something like, "Hey, we did this, we tried this, but we failed." But then we started encouraging that and also rewarding it. We were giving a small gift to do that. Then people started turning up and sharing their stories, and that way the other teams learned a lot: "Oh, okay, we were going to do exactly the same, but maybe it might not work." So the sharing of knowledge within the teams is, I think, really key.

[00:25:08] Host: Absolutely. If a mistake you have made, by sharing it, can prevent someone else from making exactly the same mistake a week later or a year later, that's really, really powerful stuff. Great question here: "Learning is key, agree. Sometimes product teams favor learning at all costs, and that might be at the expense of delivering and maintaining good quality for our users." So interestingly, a lot of "let's not focus on shipping lots of features, but really value-driven features." How do you build that trade-off between quality and learning, and is it a trade-off?

[00:25:41] Jay: As I was saying, it's not just about failing for the sake of failing. We are shipping features, but what are you learning from that? That is very important. A lot of teams don't do that. They just ship it and very rarely measure it. Sometimes it also takes time to measure the success of a feature, so that is always overlooked because you're already on to the next one. So you don't know what you really shipped. It's very important to learn from both success and failure. The goal shouldn't be "let me fail and learn." The goal should be: what kind of outcome can I get out of this feature? And if it fails, that's great, it's still a learning. Every time SpaceX launches their rocket, they weren't waiting for it to blow up. They were expecting it to go somewhere. But if not, you still learn from it. That's the idea.

[00:26:46] Host: I don't think Jeff has a dashboard that says number of rockets blown up and a number he's trying to hit. But there's stuff to learn every time. Amazing. How do you manage to drive change in early-stage startups still trying to find that product-market fit?

[00:27:06] Jay: Whether it's a big organization or a small organization, the culture remains the key. Actually, it's harder to instill this sort of culture within bigger organizations compared to startups. With startups, it's also about the founder's mission. The purpose doesn't apply only to the organization. It could also be teams, it could also be individuals. If you're a founder with some purpose to change the world, the drive comes from that internal purpose of that individual, and through that they can also set up team-level purposes: "Okay, this is my purpose and values, and how can I create something similar within the teams, within the organization, so we can achieve that goal together as a company?"

[00:28:08] Host: Nice. Yeah, that makes sense. Cool. Great question, Christian, thanks for popping your name down on this one. What would you do if you realize that the people in the company have different core values?

[00:28:23] Jay: That's a tough one. Again, when we created these values together, we wanted to be inclusive, understanding what their core values are. I hope their core value is not working; if that's the case, that's going to be bad. But how can we include that? For example, when I shared the values, there are things like trust, because these came from people saying, "Hey, I don't trust my colleague, we don't have trust among each other." So we need to have that as a value, so that we start thinking about how I can trust, and if I don't trust someone, how I can use candor to talk about it and work together as a team to accomplish common goals.

[00:29:21] Host: Makes sense. I remember a colleague of mine, when I worked in a team that was involved in pushing the culture of a company, turned around with, again, a quote that I think is misattributed, or he couldn't find the basis of. He said it takes about seven years to change the culture of a company, and he reckoned that was because by the time you got there, you'd onboarded enough people into what you said the culture was going to be, and lost the people that didn't want to work there anymore. He reckoned seven years to turn that culture wheel around. So it takes time. If you are struggling with that at the moment, it is definitely a long game. Keep talking about it, keep living it every day. Love this top question as well, let's dive into it. Last minute, sales team: how did you get the sales team on board to sell value rather than features?

[00:30:04] Jay: It depends on the organization. For example, in this particular organization we have B2B and B2C. In B2C, most of the conversion happens online, so the digital product teams own a lot of these user journeys. But bringing the business stakeholders in to think from a value perspective, that's one of the challenges. For example, when we were pushing for A/B testing for a feature that a competitor released: "Hey, we already know it's going to work. It should work. Why do we need to do the experimentation? It takes eight weeks." Then we convinced them: "Yes, but we can measure success. If it succeeds, it's going to make you look better."

[00:31:02] Host: I love it. That, I think, is your mic drop moment, Jay. If it succeeds, you look better. Absolutely amazing. Please, everyone, go absolutely crazy for Jay. Thank you so much. Amazing. It's my first after-lunch stand in front of...