We're agile. So why haven't outcomes improved?

Rory Madden at UXDX USA. Video: https://youtu.be/RiG9Pusp0nQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Everyone is agile, and outcomes still aren't improving

[00:00:00] I want to go through kind of similar content to what I went through in that really quick opening presentation yesterday, and it's why we host these events. It's the fact that when I ask people — and we'll do a quick show of hands — who is working in an agile way of working, I'd say 90, 95% of people put up their hands. But yet, when I showed the stats yesterday, how many features are we building that are getting the expected business benefit?

[00:00:32] When you do real, thorough, academic research — and Ronny Kohavi, you can Google him, look up his research, it's very thorough — he's finding that we're not delivering the benefits that we expect, even though we're doing this agile, we're doing our quick iterations, we're learning from our customers, and yet we're still getting these terrible results. And these companies aren't companies that are laggards or things like that. These are digital native companies that are still struggling to try to get the expected business value from the features that they're building.

[00:01:05] So we're going to try to have a chat about what's wrong. And the beautiful part of this is it's really easy to tell you what's wrong. It's you guys. You're doing it wrong. There's nothing wrong with agile, it's just how badly you're putting it in place. That's the problem. So what we need to do is forget complaining about agile and forget saying that it doesn't work. You should figure out how to do it better. So, yep, thank you very much.

[00:01:36] And the one thing that I like to say is there's no such thing as user error, there's only bad design. And as you can tell from that nice little graphic, I am a very talented designer. So what I want to talk about is how is agile implemented and why are we getting such bad results, and then I'll talk about what we can do about it.

The Agile Manifesto left the day-to-day up to you

[00:01:58] 23 years ago now, the Agile Manifesto was written and they came up with four principles. And that's great, that's like, yeah, we should do that, but that doesn't actually tell me anything about what I'm going to do day to day. It's left completely up to me to figure out what should I do to get this iterative way of working, get my products to customers, learn from them, iterate and make it better.

[00:02:26] And people are busy. You're busy in your day-to-day job, you don't have time to step back and start thinking about the intricacies of all our processes and how we should work. Yes, we might do a retro, but that's a very small portion of our working activity. 99% of the time we're focused on building our actual products.

[00:02:48] So what a lot of companies do is they say, I'm not going to try to build everything from scratch, I'm just going to go and find something off the shelf. Scrum. Maybe your bigger company might pick SAFe, or there's Scrum at Scale, there's LeSS, there's lots of these kinds of different platforms. But Scrum is probably the world's most popular agile framework, and the reason why it's so popular is because companies don't have to think. They can just go to the shelf, pick it up and implement it.

[00:03:19] But then if you read the Scrum guide, it tells you, okay, here's how you structure your team. You need a scrum master or a product owner in the development team. You're like, okay, well, what's that involve? Well, we've got these project managers, so we'll just rebrand them. And we've got these business analysts, so we'll just rebrand them. And then job done, we don't have to think much any further. Oh, there's some product backlogs or something, we'll do those as well.

Water-scrum-fall: what happens when funding and governance don't change

[00:03:46] But what it does do is it tells us the process that we should work. Across the top here we've got our traditional waterfall steps: you plan, analyze, design, build, test, deploy. But Scrum said, let's not do our nine month, one year projects, let's really condense that into a two week sprint, multiple two week sprints. And the idea is that you go start to finish, get working software in front of real users and gather their feedback.

[00:04:13] But it left out one thing. So, smaller iterations, but what it ignored was, how do we fund these things? It didn't tell us anything about how to fund it. It was kind of, you figure that out. But people are busy, and we already have business cases, so why would we bother change it? If it ain't broke, don't fix it. So we'll just keep our business case process, but we're going to change how our teams do it and we'll have those two week iterations.

[00:04:41] But how do we govern it? How are we going to govern these teams? Well, we have our business case, and our business case gives us time and cost, so we'll just use that time and cost. We'll get burndown charts and then we'll track you. Are you tracking? Oh yeah, yeah, your burndown is moving up and down, that's not interesting. Are you on time and are you on budget, because that's what our business case tells us.

[00:05:03] So we end up actually shrinking down. Maybe we'll remove planning from Scrum, and what we'll do is we'll put together a detailed plan, because if we're going to be held to account for time and cost we need to know a bit more up front. We can't have surprises coming up halfway through. So let's do a big plan.

[00:05:22] And actually, when we think about it, we better do a big bunch of analysis as well, because we don't want — let's say we get three quarters of the way through, we learn something and then we have to change the architecture, that's going to be a pain. So let's do a big up front requirements, or a PRD or something. And you know what, actually, we better do a technical design on this, because again, we don't want to have to change things midway through, that will throw us off.

[00:05:50] So what we've ended up with here is water-scrum-fall. Because you just keep shrinking what is available for Scrum, because that's what you're already doing. And Scrum didn't tell us how do we fund it, Scrum didn't tell us how do we govern it, so companies had to figure it out for themselves, and what ends up happening is they just revert to the status quo.

[00:06:12] Who would say that's a representation of how they actually work today? Okay, maybe 50%. So there's some people. Let's see if anything changes as we go through.

Scrum predates UX, and scaling frameworks aren't the answer

[00:06:25] And the problem as well with Scrum is Scrum was written 20 years ago. UX, I don't even think existed, or it was a very, very niche term 20 years ago. So they didn't think about product, they didn't think about UX. Their focus was on software development. So now you have this plethora of, how do you integrate lean UX with Scrum, where does design thinking fit in, what's the role of a product manager, how do we do this? And you're inundated with different opinions and different approaches. And the problem is that it's really hard to do, because there is no easy way.

[00:07:02] So when I said we're busy and we're focused on delivery, we don't have the time to try to figure these things out. And when I said at the start, if you want to make change happen there's two ways you can do it. You can focus on motivation, increase motivation and you'll get across that action line. But there's a second way, which is about the ability: you can make it easier to do as well.

[00:07:25] So Scrum came along and they recognized motivation was low as well, so: take our off-the-shelf approach, because everybody's busy, they don't really care about it, they want to tick the box that we're agile. But the problem is they only got us halfway there. They didn't actually get us the whole way. So that's why we end up with a half waterfall process and then we do two week sprints in the build phase.

[00:07:52] But somebody out there might be thinking, okay, but you're just looking at agile on its own. There are scaling frameworks that talk about the funding, they talk about the governance, they talk about all those things that I've just raised that Scrum left out. And if you've ever read one of those, I think what you'll find out pretty quickly is that it's not really agile. It's very much centralized decision making, centralized funding. It's the opposite of empowering teams. It's very much, we are going to tell teams exactly what to do.

[00:08:25] And then the other reaction that people might have is, well, you know what, it's a mindset. Agile is a mindset, and you just don't get it, you just don't understand, and you're just being so complicated. We need to do outcomes, not outputs. What can I do with that? I can't do anything with that. When somebody says that, I need to go and deliver, because I've got my time and budget timelines, and somebody's giving me this neo-hippie new age mindset stuff. That's not valuable to me, because it's not tangible, there's nothing I can actively do with it.

[00:09:02] So when it comes to it, we're still here. We're still here in the fail mode, because we haven't made it easy. And we try with that kind of motivation mindset stuff. It doesn't work. I have wasted years of my life trying to focus on motivation and I've realized: give up on motivation, make it easy. That's the way that you'll start to get change to happen.

[00:09:26] So I said at the start, if agile isn't working, you're doing it wrong. But where's the process? How are we actually supposed to work? How do you structure teams? It doesn't tell us that. How do we structure teams so that when we've got 15 different teams they're not all conflicting with each other? None of the guides tell us that. We have to figure that out ourselves. What about alignment? How do you make sure people aren't going in different directions? I don't know, figure it out yourself. So there's lots of different things that we've just left up to people, and then we give out to them saying, oh, you're doing it wrong, that's why it's not working, that's why you're not getting the outcomes that you expect.

What a real answer has to cover

[00:10:03] So we've talked about all the problems. I want to move to something that's optimistic: what can we do about it? How can we really focus on making it easy? What are the things we can do that will actually get us to where we thought Scrum was going to get us?

[00:10:21] But it has to fill in all the missing pieces that Scrum left out. It has to talk about the process, but really detailed about the process, and not just one fraction of the process. It has to include funding, it has to include everything else. Talk about the structure: how can we have multiple teams and make sure that they're not all conflicting with each other? How do we get alignment? Because if you give teams empowerment, it's so scary for managers, because they're thinking, what if people go off and do crazy things? So how do we make sure that everybody is happy that we're all going in alignment along with our vision and strategy? How do we fund it? How do we govern it, if we don't have time and cost, what are we governing people on? And then how do we scale this across an organization?

[00:11:07] So what I always think about is these stats. If we can start from there, what does this tell us, the fact that this isn't working, and this has been going, it's getting worse, from 2009? And that 2009 study is really interesting, because it's not just that 33% didn't work. 33% actively hurt the product, so it had a negative ROI. And a lot of our features end up in that way as well.

[00:11:39] So what this tells me is we don't know what our customers want. If we knew what our customers wanted we would have better outcomes than this. And the reason we can't know what our customers want — and I don't think it's a failing on anybody, it's not a "we should have planned more, we should have done more up front" — you can't predict behavioral change. It's impossible to predict behavioral change, and our new features are expecting people to change how they behave. The only way you can do it, and there's research into this, is you do something small, you see how people react to it, and then you respond to that.

[00:12:17] So that's what we need to look at. We need to be able to get from the idea of what we want to do over to getting it done as quickly as possible, because we need to get it out there in front of customers, see how they react to it, and then iterate on it.

You can't manage dependencies, you have to remove them

[00:12:36] So make it as quick and iterative as possible. But we have all of these roles that are in there to manage the complexity, because that's not easy to do. There's a lot of different people that are involved. You might have marketing, you might have operations, you might have different people coming up with the ideas. You'll have senior leadership signing off business cases. You'll have architects, designers, developers, front end, back end, operations, testing. So you have a lot of people involved in that, and then when you're doing this in multiple teams you're going to have conflict everywhere. So there's managers whose job explicitly is to control all of these dependencies and the complexity.

[00:13:19] But if you're a team trying to get through this, and you have to go to the PMO and you have to get your project funded, and then you have to go and get a project manager, and then you have to go and get an architect board to sign off your design, you have to get a change control board to approve your release — that's not quick. That's not being able to get from idea to customers really quickly. That can take months in a lot of organizations I've been in.

[00:13:45] So a lot of scaling frameworks, they'll say, well, what we actually need is more managers. We'll add in a release train engineer, we'll add in a scrum of scrum masters, we'll add in a chief product owner. Because the only thing, if you have a problem with coordination, is managers. But that's giving us more bureaucracy and it's slowing us down even more.

[00:14:07] So my principle — and I think a lot of people, when we had the workshop earlier in the week — the principle is you can't manage dependencies, particularly as you're scaling. The more teams that are interacting, the number of interaction points and dependencies just grows exponentially, and it becomes impossible. And that's why these big companies slow down and slow down and slow down, because they're constantly getting more and more dependencies to get anything done.

[00:14:39] So we take a view that we have to solve this the other way. We have to remove the dependencies. So we need to give one team ownership end to end. They need to be able to do that. But that breaks a lot of stuff. That breaks alignment: how do we make sure that they're going the same direction? It breaks funding. It breaks all of these other processes.

One team, one process: discovery, design, development

[00:15:00] But let's focus in on how this would actually work. If we said, okay, this team, you now own end to end, well, you're going to have to do some up front research to figure out what to build. You're going to have to do some design to figure out what solutions — so you've identified problems, what solutions could we build? And let's do some evaluative research to test them and see whether they work. And then you have to build it.

[00:15:24] So we say we need to have one team with one process. There is dual track development out there as a way of designers and developers working together, but we believe we need three steps. Discovery, which is generative, that's talking to customers, finding their problems. Design, evaluative, that's coming up with solutions, checking whether they work. Development, that's building the product. So this is one single team.

[00:15:51] Now, that breaks a functional structure, because you can't have a designer handing off to a researcher, or a researcher handing to a designer to a developer. It's one team. You lose your functional identity and you become a team member. So that's a big change, but it's necessary to get that quick feedback as you're going through, because you don't want to have backlogs within your team where you go, oh, I've done my research and now you go do your design. So we identify opportunities, etc.

Structuring teams around value streams

[00:16:24] So if we have that process — and I know I've run over that very quick, but we'll come back to that — teams don't exist in isolation. There's multiple teams, and I've talked about all the dependencies that happen as you grow. So it's great that you've got this process that works perfect for one team, but it all falls apart once you have five, ten different teams working on the same code.

[00:16:46] And dependencies between teams remove autonomy, and once you remove autonomy you remove accountability. Because if you can't get your product out there quickly, you can easily throw up your hands and say, I did my job, it wasn't my fault, I was blocked behind the change control board, I was blocked because it took three months to get this approved, the market changed, the economy took a dip. All these excuses, because as soon as you're no longer autonomous you can't really be accountable.

[00:17:17] So how do we structure teams to make sure that they are autonomous? Well, the purpose of a company is to build a customer, so let's structure our teams around how our customers think about our products, because that's how they're going to want the features and want things. So let's take the customer journey at a high level and start designing teams that can own parts of that journey.

[00:17:42] Now, there's a technique called domain driven design. It's a technical technique, but it's brilliant for identifying autonomous regions where teams can work completely separately. And what they advocate here is, these kinds of decomposed circles, they own their own code base. So once you've identified different value streams, you give them completely independent code as well. So now you have completely autonomous teams that don't need to rely on each other. There is obviously data passing going on, but that can be done through automated APIs and things like that to reduce the collaboration overhead.

[00:18:22] So what we end up with, if we think about this, we've got these parts of our products that I like to call value streams. So you're not looking after the whole product, you're looking after a part of it that's delivering some value to a customer. So we have things like, maybe if we have an e-commerce platform, you've got a catalog, somebody needs to be creating all the products. You could have another team that looks after search. You could have another team that looks after the cart and the checkout flow, payments, maybe loyalty, those kinds of things.

The product team: vision, strategy and alignment

[00:18:53] But they need to be aligned. So we can't just have all of these streams, we need a second type of team. So what I'm introducing here is two kinds of team types. The ones who own a part of the product, because as our products get big you can't have one team own everything, you have to break it up into those value streams. But you need a second team to manage them, to give them direction, to make sure that they're going in the same direction. We'll talk about alignment now. I'm seeing some confused faces, so grab me afterwards if I'm not making this clear, the distinction between a value stream team and a product team.

[00:19:30] Because the fear from management is, let's say we isolated the code, we gave everybody their own teams and we said, okay, now you go off and just build things that you think are worthwhile, because we're no longer doing a business case sign off, we're no longer getting that senior leadership stamp of approval. The fear is that you end up in the bottom corner where everyone's just going and doing their own thing. So we need a way of aligning people, and that's that product team.

[00:19:59] The role of the product team is to set the vision. Where do we want this product as a whole? Because we don't want a different experience coming along in the search versus the cart. We don't want people focusing on different areas of, oh, there's an opportunity here, there's an opportunity there, but then to the end user who looks at it as one product it starts becoming disjointed and a Frankenstein style product. So you need somebody who is giving everybody the direction of, we're going to target these markets, we're going to target this opportunity space.

[00:20:32] So they have a vision, you have a strategy which is more focused in, and then you set quarterly objectives for those teams, because you need to be able to influence without telling them what to build. So the product team doesn't tell the stream teams go build this feature, go build that. They're saying, these are your objectives that you have to figure out how to solve yourself, because the teams own it from idea through to satisfied customers.

Funding value streams, and governing on outcomes

[00:20:58] So how do we fund it? Let's say we had a product, and I've just given five different example stream teams, but how do we fund it? Because business cases are great, they say, okay, we'll give you x amount of money if you go build this feature. But now we're saying we're going to empower the teams to come up with the ideas themselves. So how do you fund that?

[00:21:20] You don't fund teams. You're not giving money to teams. You're funding value streams. So you say, our strategy is that we want to increase upsell, I'm going to fund the cart more. Or, our strategy is that we want people to be going from search to booking, we want to increase our conversion rate from search, I'm going to fund search more. So you're funding the value stream, you're not funding the product.

[00:21:47] And then what you can do is you can decide, oh, I'll give one team one high priority value stream, I'll give another team two or three lower priority ones, I'll make one team smaller, one team bigger. So once you've got your funding, that determines the work, determines the size of the team and the scope of the team.

[00:22:07] Now, how do we govern? Time and cost is brilliant, it's so easy. It's really easy to check is something on time and on cost. It's even easier to check how many hours somebody's working. So there are the metrics that go: effort, how many hours are you working; then it goes to output, what are you producing; then it goes to outcome, what has the business, how are the customers responding. And that's our sweet spot. So now we need to move our metrics to being this outcome.

[00:22:34] And what the product teams do, because a business doesn't care about, oh, are users using this feature — they care about, are we making money, are we reducing costs, are we on strategy. So a product team acts as a translation layer and a political buffer for the stream teams to be able to go and focus on doing what they want. They convert the business goals into those product outcomes.

Scaling: internal product teams, enabling teams and the ecosystem team

[00:22:59] But we haven't talked about scaling. So we've got this structure, but it introduces some problems, because what if we want a shared platform? There's a lot of duplication happening across all of these teams. So what we can introduce is a new type of team, an internal product team. And internal products deliver value to the stream teams. So you might have a technical platform, you might have a design system, because that's helping all of the stream teams improve.

[00:23:26] But you might also have finance and legal. We need to think of those as delivering services to our stream teams, because if they need some legal advice for GDPR or some new thing, that's going to block them. And once you remove autonomy you remove accountability. I'm sorry, couldn't get it done, it's not my fault. And we need to make sure teams can be held accountable.

[00:23:48] There's also the problem that once you isolate people in different teams, they no longer have the functional backing of their peers. You're not sitting with all of your peers, so you're isolated, and that can be difficult. Then how do we upskill people? And that's where the concept of enabling teams from Team Topologies comes in. So these are your senior staff, principal level people, and they float around and they see, okay, where are the problems, who needs to be upskilled, what training can I do, what tools can we introduce that will help everybody, what kinds of practices can we document?

[00:24:24] And then the final thing is we need one overarching team who's giving the direction. Because the product team is giving direction to the stream teams, but you need somebody who's making sure that the internal teams, the product teams, the enabling teams are all going in the same direction as well. And that's our ecosystem team. So in a smaller company that could be your C-suite, in a bigger company it could be a business unit.

Zero Blockers: the documented reference

[00:24:49] So you're probably wondering: Zero Blockers, you've got water bottles, you've got caps. We've spent the last few years documenting this, because somebody asked me five years ago, just tell me the process, we've got Scrum, tell me how to do this and I'll implement it, I don't have time. And I thought that'll be quick. Took me five years.

[00:25:09] But what we've done is, if you go to docs.zeroblockers.com, it goes into detail about how to do all of those steps that I talked about, that I gave a two-second overview of, oh, we changed the process. But it goes into really big detail, so you can use it as a reference, that if you're trying to implement this there's a reference go-to about how to solve all of those problems that we talked about.

[00:25:40] It's structured like this. You have the framework, which goes through all of the high level why it's structured, the principles, how we deal with structure, scaling, all that. We have the five teams, and then down the side we have the processes. So for a stream team you have continuous discovery, continuous design, continuous delivery. You click on one of those, it gives you the purpose, it gives you the context of why we favored this practice over another, and then it goes down to the artifact level as well.

[00:26:12] So we've just released this. This is definitely a v0, but we would love to get your feedback, and we've released it on Product Hunt, so please do support us. If anybody wants to scan that and give us an upvote, we really would appreciate it, because we're trying to get it out there, we want to see how teams can use this. It's completely free for you to use, the references and the resources. So we really would appreciate it if you can give us an upvote. And yeah, I hope you found that useful. We're open now for some questions. Thank you very much.

Q&A

[00:26:44] Host: Amazing, that's great stuff. All right, so, with this approach of creating teams focused on value streams, how would we ensure we are not creating silos?

[00:27:00] Rory: Yes, it's a very good point, because if we have functions you often hear about functional silos, but as soon as you create value streams it's effectively another form of a silo. And the thing is, product development is complex. There is no way around it, it's really hard to build good products. And in a complex area you can reduce to a degree, but often you're just moving where the complexity is, and you have to balance what's the value of where I'm putting the complexity.

[00:27:34] Rory: So the reason why we go with value streams is because to a user that's how they look at our product, and they think of it in terms of value streams. So yes, you might end up with a bit of a silo in each value stream, but that's how our customers are thinking about it, so they can go end to end, they can go from idea to working software without any other dependencies.

[00:27:53] Rory: And then those enabling teams and the product teams need to be coordinating all the different silos to give them the direction. So we're trying to combat them going in different directions, and then the enabling teams are trying to make sure that they have the right skills and the right kind of experience, so that you don't end up in a clunky part of the product as you're going through. So yes, it is a silo, but we have to decide where we're going to put the boundaries. We have to split somewhere, and we believe this is a better way of splitting them than functions.

[00:28:26] Host: Excellent. What resonated with me a little bit, or something that stood out for me, is the funding aspect: funding the workstream, and also having the vision and communicating that across to create alignment. It's interesting that one of the questions that came up is, this is a major shift for a lot of companies, so can you discuss navigating executive buy-in?

[00:28:52] Rory: Big topic. Yeah, it's not easy, and I don't pretend it's easy. I would say it's suicide to try to say, tomorrow morning, from Monday, we're all going to start working in this way and we'll just do an overnight switch. It will not work, it'll fail, it's too much change at once.

[00:29:08] Rory: Pick a team, pick a product, identify one value stream, isolate out that code, separate them out, try it, figure out what breaks, figure out what doesn't work, iterate. And then once they're comfortable, then you start to evolve it out. So what I've done in the past is ask for a trial: give me three months, give me this amount of budget, pitch it as a business case, because you have to work within the current system to change the system. Pitch it, pitch it, pitch it.

[00:29:38] Host: Yeah, right, Rory, we're out of time. I'm sure there are a million of questions left here, because it's really, really interesting content. I do agree with you, change is hard, but there is a way and there is a will. So thank you very, very much for the thought process here and the leadership. Thank you very much.