Dual Track Agile

John Schrag at UXDX Europe. Video: https://youtu.be/q2dRk3hokEw

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Autodesk and the origins of dual-track agile

[00:00:00] Good morning everyone. As I said, I work at Autodesk. Autodesk is a huge company, it's like ten thousand people, and we have hundreds of products, but the part of Autodesk I work in is the Entertainment Creation Products division. We're the part of Autodesk that makes tools for animators, people who do computer animation, special effects for movies, TV shows, simulation, anything like that. These are some pictures by artists who use our software that they published on our forums. We make software, but it's our customers who make magic with it, and I love working in this field. I just love the creativity and getting to help create tools for these people.

[00:00:42] As I said, I'm the director of experience design right now, but in my past I've been a software engineer and I've been a UX designer. My own background is in design and software engineering, and I've been an agile coach. This will tell you a little bit of the story of how we came up with the idea of dual-track agile, and how that can help you adapt your UX practice, or in fact adapt anything else, to work in an agile environment.

[00:01:07] This is what I'll be talking about today. I want to ask how many of you have seen that diagram before? I'm curious. Okay, yeah, a good number, so I know which of you are process wonks. That's from a 2007 article that we wrote, and the diagram is by Marsha, who still works on my team.

How agile ruined experience design

[00:01:28] I've got to start off by talking about how agile ruined experience design. For those of you who are younger than me, which is probably most of you: back in the 1990s, well, I started working in the 80s, but all through the 90s I was working at a company called Alias. We were doing the same kind of work, designing software for computer animation, and we worked in a waterfall development practice, like pretty much the entire industry. I worked for a woman named Lynn Miller, and Lynn and myself and my colleague Desirée were this all-unicorn UX team. By unicorns I mean we did everything: we did our own research, we did our own design, and we did our own usability testing and validation. That's the way we liked it. We didn't do graphic design; we hired Marsha for that.

[00:02:12] We had a great practice, we had great rapport with the developers, we were doing everything right, everything was working beautifully. And then in 2001 the Agile Manifesto happened. You should all have seen this page; it hasn't changed since 2001, same graphic. Agile came along, this idea started, and my company was a really early adopter. One of the signatories of the Agile Manifesto, Jim Highsmith, was brought to our company the year after, in 2002, and he trained all of our developers. He trained all of us: we all got into a room and he did two days of training. Jim is a great trainer, he's really smart, I think he's still doing it.

[00:02:50] He totally revolutionized our software practice, but when he was describing how agile works, UX didn't exist in it. There was no mention of UX. They'd say, how do you know the thing's good? Well, you do a cycle, and they said sprint, and at the end of the cycle you would show your software to the customer and say, is this good? If your customer says yes, this is good, then you're done. And of course those of us who work in UX were like, this is terrible. You don't get the insight, you can't tell anything about learnability, you can't tell anything about usability. It's just a really bad way of doing things. I got into a big discussion with Jim, but that's another thing.

[00:03:27] Anyway, we adopted agile and our engineers were happy. Our practice worked really well, a few bumps along the way, but it was great for our engineering teams. They loved it. Suddenly they were in control. They used to be controlled from above; now they were driving the train and they could make things happen the way they wanted. But that was the engineering team. The design team had had a really good life, and suddenly we were at a loss [?]. We couldn't fit in the process anymore. We would do a usability study, but by the time we were done, development would be three phases on. Or we would be designing something and they'd have changed direction in one of the agile sprints. We couldn't give them feedback when they needed it, and we got disconnected. It was horrible.

[00:04:11] The funny thing is, we were going to other design conferences and seeing the same problem among other people. These are some quotes that people from other companies said to me. I wrote them down at the time because I thought they were kind of funny. Maybe they think, well, I'm just going to keep doing things the way I've always done, and they'll come back begging to me. That's from someone who I might assume has never met a developer.

Revelation one: to serve agile, design had to become agile

[00:04:33] This led us to our first revelation, of the three revelations I'm going to talk about today, and our first revelation was that to serve agile, design had to become agile. We thought about what that meant. Me and Desirée and Lynn, the three of us always worked together, went back to the principles of agile. We looked at the things in the manifesto and we thought about what it is that makes agile work. What's really happening in agile is you have this loop. You've all seen this a thousand times, the build-measure-learn loop, and you've brought this into development. But this is a basic idea in usability testing, that it has this loop. So we should be compatible. Why aren't we compatible if we have the same kind of loop?

[00:05:18] We had to look a bit more into agile practices and what makes them agile. There are a couple of different attributes, but here are a few I've picked out that make agile practices actually agile. They're more conversational: they're more about talking to people than writing documents. They're collaborative: you're working together. Incremental and iterative is super important. An agile process is something you should be able to do over and over again continuously. You don't do a big scary thing at the end of your release cycle and then everyone collapses. You should be able to steadily go along and produce stuff and incrementally add value. And of course things are time-boxed: you're working for a specific amount of time on a specific thing and then you stop, and everyone's working together.

[00:06:01] We realized this is part of our problem, because as designers we're not working on the same thing the developers are working on at the same time. We're supposed to be in front of the train, laying track, clearing bushes, making way for the train to go, and we're supposed to be behind the train, validating things, making sure everything went the way it was supposed to go. That's the kind of time box we needed. We needed to be desynchronized from development, and this brought us to the dual-track model, which we developed on Lynn's whiteboard back around 2003.

The dual-track model

[00:06:36] Here's the basic idea. At the beginning of a project you're in your sprint zero, and at this time you're figuring out what you're going to do, you're chartering. What you look for at this point is code that needs to be written that doesn't have any UI attached to it, because there's always something in the back end, or some technical debt, that the developers can work on that doesn't need UI help. Then once you've finished your sprint zero, your work divides into two different tracks, a design track and a development track.

[00:07:08] In the first sprint, the developers are doing that stuff you've identified that doesn't need design work, no problem. While they're doing that, design has a bit of time to look ahead: what's the next thing? The designers can be getting it ready, they can be validating it, making designs, usability testing with paper prototypes, and at the end of the sprint anything that's been validated can be passed back to development to work on in the next sprint. Design is always working one or two sprints ahead. They can also start researching further-forward stuff, so the design team can say, we see a few more things coming down the pipe, so we can start researching that. And you can just repeat this cycle over and over again. In sprint two we're working on sprint three stuff, but we're also validating what went into the code in sprint one. I think this whole slide deck is going to be on SlideShare afterwards too, so you should be able to get that. That's the model that we came up with around 2003.

[00:08:08] The one thing that wasn't really in our first diagram that Marsha did for us was this, and because of that we left people with a false impression that we were handing things off between sprints. That's not really true. There's constant communication that has to go on all the time. I want to be really clear about this, because the dual-track agile model has been written about all over the place. If you google it, you'll find dozens of articles about it by different people, and a lot of people get the same things wrong when they talk about it. Number one, be really clear: it's not two teams, it's two tracks. The same team is doing two different tracks of work, which are tracked differently, and that is really important.

[00:08:52] We made a whole lot of mistakes when we started out. For example, one of the first adaptations we tried was to just put design work in the backlog and treat it like development work. But that doesn't work in agile. That violates the agile principle that working software is the only measure of success, and it screws up your velocity and your calculations. We tried doing complete designs: we designed the whole thing, and at the end of every sprint we would redesign for the end product, over and over again. That killed our designers; they just got so far behind.

[00:09:23] But anyway, back to this. Things people get wrong: number one, designers are on the same team; that is correct. Insight is shared. We're not throwing design over the wall. We're not saying, we've done the design, now we're throwing it to development. That's not how it works; we work together on it. Developers have to be involved in design, because they're the ones who can determine the feasibility of building what the designers want to do. So developers have to be involved. Similarly, while the developers are building the actual features, designers need to be involved, because the developers are going to learn things: oh, that code's too hard, the API doesn't support that, and the design has to have modifications. So everyone's working together, one team, but doing two different tracks of work.

Making UX practices agile

[00:10:06] That wasn't quite enough, though, because we had to change the way we did our UX practice. If you go back to that list of agile attributes, you can find that you can pretty much adapt any practice to be agile by making sure it has those attributes. I'll give you an example of how we did it.

[00:10:23] Take, for example, usability testing. This is the way people did usability testing, and maybe some of you still do it that way. You wait until you have something to test, and then you create your prototype, and you plan your test protocol, and you recruit users, and you bring them in, and you run the test, and you write this big-ass report at the end that lists a hundred things that went wrong, and you hand it back, and you've taken three weeks to do it. That's not agile. It's not time-boxed, you don't know how long it's going to take. Can you do it iteratively? Is it collaborative, is it conversational, is it just in time? It doesn't really fit all that well, especially time-boxed.

[00:11:04] What we did is we changed our practice. Instead of waiting until we had something to test, we planned the tests first. We said, every two weeks we're going to have a usability test; we don't know what we're going to test. We built a pool of users, about 40 people, that our recruiter could just pull from. They'd pull five people in every two weeks that we could test with. And when the tests came up, we'd say, well, what do we have? We've got this paper prototype from the designers, we've got this bit of code that just went in from the developers, and we would just test whatever was ready. And we'd invite the team to watch, so that everyone could see.

[00:11:38] We'd effectively just made this practice hit all of these agile attributes. The other thing is that at the end of it we'd say, okay, here are the three issues we're going to fix. Maybe we have a list of 30 issues, but there is no point in writing a big report with 30 issues, because the fact is, when you fix the top three or five things, which is all you'll have time for in the next sprint, it's going to change people's behavior, and the rest of the issues may disappear, or new ones will show up and be more important. So you don't really need the rest of the list. You're going to have some completist designer who can't not write it down, so let them write it down, but it's not what you need to report.

[00:12:13] Look at the way we used to do user research. User research in waterfall: you'd go in at the beginning of a project and you'd talk to hundreds of users, and you'd interview them, and you'd build all of these artifacts, these big reports and personas and use cases and whatever else. I don't think we were doing journey maps back then. You'd deliver that to the team, and it would take like three months before you'd start building anything. That of course is not agile at all. You're making a bet there on the next two years of development, and that's just not a good way to work.

[00:12:48] To be agile, we had to find a way to make this iterative and incremental. We talked to users regularly, in different ways. Our usability testing was regularly scheduled, we'd go visit people, and we'd focus on what the next thing we needed was, the next piece of information. But we're also always updating our big documents. We have a persona document, and you can update it as you learn more, as you gain insight. We can update our journey maps, or whatever it is we're building. The idea is those artifacts stay around, but they're living documents that you're always touching and redoing continuously as you're visiting customers or gathering more research. You want a conversation in your daily scrums: hey, we talked to people and we got this new insight, so that insights are shared throughout the team as well.

[00:13:40] We have all different ways of touching users, and we're not afraid to mix methods. Sometimes if we have a usability test, we'll say, hey, we want to interview for some upcoming stuff, or can we run the usability test at your place, and while we're there we're going to do some contextual inquiry. We just put everything together and cut it into little bite-sized pieces focused on our immediate needs coming up.

[00:14:02] Design specs. Those of you who came from the waterfall world may remember the 60-page design spec with every detail written down. There's this law I realized: the longer your design spec, the less likely a developer was to actually read it. The way we do designs now, typically, is just a couple of pages, pictures, call-outs, and a conversation where you sit down with people and talk through the design. It takes so much less time, you're just so much more effective, and of course it's agile.

Publishing, naming, and losing agile

[00:14:29] We figured out how to adapt our process, we figured out how to time-box it all, and we got it working. It worked for our team; in 2003 we got this stuff going, and we started to publish on it. This is Lynn and Desirée and I in 2006. It was funny: Lynn wrote a paper for UPA 2004, I think, on this whole topic, and the paper was rejected by the conference because they said, and I quote, agile is a niche thing of interest to few people. So she ended up presenting it at Agile 2005. I did a paper on some of our adaptations at UPA 2006, and in 2007 Desirée put together all of our work and wrote an article for the Journal of Usability Studies, which for the next five or six years at least was, I think, the most cited article in the journal ever.

[00:15:25] That's when the story got spread around a lot. But we weren't the only people working on this; I don't want to take credit for that. A lot of people were working on this problem at the same time and coming to similar conclusions. One thing we didn't do, and we should have done, is come up with a great name for our method. We did not. We were calling it parallel tracks, but because we didn't have a really good name, other people were writing about it and calling it something else. You may hear this referred to as staggered sprints, or dual track, which is what I'm calling it today, but it's all the same thing.

[00:15:56] Anyway, we finished this and suddenly we learned that we love agile. When agile arrived at Alias I hated it, oh, I hated it so much, but we learned to really love it. We loved working this way, and our experience wasn't unique. This is Jeff Patton, who you may know as the story mapping guy. This is a quote from him; he had the same experience, when it was really bad. He interviewed us back then to see how we had been fixing that. Again, we loved it.

[00:16:23] But then in 2006 my company was acquired by Autodesk, and at that time Autodesk was not agile, and they crushed our agile development practice. Just like, nope, back to waterfall, back to the 60-page design documents. I've got to say, you don't realize how good you have it in an agile development environment until you lose it and go back. Oh, it was just awful. I felt like I was running at one-third speed, slo-mo. But luckily Autodesk was catching up with the rest of the industry, and they decided to go agile as well. Because I was complaining a lot, I was put in charge of the agile transformation for our division. I was the lead agile coach for that transformation.

Revelation two: friction at the agile boundary

[00:17:04] I made lots of mistakes, which I'll tell you some of. But one of the things I realized when we were doing this, and this came to my second revelation, is that there's always friction between the parts of the company that are agile and the parts of the company that aren't agile. For example, when development went agile, we'd ascribe it to the kind of friction that design was having with them at the time, but frankly there was friction with every other part of the company too. Marketing would say, well, we need to know what features you're putting in by the end of the year, before we do the big release, so we can write marketing materials. The same thing with the documentation people: you have to freeze the UI for months beforehand so we can write all the documentation. And the localization people: we have to translate this, so you need to freeze everything. But the development team was saying, well, we're working agile, we don't know what features are going to be in, we're changing direction, we're learning as we go along. That caused a huge amount of friction and fighting.

[00:18:01] Of course, once we got dual track working, we were able to move ourselves over, and suddenly we no longer had friction with development, but we were having friction with everyone else as well. Which takes us to 2011 and the publication of The Lean Startup. I hadn't heard of this book. Desirée, my colleague, said, John, you've got to read The Lean Startup. I said, what is it? And she goes, it's agile for product management. And that's really what it is. If you look at The Lean Startup, which is of course a way of doing business discovery and figuring out how to build a new idea, product or service, it does it in a way that hits all of these agile attributes. It does it with time-boxed experiments with hypotheses, incrementally, collaboratively. It's agile for product management.

[00:18:48] That allowed product management to move to the other side of the friction boundary, and that's where we want to be now, with these three things going on. That's what this conference is about, as it turns out. There are ways of moving everyone over here. Right now our localization team can work agile, with incremental translation, and we've worked out other ways of doing this stuff.

How the model has evolved

[00:19:08] Since we came up with this process and it's been spread around, there have been a lot of improvements, of course, that people have made. Originally we called them the interaction design track and the development track, but somebody, we think it's David Hussman, came up with the names discovery track and delivery track, and I think that's way better. You have one track of work where you try to discover what it is you need to do, and the other track where you're actually building and delivering it. I think that is a way better way of thinking about it.

[00:19:36] Marty Cagan made this generalized model of it, which he's published. Again, this isn't a particular methodology; it's just a model that fits a lot of different things like this. He points out something really important which I didn't mention: not everything makes it from the first track to the second track. Part of discovery is learning what you shouldn't do, and that's a key thing as well.

[00:20:03] Carol Smith, Thyra Rauch [?] and Hannah Moyers [?] just did a paper this year expanding dual track into triple track. They're postulating that you should have another track which is just research, to build up knowledge, that feeds into the problem-solving track, which then leads to the actual delivery track. It's an interesting paper. I'm not sure I would do it this way, but Carol and Thyra are super smart. I don't know Hannah personally, but it's probably worth looking at. And Jeff Gothelf, just today actually, tweeted a high-res version of his model that matches UX and agile, his way of seeing it, which is a variation on what's going on. I don't have a slide of that because he just tweeted it today, but if you follow Jeff Gothelf you can look at that.

[00:20:53] I was looking at, of course, the UXDX model. I pulled this up and I was considering it. What you have here is again the same build-measure-learn loop, but you have it for product, you have it for development, and you have it for deployment. We work in packaged software, so we don't have deployment the same way those of you who have web services do. But it's the same build-measure-learn loop and it's all synchronized, so it's the same idea brought out to a larger scale. My chief architect saw this image and said, leave it to an Irish design conference to have a shamrock-shaped model.

Revelation three: culture eats process for breakfast

[00:21:31] Before I finish up here, I want to talk about my last revelation, and this has to do with when we deployed it, when I was the chief agile coach for the division. The way we chose to do this was to say, okay, we're going to come up with a process, we're going to teach everyone the process and impose it on the whole division, bang. What we learned is that for some teams that worked beautifully. Some teams ran with agile, they did great. And other teams just seemed to hit the walls, and it didn't work well, and I had to really come to understand that. That was basically a mistake I made, or that we made; I'll take my part of the blame for it.

[00:22:05] What I learned out of that is our third revelation: you can have a good process, but culture eats process for breakfast. If you try to put in a process where the culture doesn't match, culture wins every time. You can't fix culture with process. I'll leave you with a few points which I think are useful on that one.

[00:22:29] I think the right way to do this is to know that practice grows in fertile soil. You want to find a place where people want to pick up a new practice. Imposing a practice from outside can cause a reaction where people say, don't tell me what to do, I want to work the way I'm used to working. So you need to find someone who's keen on doing it. Instead of imposing it, you want to find people who are champions of the new process. You want to find a team who says, we want to work agile, we want to work this way, let's do it. You let them be successful with it, and then other teams will want to copy them. So it's find these champions, then clone this out.

[00:23:09] Beware of heroes. Agile is fundamentally an anti-heroic thing. The team is the hero; people work together. If you have that one person, especially the one person who draws a lot of their social standing from being a hero on the team, you know, oh, Bill worked really late and wrote this magic code, that can be poisonous for an agile team. It has to really be teamwork, it has to be collaborative. If someone is used to getting their social standing that way, they may deliberately undermine the process, and that's what you don't want to happen. You want people to draw their strength from the team.

[00:23:51] You always have to be careful: any time you're imposing a new process, you're changing the power balance, and that can be threatening to some people. I've seen teams that have a strong development background, where developers always had all the power and everyone else was serving them. There are other places I've talked to where it's the other way around, where design is driving everything and developers are just there to write whatever the designers come up with. Both of those things are unhealthy. You need a balance of inputs on the team to come up with the best decisions and make the best choices.

[00:24:22] You have to watch your incentives. A lot of companies incentivize individual work rather than teamwork, and you can actually undermine teamwork again. If people think, oh, there's only so much of a bonus and we're dividing it between members of the team, then why would I help my team succeed, if I can make myself look good? So your incentives have to line up with everything else we want to line up.

Inclusion and choosing scrum masters

[00:24:46] And inclusion is super important in any of this stuff. The reason these processes, where product and development and design work together, work well is because all three of those groups bring a different perspective. A lot of people don't understand diversity and inclusion. A lot of people think, oh yeah, that just means we have different cultures all at the same table, and that's not the point of it. The point of it is making better decisions for your business, and there's a lot of research about this.

[00:25:17] Here's the thing. If you bring different people with different perspectives and different knowledge into a room, and you sit them down and get them talking together, naturally human beings will gravitate towards talking about the knowledge that everyone has in common. And it turns out that's not very useful, because everyone already has the knowledge they have in common. That doesn't help you make better decisions. So inclusion is about practices you can do to make sure that diverse people, the diverse opinions, get heard.

[00:25:49] It turns out the research shows a few things that make this work. Number one, there are different meeting practices that get people to share their opinions individually before a group discussion starts. You can do things like get people to write their thoughts down on Post-it notes, so everything gets posted, and that will get the different opinions out. But it also turns out there are necessary conditions. One of the conditions is that everyone in the room has to respect that other people have information that they don't have. If you come from a culture where somebody thinks product management knows everything, then they don't respect the opinions of others, and your diversity will have no effect. If you come from a culture where everyone's like, I know my bit, but I understand that designers know things I don't know, developers know things I don't know, and you all work to get that unknown stuff on the table, you will have good decisions. That's the difference: diversity is having the right people in the room; inclusion is your practices so that they can all contribute fully, and that will lead you to better business outcomes.

[00:26:47] My last point is to choose your scrum masters wisely, because we've had problems before. Usually, when people are starting an agile practice, they don't want to have full-time scrum masters, so they'll make the development manager the scrum master, or whatever. But the scrum masters are the ones who own the heart of the process, and they're the ones who have to care about all this stuff. So that's the important thing: when you are choosing scrum masters for your team, choose people who really, really care about making it work. To the scrum masters out there: you guys are heroes if you're doing it right. That's all I have to share with you today. Again, that's me, and I'm happy to take questions if we have a few minutes left.