Speaking with John Schrag, Director of Experience Design

Autodesk is a huge company with many divisions. I work with the division that makes tools for the media and entertainment world – for the people who create movies, animations, special effects, games, and architectural visualizations. We make some cool software, but it’s our customers who make magic with it.

What was the challenge that you faced?

In 2001, I worked at a computer graphics company in Toronto called Alias, which was acquired by Autodesk in 2006.  Alias was an early adopter of Agile, bringing in Jim Highsmith (one of the signatories of the Agile Manifesto) to train everyone in Agile methods.  He revolutionized our software development practice, but Agile as he taught it at that time did not involve any kind of design.  That was a big problem for our design team, who up until that point was well-integrated into our previous waterfall-ish process.

While our developers were really happy working Agile, we found ourselves frozen out, no longer fitting into the development process or cadence.  For example, we might do a usability study, but by the time we were done, development would be three sprints past that work. Or we’d be in the middle of designing something large, only to discover that dev had changed direction during their last sprint meeting and our work was wasted.  We couldn't inform their process at the points they needed the information, so our work was wasted, and they were building undesigned software, with predictable results.

It wasn't just designers that were feeling the pain – there was friction between the Agile development teams and all their non-Agile partners.  Marketing demanded to know what as going into each release months ahead so they could write marketing materials. Documentation and localisation team wanted the UI's frozen long before release so they had time to write and translate the required documentation. But the development team was saying: “We're working agile, so we don't know what features are going to be in. We're changing direction, we're learning as we go along,” and that led to a lot of friction.

How did you resolve that friction for the design team?

As designers, we could see the benefit of Agile for developers, but didn’t know how we could fit ourselves in.  Previously, at the start of a project we would talk to hundreds of users and create a lot of artefacts: test reports, personas, use cases, task analyses, etc.  It might be several months before we'd start building anything, and there was not much course-changing until we got to usability testing.

For usability testing, we would wait until we had something significant to test, and then we would create our prototype, plan a test, the protocol, recruit users, bring them in and run the test. We would write a big report at the end that listed a hundred things that went wrong and give it back to development.  That could easily take three or four weeks.  Front loading and big projects were not an option.

Finally, we came to realize this: to serve agile, design had to become agile – valuing conversations over documentation, collaboration over contracts, and working in time-boxed, iterative ways in sync with development.  So we started to figure out how to do that.

How did you change your ways of working?

We realised that as designers, we're not working on the same thing the developers are working on at the same time. We're supposed to be in front of the train; we're laying track, we're clearing bushes, making a way for the train to go.  But we're also supposed to be behind the train, validating completed work, making sure everything went the way it was supposed to go.  That couldn’t fit in a sprint.  We needed to be desynchronised from development, and this brought us to the dual-track model.

Here is how we set it up originally:  In sprint 0 of a project, developers focus on back-end work or tech sprints, which gives design a bit of time to look ahead to customer-facing work.  Designers spend the sprint looking at upcoming stories in the backlog,  validating their importance, designing UX, and usability testing paper prototypes. At the end of the sprint, we have validated designs for development to work on in the next sprint. So design is always working one or two sprints ahead.

We completely changed how we did usability testing. Instead of waiting until we had something to test, we scheduled regular recurring tests, bringing in five users every two weeks.  We built a pool of about 40 users that our recruiter could just pull from regularly.  We never knew what we were going to test until the day before.  When the tests came up we’d look at whatever we were working on – paper prototypes from the designers, implemented code from the developers, research questions - and we would just test whatever was ready. We'd invite the team to watch so that everyone could see to build trust in our recommendations.

We also changed the way we reported our test results.  We realized that there was no point in writing a big report listing 30 issues, because the developers would only have time to fix the top 3-5 problems.  And once those were fixed, the user behaviour would change, so you had to test again anyway.  So we started delivering next-day reports, listing only the top five issues to fix.  Our 3-week test turnaround became 2-3 days.

We changed the way we did design, switching from big detailed design docs to one-or-two pages of diagrams and callouts, supplemented with face-to-face conversations.  We stopped doing “Big Design Up Front”, and instead did just-in-time design – but always with the long-term future in mind.  We found ourselves spending less and less time making documents and more and more time iterating and improving.

What structural changes did you make?

As much as possible we tried to build cross-functional Agile teams, with embedded designers.  One of the key things to being successful with Dual-Track is understanding that it is a single team doing two tracks of work in parallel – not two separate teams handing things off to one another.

For example, developers have to be involved in design because they're the ones who can determine the feasibility of building it.  Similarly, while the developers are building the actual features, designers need to be involved, first to keep them on track with the design intent, and secondly because the developers are going to learn things (“oh, the API doesn’t support that”) and design modifications may be required.  There was a lot of walking around and looking at each other’s work.

Did you face any challenges along the way?

After our success with dual-track, we began to teach it to other teams at Alias, and then we taught it at conferences to people from hundreds of other companies.  After Alias was acquired by Autodesk, I was given the job of coaching teams across our division in our dual-track Agile process.

I made a number of rookie mistakes rolling this process out to a whole division. My first very naïve error was in assuming that this kind of change was simply a matter of training.  Teach the new process, everyone adopts it, and you are done. But of course Agile is not just a process change, it’s a cultural change.  And culture is a lot harder to shift.  To misquote an old saying, culture eats process for breakfast.

For some teams we trained, dual-track worked beautifully, whereas others team seemed to hit walls implementing it.  Things were especially hard in teams where the new process would threaten existing power dynamics.  At our company and at industry conferences, we’ve talked with people on teams where developers assumed they were the smartest people in the room, teams where PMs thought of developers as stenographers for their own whims, and (more rarely) teams where designers were lionized.   All of these imbalances are unhealthy.  For Dual Track to work well, you need to have cross-functional teams where members trust and respect each other’s expertise, and are comfortable sharing influence and ownership.

Another cultural issue we’ve seen is teams with a ‘hero developer’ – one individual who is seen as the unique rock star, and who gets a lot of social standing from that. Over-valuing one person’s opinion can be poisonous for a team dynamic, and result in sub-optimal decision-making and the disengagement of everyone else on the team.  In the worst cases, the ‘hero’ is allowed to get away with toxic behaviour because of their standing.

Another thing to be careful of is incentives – does your merit system reward teamwork, or individual contribution only?  If so, you could be incentivizing people to put their own status ahead of team success, which can be a problem.

How has Dual-Track changed over the years?

Originally we saw ourselves fixing a unique problem between designers and developers.  We even named our two tracks of work ‘Design’ and ‘Development’. But solving that problem just moved the line of friction.  Over time, ideas like Lean Startup brought PM and other roles into an agile way of working.  Now we call our two tracks “Discovery” and “Delivery”, which gets to the heart of what they are really about.  (Thanks to Marty Cagan for those names.)  Our teams still comprise developers and designers, but also PM, content designers, QA, and UX Research.  Everyone is involved in both the Discovery and Delivery track activities, but not to the same degree.  Discovery activities help us filter out ideas we shouldn’t build, and de-risk and validate the ones we should, all in service of the Agile idea of “maximizing the work not done”.  We’ve changed the way we track Discovery activities in our tools several times – we haven’t yet found a clear ‘best’ answer to how to do that, but we do try to keep them separate from the Delivery activities.

What’s been a big help is how Autodesk in recent years has made a big investment at driving culture improvement from the CEO down through the entire organization.  We place a lot of value on things like fostering psychological safety in teams, which is key to the success of this kind of collaborative process.  This is a skill that every manager is expected to have and model.

And like many organizations, we’ve been investing more into Ops – dev, design, and research – to help us scale and keep the core teams focused on bringing value to their customers.

What outcome was achieved?

Working dual track has given team more team members visibility and a voice on ideas as they move from roadmap to delivery.  This allows us to foresee issues earlier, predict capacity problems, and eliminate low-value or overly-risky work before we invest too much in it.  Better decision-making leads to better products and services.  Of course, no process is ever perfect, and we continue to evolve it as time goes on.