Creating A Development Process That Fits Your Organisation

Bjorn Sundman at UXDX Community: Stockholm. Video: https://youtu.be/TDgxm0lojrI

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introducing myself and Loop54

[00:00:13] I wanted to talk to you about creating a development process that fits your organization. Can you hear me okay? Yeah. And more specifically, about how we did it at Loop54. But before I do that, I want to introduce myself. My name is Bjorn Sundman and I'm the VP of Engineering at Loop54. I have a background as a computer scientist and a backend developer, but for the last 12 years I've worked in various technical management roles such as product manager, CTO, tech director and VP of engineering. And, also relevant for this whole case, I'm also certified as a scrum product owner and a scrum master.

[00:00:58] What I want to achieve with this is to give you some ideas on how you could maybe be tweaking or improving the way that you work with your team. And maybe I wanted to ask you, by the way: how many of you use scrum or Kanban in the way that you work with your teams today? How many don't? Well, that's pretty telling. We don't, actually, but we started with scrum, and that's what I'm going to tell you about. First I want to introduce the problems that we experienced using scrum, and what we did instead, and to be pretty hands-on and maybe give you some ideas on stuff that you could ask yourself afterwards, to see if that applies to you.

[00:01:47] But to begin, I want to establish our starting point, and I want to introduce you to what kind of organization we work in, what product we are building, and what the team looks like, and then move on specifically to talk about the problems we had. So, Loop54: how many of you have heard of it? A few, then. Cool.

[00:02:06] We run a search engine for e-commerce. That's the only thing we do. It's pretty complex, it's built on in-house AI technology, and it's staffed by PhDs in mathematics, linear algebra[?] and neural networks and so on. On the scale of technical complexity we're pretty high up there, at least we like to think so.

[00:02:33] With a product like that, a lot of the things that we work on and a lot of the innovation that happens in the product springs out of the development team, because you have to have a very detailed and thorough understanding of how everything works in order to be able to actually come up with ways of improving it. And we're a small team, six people. We manage everything from our hosting, our operations, our infrastructure, developing the AI algorithms, front end, backend, data processing. So it's a pretty wide spread of responsibilities, and to counter that, of course, we have a team with a very wide set of skills and experiences.

The problems we had with scrum

[00:03:28] Moving on to our challenges. I joined Loop54 two years ago and the team was using scrum at the time, and we kept on doing that, and pretty soon it became apparent to me that there were some issues. If you're familiar with scrum, then you know that a big part of scrum is your planning meeting. You plan your next iteration, and in that you try and fill your sprint backlog with enough stuff that you think you'll be able to do, and you look at how much stuff did we do last iteration. We were using story points. I don't know if that's maybe old, but we were.

[00:04:10] What we found was that the amount of just stuff that happened in a two-week period was so much that when we estimated it and put it in the sprint backlog... And what I mean by stuff, I mean the things that happen: a hard drive is failing, something's wrong for some customer, something has happened. Not the things that we wanted to focus on, but things that needed to be fixed, otherwise things would break. That accounted for, on average, between 50 and 80 percent of our sprint backlogs.

[00:04:47] So, combined with this, we had at least three major things that we really, really wanted to focus on. Really, really important things that would take our product to the next level, things we knew that we needed and had to do. But the reality was that we were left, each sprint, with something like 20 to 25% of the entire sprint backlog to squeeze that in. So yeah, that's what we did, and that's how it ended up, and it just didn't feel very good.

[00:05:18] And at the same time, as I mentioned, I had a very diverse team that did different things. A lot of times in these meetings we ended up having really lengthy discussions that weren't concerning everyone. So we would have really long discussions about infrastructure, and it's quite a backend-heavy product, so my backend developers really got into it and talked about details, and my front end developer and my PhD in AI maths were like, why am I here, why am I listening to this, this has nothing to do with what I'm good at.

[00:05:51] And also, with that 25 percent that's left, how do you make a good plan that gives everyone in this team something of value to do? That was really challenging for us. I know we like to talk about cross-functional teams tonight, and there's some sense to that, but for us the cross-functional part is just the reason why we had to do a few things at the same time. They were very far apart from each other, so somebody had to improve the AI, another person had to improve our infrastructure, and the other one worked on our customer portal, and they just didn't really have anything in common. And yet we needed to do them, and we can't do one at a time.

[00:06:40] So yeah, that's where we were. And what do you do? Well, the consequence, to begin with, is people who were really frustrated. I was frustrated, everybody in the team was really frustrated. We really wanted to get on with these important things, and yet we found ourselves with just 25 percent of what we thought was our capacity to do that. And also the consequence is we had low progress. We really didn't focus on the important things. It wasn't just a feeling, it really happened: we struggled to do the things that matter to us.

[00:07:13] And when the product manager came and asked me, when is this important thing that we want to do getting done, I didn't really have a good answer for him. What am I going to say? And he was part of the meetings too, so he didn't really ask me, he knew. And my team were pretty unhappy. They didn't feel like they were moving forward, they didn't feel like they were developing personally, they didn't feel like we were moving forward. And that's not a good feeling to have in an engineering team, in my opinion.

Ten insights

[00:07:46] So we thought about what do we do instead, and we thought, let's try to do it our own way. And where do you start? I had a lot of regular one-on-ones with all my team members, and we also have regular retrospectives, so there was a lot of information boiling over, ideas. I kind of wanted to put them down and present them to you. So I have 10 insights. Each one on its own, they seem quite simple.

[00:08:36] The first is that understanding all aspects of a complex enough problem takes time, requires focus, and is a continuous learning process. And in addition to that, in order to genuinely do something good, something that's good in the long term, you need to fully understand what you're working with. Thirdly, you can only work on very few things at once and still do it well. I'm sure you've all been a victim of trying to work on more things.

[00:09:07] And fourth, goals are not always obvious to everyone, and I think it's very, very easy to forget this. Once you've internalized a goal yourself, you tend to assume that everyone else has the same goals you do, but that's not true. And what's also true is that people create their own goals. If you don't give them goals, they'll find their own goals and they'll work towards those, but you get a lot more out of it if you try and have a shared one.

[00:09:38] This one is sort of my personal — well, I like this one — developers like to design and build good solutions. I think that there's a lot of power in that, and that's what really made me want to design the process better and unleash the force of developers. You should be cheering in the back, developers.

[00:10:09] Working together also results in better solutions. Not only that, it increases sharing within a company, and it makes you more redundant when people are on vacation or sick or whatever. There's usually no way around problems other than fixing them. It sounds super simple, but that's really the case, and especially if you have a problem in your production environment. They're not going to go away, they're going to keep being problematic, and you just need to fix them.

[00:10:40] Estimates are unreliable, and that's just the truth, whether you like it or not. And I think, combined with this one — if it's important you should do it — it means that if you think something is important, don't worry so much about estimating it, just worry about getting it done in the right way. And of course this shouldn't be interpreted to mean that everything you could ever think about that could potentially be important should be done. It just means that you should do the most important things, or preferably the most important thing first, and then after that the second most important thing.

[00:11:13] And this is my last insight: time plans are a good tool for picking the right discussion. We don't believe so much in deadlines. We work with deadlines when we release something, but it's not nearly as important as the quality of it, so we focus more on quality. But having a time plan for what you're trying to do raises a lot of the right discussions, and it can help you talk about things like scope, or if something is more or less unclear, or if you need more resources on a problem.

Trails

[00:11:47] So, taking all of these things together, we created something that we call trails, and this is how we describe it internally. Trails are a focused development effort with a specific goal, where a smaller group of developers work in a dedicated way towards achieving a result, with high degrees of freedom and responsibility.

[00:12:09] And more specifically, all trails have a clear goal. Two to three developers. No developers are on more than two trails. They have a very lightweight steering document with a time plan, a list of what to deliver, a definition of the goal, and the limitations. But importantly, they're owned by the developers. Basically we give the developers in the trail a goal, they come up with a solution, they come up with a time plan, and then of course we talk about it. So the product owner and the engineering manager, being me, we discuss this, and we discuss this regularly, and we follow up and see where's the progress, where are the pain points, do we need to adjust the time plan.

The other stuff that has to happen

[00:13:01] So as you can see, the trails are our solution to how do we get on with the things that matter to us. And then of course the other stuff that I mentioned earlier didn't just go away because we invented something called trails, so we still need to address that somehow. And we just said, that's not a trail, that's just stuff that needs to happen, and basically we just did it. Whenever there's something that's important that needs doing, we don't spend so much time talking about it, we don't spend an awful lot of time in meetings discussing whether it needs to be done or not, we just did it.

[00:13:35] There's a fast lane for the product owner, so sometimes he cares about things that we're not that worried about or aware of, and he tells us this needs to get done pretty quick, so we do that. But we also gave the power to the developers to make judgment calls, so on anything that's a smaller issue to fix they could just do that without really asking us. Also, whenever we had any kind of incident or something that impacted us in production, we fixed that right away.

[00:14:11] So, to put this all together, we meet weekly in the team, we review our roadmap, we look at what are the different trails going on right now, what are the trails that are coming up in the future, just to give people an understanding of what's going on, to give people that weren't on a specific trail knowledge about what's happening in the other trails. And it's also a good time to reiterate the goals of each trail.

[00:14:42] In addition, we allocate 20% time to fixing technical debt, or innovation if you're working on your own ideas. So every fifth week developers are basically free to fix the things that bother them or the things they care about. And we visualize all of this on a Kanban board, so that the current state of what people are working on is always visible to everyone in the team, or everyone in the organization for that matter.

[00:15:12] Just to draw it out simply, what it looks like coincidentally looks a lot like a roadmap planning document, and it also makes it super easy to do any kind of longer-term plans. You can see here that developer A, for example, is busy on trail 1 and 3, and then as soon as trail 1 ends they can start trail 2. So the developer on trail 2 can't start earlier because of developer A, as you can see here. And then at the bottom you have this continuous non-trail stuff, work that just kind of happens as it needs to.

[00:15:44] It's not difficult to get developers to work on the trails, that's what they want to do. The other stuff they do because they have to, so there's no reason that it's going to overrun.

What the data and the results look like

[00:15:58] This is data from 2018. We've done this for a year and a half now, and the graph shows you the amount of hours spent on stuff that was finished, and the color represents the trail, or in the case of blue it's actually no trail, so that's the other stuff. It's a stacked graph, so what's accumulated is the entire amount of things that were done. But you can see that blue is still what we do most of: 40% for that year. And 40% is a whole lot less than how we experienced our sprint plans to be. It's quite spiky. That's partially because things only get accounted for when they're done, but also the blue is spiky by nature, so sometimes we have more problems to work on, sometimes we have fewer problems to work on.

[00:16:57] And the result: it's really hard to know what things would have looked like if we hadn't done this. I can't A/B test this as much as I'd like to. But the truth is I have a very, very happy team, and they were quite outspoken about not being happy before, so I'm very confident that they would tell me if they weren't. We have a really strong feeling of making progress, and it's not just a feeling, we're actually making a lot of progress. We've delivered five important trails in a year. I hesitate to use the word timely, because we're not working with timeframes or deadlines, so we can't be on time, but we are releasing good stuff at a good pace for the team size that we are, and importantly, with high quality.

[00:17:52] Now we have a drastic reduction in the amount of firefighting we do. I didn't show you 2019's data, but our constant struggle towards just fixing bad things as we encountered them has created a situation where we get a lot more focus on the important things. So we really changed it from being quite reactive and focusing a lot on firefighting, to a situation where we can work a lot of our time dedicated on the things that matter the most to us.

[00:18:35] I encourage you to challenge the way that you work with your development. I think this is the best team I've ever worked with, and I'd like to think that part of it is because of the way we work. There's a lot of developer creativity, people are very engaged, they care a lot about what they do, and we produce great results. And that's how I think engineering should be done.