How Long Is A Piece Of String?: The Key To Solving The Conundrum Of Software Estimation

Jonathan Rigby at UXDX Europe. Video: https://youtu.be/xgyK5QAPrGQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

How long is a piece of string?

[00:00:00] I've entitled my talk this morning "How long is a piece of string?" If you were to Google that as an expression, you would get a definition something like this: it means that the quantitative answer is not known, and given the information we have available, it's going to be difficult to come by. In my experience, I think that's a pretty good definition of what it's like to do software estimation.

[00:00:33] I'd like to start with a simple exercise for us to think about. Imagine we take a deck of playing cards, 52 cards. We split them in two and put one part face down on the table and the other part face up, and then we mix them up. If I asked you to estimate in seconds how long it would take to do the following, what would you say? First of all, I want you to sort the cards face up into their suits. And secondly, do the same exercise, but put all the cards in the suits in numerical order.

[00:01:19] I'm a bit intrigued. How long would people estimate for the first task? Just shout out some numbers. A minute? People are nodding, a minute. Anybody else? Two minutes, okay. What about the second one, a bit more difficult? One more minute, two more minutes, five minutes. Okay. Why is there such a range? We're all very familiar with cards, we're very familiar with sorting things, but we've seen a real range in those estimates. And if we were to do a project with this, which value would we go for? You might like to go home tonight and try it out and see how accurate your estimates really are.

[00:02:13] Sometimes it's hard to work out how long it's going to take to do something, and we have this situation a lot of the time when we're doing software development. We're given a problem to solve, maybe a feature to develop, and we're asked to estimate how long it's going to take.

A painful story

[00:02:29] I want to tell you a story born out of a very painful experience. A software company had signed a contract with a blue-ribbon client, and this was to take the product in a totally new and exciting direction and open up very many possibilities for the future. At the beginning, a scoping exercise was undertaken, a number of epics were defined, and these were taken to the software team: "We need you to size these. Rough estimates are fine for now, but we need them by the end of the week."

[00:03:06] After the protestations had died down, about the lack of time, the lack of information and the fact that this was in a technology nobody was very familiar with, the team did their best and came up with an estimate range of six to twelve months. "Hmm, that's no good," came the response from the CTO. "We need it done by the end of the year. But okay, because of the uncertainty, I'll give you an extra four people." Well, four people would be great, but what about the time it was going to take to recruit and to train? And by that point, the company had, amid great fanfare, announced to the whole world that the product was going to be delivered on the 31st of December. Just a side note to anybody involved in project management here: that's clearly a bad day to pick, let alone doing it without the software team's agreement.

[00:04:02] I probably don't need to complete the rest of the story. You could probably work it out for yourself. Actually, the product was delivered in some form by the end of the year, but it involved very many people working many weekends and over Christmas to achieve it. Plus it dragged in people from other projects, impacting them too.

[00:04:27] The longer-term ramifications were more severe. Over the next six months, most of that software team left the organization, and relationships between the product and tech teams broke down so much that when any subsequent estimates were asked for, the developers would just massively inflate them, because they didn't want to be in that situation again. And the product team would view any estimate that came their way suspiciously. I expect most of you will have your own version of that story, probably with very similar outcomes.

The ways we try to estimate

[00:05:03] Software estimation is hard. I know that, you know that, but maybe other people in your business don't necessarily know that. The CEO, for example, just wants to know when they're going to get something, and we just go along with it, because they're the CEO. So we create a raft of different ways of trying to estimate more accurately.

[00:05:28] For example, we might do some of the following. We'll use our expert estimators, a principal developer or a technical lead. Of course, the challenge with that is they're not always the people who are going to do the work. We might count things, use cases, stories, features, and multiply them by a common factor. Let's say we know there are 10 stories in this particular feature, and we roughly do a story a week. Okay, we know it's going to take us 10 weeks, don't we?

[00:06:02] We might compare with similar historical projects. We've done something like this before, haven't we? But do we really remember that project in detail, and how similar can two software projects really be? We might use ranges to encapsulate our uncertainty. That's fine, but depending on your personality type, you might fixate on one end or the other of the range. "How long is it going to take?" "Three to six months." "So you can do it in three months?" "No, I didn't say that."

[00:06:38] We'll go with our gut feel: "Oh, I've done this before. I know what this is. That's just a couple of months." Or, "It's just a one-line fix. How hard can it be?" We may just pad the estimate: we'll just double it to be safe. We might use t-shirt sizing: small, medium, large, extra large, extra extra large. Agile and Scrum brought us story pointing as a way to abstract the time it was going to take to do something, but you know that somewhere in the background somebody's converting those story points to days, weeks, months.

[00:07:17] If we're really keen, we can do Monte Carlo simulations. At the extreme end of the scale, there are things like this: complex formulaic ways of determining software estimates from a series of input parameters. I like this one. This is the COCOMO II model, and it takes over 20 different input parameters and spits out a software estimate at the end. It takes parameters such as programmer capability, process maturity, the language you're trying to use and even team cohesion, if you could put a value against that.

[00:07:52] We try all these things, and what happens? We still get it wrong, don't we? Maybe that's just because of Hofstadter's law.

No estimates, and a third way

[00:08:05] It was out of this that the NoEstimates movement started some time ago. The premise was simple. Software estimation is hard. We've tried many different ways and we're still getting it wrong. Lean methodology tells us to eliminate waste, and if software estimation has no value, the time spent trying to do it would be better spent writing the software we're trying to estimate.

[00:08:34] A colleague of mine had a team who were very diligent in producing software estimates, but inevitably, at the end of some of the sprints, they were out, and it was the worst sprint review you could imagine. You see, their product owner was a former lawyer, and they wanted to know why every three-point story that had become a five-point story was so. So this technical lead used to spend hours coming up with all the mitigating factors for why that had happened. It was just a waste of time.

[00:09:06] So I have a lot of sympathy with the NoEstimates line of reasoning, because of situations like that. And I'm sure you've been in that same situation where you've been in a meeting, you've been given some really sketchy requirements, and you're asked to come up with a high-level estimate. We protest, we caveat the life out of it, but what happens? Those estimates just become reality.

[00:09:37] Now, who loses out without our estimates? Project managers couldn't produce Gantt charts, and where would we be without those? No, seriously, I believe there are times when we need to give an indication to the business of how long something is going to take, so that they can make relevant decisions on how to prioritize things.

[00:10:00] So software estimates are hard, we know that. The NoEstimates way is tempting. But is there a third way? I believe there is. Having been involved in these types of situations many times as a development manager, I've come to understand that the key to solving the conundrum of software estimation is trust. For sure, we must use those estimation techniques we talked about earlier, but having trust trumps all of those things.

[00:10:36] Trust is obviously the key to any relationship. I think we'd all agree on that, and it's certainly true for the relationship between tech and product. Now, don't get me wrong. I'm not saying we can throw everything else we've ever done out of the window and just base it on trust. Please don't go home to your teams or your business and say, "Hey, I went to UXDX, there was a talk on software estimation, and the guy said we don't need to do any more of that, we just need to base it on trust." No. I think we need to do all those good things, but we need to underpin them with a foundation of trust.

Five ways to build trust between product and tech

[00:11:17] Okay, he says trust, but what does he mean by that? I can't define what trust will look like in your particular relationship, but what I want to leave you with is five things that can help build or maximize trust between the two teams. For each, I've got an underlying principle, and I'm going to look at what it might look like for both product and tech teams.

[00:11:41] Firstly, making and keeping commitments. For the tech teams I manage, I want them to make the commitments they can, the ones that are knowable, the sprint commitments, and deliver on them. I also want them to give quality demos. Product teams, as far as possible, fix the scope. Push back on your stakeholders who are asking for more. It's really hard for a development team to restart midway through what they're doing because something's changed again. Of course it's not always possible, but make it a key aim.

[00:12:24] Secondly, be transparent. Tech teams, if you don't make your sprint commitments, for whatever reason, then be transparent. Don't hide behind tech speak. If you've had a shocker, just call it out. Product teams, there'll be times when the stories you've written are ambiguous or have missed out some details. When that happens, just hold your hands up and say sorry. When we fail and we're open about it, it creates psychological safety, which is really great for our relationship.

[00:13:05] Thirdly, don't overdo things. I want you to understand and live out the commercial aspect [?] of the business. What do I mean by that? A lot of us have to understand that we're in a business that's there to make a profit, or, if we work in a not-for-profit organization, to create value for others. Tech teams, we don't help ourselves, do we, when we gild the lily, when we try to write that new feature in some funky new complex technology just because we think it'll be fun. On the product side, work hard to go for MVP. Do you really need all those bells and whistles? I'm no product expert, but I see a real shift towards simplicity when we think about product and product design. Make this a goal.

[00:13:54] Fourthly, show empathy. Tech teams, your product teams inevitably have a lot of stakeholders who all want to know when something is going to be done, and they're in the difficult situation of having to convey delays without having any direct influence over the software, not as much as you may have. I've heard stuff like this, so please don't say things like "I can't tell you how long it's going to take because we're doing agile", like it's some sort of magic veto card. Product teams, similarly, don't say things like "Can't you do it quicker?" or "Surely it can't take that long, you're only adding a button", or "My other team did something very similar and it only took half the time." Don't be like that lawyer.

The bad step, and why this matters

[00:14:43] Fifthly [?], educate ourselves and our leaders on the complexity of software estimation. For example, use thought-experiment analogies to try to convey the complexity. One I like is: imagine how long it would take to add a cantilevered swimming pool between the 30th and 31st floors of the Shard. Because I think that's what we're doing with our software architecture sometimes.

[00:15:08] Let's take a more tangible example. A while ago I went on holiday with my wife to the Isle of Skye, and one day we decided to do a trip to a place called Loch Coruisk. We knew you could take a boat there and hike back along the coast. It's a beautiful place, and if you're ever in that area, I would really recommend it.

[00:15:33] Now imagine we needed to work out how long that walk was going to take. What would we do? We'd want to measure the distance, to understand how long the route is, and if we knew we walked at a certain pace, we could easily calculate how long it was going to take us. Say we drew the lines on the map like that, and it was looking like about six miles. We knew we walked at three miles an hour, so it's going to take us two hours.

[00:16:06] Of course, you can't just draw a line on the map, because that's not how paths work, particularly around the coast. You need to zoom in to understand exactly where the path goes. Interestingly, the more you zoom in, the closer you get, the longer the path gets. At an extreme level, if you zoom in, it gets infinitely long. It's known as the infinite coastline problem. But for now, imagine we zoomed in some and found out the route is actually eight and a half miles long, and now it's going to take us three hours, which is a 50% increase on our original estimate.

[00:16:47] However, when you get on the route, you discover a few obstacles. You discover something like this. About a mile into the journey is something called the Bad Step. The path stops and then continues along a rock face, and what you have to do is crawl along it like this. When we got there, my wife just refused to go across: "I'm not going to do this." But we didn't really have an alternative, because the boat that had taken us out there had returned, and the only other route was to retrace our steps, walk nine miles the other way, and then take a bus back to the beginning. Both options meant we were going to be about 75% or 300% off our original estimate.

[00:17:45] This is how it is with software projects, isn't it? At the beginning we're given those high-level requirements: walk from A to B. We might decide to do a spike. We might decide to zoom in closer to inspect the route and see how long it's actually going to take. But there are always going to be those obstacles, those bad steps, that we couldn't have predicted at all until we were right on top of them. When we're at that point, we either have to take additional time to get across them, or we have to retrace our steps and go in a different direction.

[00:18:23] So there you go. One final warning on trust, and it's something we all know: trust isn't something that's easy to develop, something we can just switch on. On the journey there, it may be tempting to revert to relying solely on those old estimation techniques, but we really should resist. Why? Because solving the conundrum of software estimation isn't just an academic exercise, one that makes us look good if we get it right. It's because it clearly impacts people's lives significantly: those people who are up late at night coming up with all those mitigating reasons to appease their product owner, those people who are working many weekends and over Christmas because of an underestimated project. It can and it does affect people's relationships and well-being. It's serious stuff. So let's all try and get better at it. Thank you.