Taking Back “Software Engineering”: Craftsmanship Is Not Enough

Dave Farley at UXDX Europe. Video: https://youtu.be/tNsh1wHyyTk

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

From software engineer to craftsman to engineering

[00:00:00] We're here today to talk about software engineering and what that means to us as a profession. That's really what I want to explore today. In my own career, I spent the early parts of my career working in organizations with job titles like software engineer, junior software engineer, those sorts of things. And if I'm honest, during that period the stuff that I was doing bore very little relationship to any form of engineering that anybody outside of our profession would recognize.

[00:00:29] In the middle part of my career, I started getting jobs with organizations that didn't really like the term engineer so much, and tended not to use the term software engineering. Instead they talked about ideas like just software developers, software craftsmanship, if you'll forgive the sexist nature of that term, or the gender-specific nature of that term. Again, we weren't really doing engineering, so that was probably a more accurate description during those days.

[00:00:58] In the latter part of my career, I started working in some organizations that were genuinely doing some hard things. We were trying to solve problems that none of us had come across before, and really pushed the boundaries of what computers could do in some narrow areas. In those organizations, we started applying some approaches, some disciplines, that I think now are more in line with what we might think of as engineering. And again, those are ideas that I hope to explore today.

Tools, process and craft are not engineering

[00:01:25] Let's start off with what tends to spring to mind when we're thinking about the term software engineering. I think for many of us, certainly for me, the sort of thing that used to spring to my mind are pictures like this. This is not software engineering. This is to software engineering what a soldering iron is to electrical engineering. It's a tool. It might be a good tool, it might be a bad tool, but it's only a tool. So this doesn't describe what we mean when we're talking about software engineering.

[00:01:59] In other contexts, we might think about ideas like this. This is the idea of Scrum. This is a process: development process, SDLC, as some people refer to it. It's really about the techniques and the procedures and the approaches that we apply to solving problems in software. Scrum is a very good project management exercise, but have a guess how many times the documentation for Scrum mentions the terms software or code. I'll give you the answer: zero times. It doesn't really talk in those terms. It's much more about project management. It's an effective project management strategy, but it's as applicable to plumbing as it is to software development. This tool is not really a description of what we would think of as software engineering.

[00:02:46] In more recent years, the idea of software craftsmanship has come to the fore, and craftsmanship, I think, was an important step forward. I'm going to talk a little bit more about why I think that was the case shortly. But I think this was a step forward in recognizing that what we were doing wasn't engineering, and by definition software craftsmanship is not engineering. It's something else, it's craftsmanship, and craftsmanship is a good thing. It's about creativity and so on. But if you think about craftsmanship, it's limited to human-scale effectiveness, human-scale creativity. Engineering tends to allow us to go beyond that. So I'd like to explore that idea.

A short history of making things

[00:03:25] One way of thinking about these things is to think about the history of producing things. If you think about human beings making stuff since the dawn of time, then pretty much all of our history, the production of things has been based on craftsmanship. It's been based on the skills and the techniques of an individual to handcraft some wheel, flint axe and cars and so on. But it wasn't until much later that we started learning different ways of undertaking those things.

[00:03:59] In the middle of the 19th century, the first steps in what we would think of as mass production began to take place. The first example of mass production was actually in the American Civil War. The man who wanted a contract to supply the rifles to the Northern States in that war went into Congress with a bag full of components. He tipped the bag out onto the floor of Congress and asked the congressmen to select the components from which he assembled a rifle. That was the first time in history where that was possible. Up to that point, standardization of craft-based production was not sufficient to allow the random selection of pieces and assembly of parts into a whole. The tolerances weren't good enough. Each thing was individually handcrafted and was different from every other.

[00:04:55] So that was a big step forward, and then through the early parts of the 20th century and the latter part of the 20th century that effort was improved. Towards the latter half of the 20th century, though, after the Second World War, Deming went to Japan and introduced the ideas that ultimately ended up as the ideas of lean production. He worked with Japanese culture, which had already had some of these ideas in place, but essentially what Deming was doing was applying scientific reasoning to solving problems in production and product development. That's really what his work was about. And that amplified the effectiveness and the art of being able to produce physical artifacts.

Software is always a design problem

[00:05:47] Now, there is an important thing to recognize when we start thinking about these things. The history of our industry has been colored by humanity's experience up to that point, perfectly understandably. And humanity's experience up to that point of producing things was nearly all to do with producing physical things. If we want to produce something physical, something like this pointer, then the design of this thing is a tricky problem, but the really hard part is how do you make these things? How do you mass manufacture these things? How do you make sure that you've got the right sorts of plastic in the right place at the right time, the right electronic components, and so on and so on? The production of physical things is a complicated problem.

[00:06:39] For software, that's a problem that we don't have at all. And so the techniques that we apply for producing things in the real world are not directly applicable to software. Our problem is always a design problem. It's always only the problem of creating the ideas in the first place, and we should be looking to optimize for that, rather than optimizing for the production of things. We've made this mistake in our industry profoundly, and it's affected the way in which we think of, plan and undertake software development across the world. The waterfall software development approach is really a production-line way of thinking about solving problems, and it's completely inapplicable to problems in software development.

[00:07:32] So craft and mass production are certainly not what we're looking for. I think the advantage of a craft-based approach is that it's not making the mistake of assuming that we need production-line techniques to solve problems. And I think in reality, that's where most of the industry is at the moment. Where we should be aiming to be is lean production. We should be looking to use a more scientifically rigorous approach to solving problems and optimizing our processes, and making those as efficient as we can through lean-style techniques. When we start applying this kind of thinking to software development, the impact is profound. We see dramatic improvements in productivity, quality and the impact that changes have on our user base. Users like the fruits of this way of working. We are better able to hone in on what users need from us.

Software isn't bridge building

[00:08:31] I've been having conversations like this about software engineering for a couple of years now, and I get involved in lots of conversations with different people online, through social media, at conferences and in pubs and bars. One of the weird things about these kinds of conversations is I end up having conversations about bridges quite a lot. I start talking about software engineering and describing my thoughts, and people say, "Yes, but software isn't bridge building." And I think, "Well, no, software isn't bridge building." And bridge building probably isn't what we think of as bridge building either.

[00:09:04] Bridge building is going to be different if you're building the hundredth version of a bog-standard steel and concrete suspension bridge versus the first ever version of some unknown kind of bridge. If we were going to try and build a carbon fiber bridge across the Atlantic, that would be a completely different kind of exercise to building the hundredth version of some kind of concrete and steel suspension bridge. It's not the difference between bridge building and software engineering. It's to do with the novelty of the problem that we're trying to solve, and this gets back to my point. It's the difference between production engineering and design engineering. If we're building that hundredth concrete bridge, we've probably got a bill of materials. We've done 99-odd ones before, and we can say precisely on what day we want what volume of concrete in order to be able to make progress, and so on and so forth. If we are doing the first kind of a bridge ever, we're going to be doing all sorts of research and exploration to understand what's really going on.

[00:10:08] The other thing to say about bridge building is that bridge building isn't the same as house building. It's not the same as aerospace engineering. It's not the same as chemical engineering. It's not the same as any other kind of engineering. Each engineering discipline in different fields is tailored to that discipline. It's different in different contexts. So one of the things that we can be certain of is that, were we to come up with a genuine engineering discipline for software, it will be ours, and there will be attributes of it that are unique to us. It would be about creating software and not about building bridges. But that doesn't reduce the impact of applying engineering-style thinking to software development.

Engineering at different scales

[00:10:51] There's another way in which engineering is different in different contexts. It's different at different scales. In these pictures here, I'm showing you two engineered structures. The one on the left is a shed. If this was built by Ikea, you can bet that they would have engineered it to the bottom dollar in order to be able to drive cost down, and make sure that it uses the minimum materials, to make it cost-effective to sell sheds at a good price. On the other hand, if you're looking at the building on the right, that's the tallest building in the world, the Burj Khalifa. If you're creating a building like that, you're going to be doing exploration. You're going to be building models in wind tunnels, doing finite element analysis on the structures, and you're going to do experiments to measure the tensile strength of particular kinds of materials, and all of that kind of stuff.

[00:11:45] These are very, very different. This is absolutely clear. This is self-evident when we look at something like a building, or two buildings in this case, but it's less evident if we look at two different pieces of software, certainly not to the layman. If we look at, I don't know, the software that sells cakes for my mother's cake shop versus the software that flies an airplane's control system, we would expect there to be different levels of engineering involved in those two different cases. But there should probably also be a level of commonality. There's going to be more rigor in one case than the other, because the consequences of it going wrong are higher in one case than the other. Nevertheless, if we're looking at being effective and efficient and doing each one in an efficient manner, then there are going to be these common behaviors that work in every software development case. Those are the things that I'm interested in.

A definition of engineering

[00:12:43] So let's start thinking in terms of definitions. Here there's a definition from Wikipedia which describes engineering, and it mentions a bunch of words. These words tend to crop up in the definition of engineering. It's about scientific reasoning. It's about applying mathematical thinking. It's about using evidence-based decision making, and it's about working within economic constraints. For the purposes of the rest of my talk, this is the definition that I'm going to use. This is my definition of what I'm talking about: engineering is the application of an empirical, scientific approach to finding efficient solutions to practical problems.

[00:13:19] If you think about that definition, each of those words is important. Each of those words is something that we can't do without. If we're going to think in terms of engineering, we're trying to make evidence-based decisions. We're going to be reasonable, and the economics of the situation are part of the game. We're trying to do things to a certain cost, and that's in terms of performance or price or whatever else.

[00:13:45] I think when we start thinking in these sorts of terms, about these profound terms of what engineering really means, it breaks down into two different groups of problems. I had a conversation with somebody via social media a little while ago, and that person said something profound to me that really resonated with me. They said that if we were able to identify some principles for software engineering, those principles would be as true in a hundred years' time as they are today. And I think that's correct. The things that we're talking about here are foundational, fundamental. Those should be the building blocks of our discipline.

Managing complexity and optimizing for learning

[00:14:26] Following that line of thought, though, I think that software engineering breaks down into two problems. First, we need to be able to manage the complexity of a problem. We need to be able to put a cap on how complex the pieces that we are working on are, so that we can fit them inside a human head. Modern software is vast and complicated, and many systems are beyond the realm of any one person understanding them fully. So we need to apply ideas like modularity and separation of concerns and information hiding and cohesion, and all of those computer-science-y terms that describe the approaches to development, as a foundation for an engineering discipline, to allow us to make progress.

[00:15:13] The other aspect, though, which I want to spend the rest of today talking about, is really about optimizing for learning. If we are working in a discipline that is focused on design and creating innovative solutions to the problems that we undertake, then our problem is nearly all about trying to learn. It's exploratory. We're trying to discover what the problem really is that we're trying to solve. We're trying to discover how we can apply the techniques of software to solve and address those kinds of problems, and we're trying to learn how to effectively use the tools and technologies at our disposal to fulfill those needs. So it's nearly all learning, and that's where these five things come in.

[00:16:03] It's impossible for me to imagine any kind of engineering discipline in software that doesn't embody these five principles. Our approach needs to be iterative. We need to employ feedback. We need to be incremental, so we can make stepwise progress. We need to be experimental. We need to be able to try ideas out and discover the things that work and the things that don't. And we need to be empirical. We need to react to what happens in production, and really in the live situation, what's going on. So let's go through those in a bit more detail.

Iterative

[00:16:45] Being iterative is important. Iteration is at the heart of our ability to learn and to deepen and evolve our understanding over time. The sense in which I'm thinking about iteration here specifically is being iterative to navigate towards an outcome, but there are many positive aspects to working in an iterative way. Being iterative is deeply important to agile thinking, because it allows us to learn from something. We can react to what it is that we learn, and we can adapt our approaches, thinking, processes, technology and design to fulfill the needs that we've now noticed through learning. This is a powerful approach to solving problems.

[00:17:34] Being iterative also allows us to navigate towards some desired outcome. If we've got a goal in mind, we can figure out where that goal is, and then we can keep reflecting on whether the things that we are doing are moving us closer to that goal or further away from it. Even with something as seemingly straightforward as sailing a boat from Copenhagen to Oslo, the captain does not naively take a bearing from Copenhagen to Oslo and stick to that come what may. That doesn't work. In the real world, there are things that will push the ship off course. The winds and the tides will change the outcome, and the planning needs to take account of that. There will be traffic along the way. We need some reactive planning to be able to adapt what we're doing as we learn new information, as we gather new information, and that's what iteration allows us to do.

[00:18:26] Iteration also allows us to improve things over time. Step by step we can get a better and better outcome as we learn more and more about the products that we create, the techniques that we use and so on. We can refine them to get a more effective solution. Being iterative also allows us the opportunity to enhance our own skills, our own learning, our own understanding and our own technique as well. It's at the root of any process of continuous improvement.

Feedback

[00:18:55] The next in my list of five things is feedback. The technical definition of feedback is information returned to the source of an action, and feedback is at the heart of modern software development. Feedback allows us to reflect on what it is that we've done, to observe the outcomes of our decisions. And again, feedback combined with iteration allows us to then refine those decisions over time.

[00:19:30] When I speak about continuous delivery, I often use this model to describe what I'm talking about. This is a simplistic view of continuous delivery. There are many more feedback loops involved, but these are the three primary continuous delivery feedback loops. On the outside, we have the feedback loop: have an idea, get some working software into production, figure out what our users make of those ideas. On the inside, we have the tight feedback loop of test-driven development, and in between you have executable specifications and acceptance test-driven development. These, for me, are what define continuous delivery, and so it's a feedback-driven approach.

Incremental: the modular Saturn V

[00:20:06] The next thing in my list of five things is incremental. Incrementalism is about modular design, and thinking of ways in which we can cut and break the design into a series of pieces, so that we can concentrate on those pieces independently of one another. My wife describes me as a serial nerd, and she's not quite right. I'm a parallel nerd. I can be a nerd on multiple fronts at the same time, and I'm going to demonstrate that to you now by exploring one of my nerdy notions, which is software engineering and software development, with an example from another, which is aerospace, and particularly space exploration.

[00:20:46] In this picture, I'm showing you the Saturn V space rocket. This is the machine that took astronauts from the Earth to the moon and brought them safely home again. This was a modular design. That wasn't the way that NASA started out thinking about this, but it was a profound step in the advancement of the Apollo program when they realized that a modular design was important to them. In this diagram, there are two coarse-grained modules that are visible. The first part is the Saturn V itself. The job of that part of the spacecraft was to get the rest of the spaceship into Earth orbit. That's a dramatically difficult problem, to get all of that mass out of the gravity well of Earth, and so it takes all of this rocketry and fuel to achieve that.

[00:21:34] The rest of the spacecraft was split between four pieces, four modules. Here are two of them: the command module and the service module. The command module was responsible for getting the astronauts from Earth orbit back down to Earth. It was also the place where they lived for most of the trip, but its primary job was to get them safely back to Earth again. The second part is the service module, and its job was to get the spacecraft from Earth orbit to the moon, and from the moon back to Earth orbit again.

[00:22:02] There are two other modules that were involved in the process of getting men to the moon. This is the lunar excursion module. The top part of the lunar excursion module is the lunar ascent module, and its job was to get two astronauts from the surface of the moon back into lunar orbit, so they could rendezvous with the command and service modules, where a third astronaut was waiting for them. The bottom part of the lunar excursion module was designed to get the two pieces from lunar orbit to the surface of the moon.

[00:22:42] This was important. Modularity has a number of benefits. First, it allows you to decompose the problem. In this case, when NASA was constructing the spacecraft, they were able to farm out the jobs of creating these different modules to completely different firms. By defining the interfaces between the pieces, they were able to give one company the job of building the lunar excursion module, another company the command module, another company the service module, and so on. This meant that work could be done in parallel, and so it could be done more efficiently.

[00:23:17] It also gives you the flexibility to compose the system in different ways later on. When the Apollo 13 accident happened, and part of the service module exploded, damaging systems throughout the spacecraft, the astronauts were able to reconfigure the system and use the lunar excursion module, the ascent module, as a lifeboat. Against the original plans, where they would have discarded that in lunar orbit, instead they kept the ascent module and traveled back to Earth in it as a lifeboat, and this saved their lives. This flexibility is another of these fundamental properties of incremental design.

Iterative versus incremental

[00:24:02] I've used two words in the English language that are very similar. It may be confusing, and this is the best way I know of disambiguating those two terms. I've stolen this from a really good friend of mine, Jeff Patton, who uses them in his great book, User Story Mapping. Being iterative is demonstrated in this picture. We start off with a rough sketch of the system that we intend to get to, and then over time we refine and fill in detail until we get some sort of outcome that we're satisfied with. The second picture shows what incremental means. In this approach, we're going to divide the system into a series of pieces and components, and each of those pieces can be developed independently of the others, maybe even iteratively, but the system as a whole is not assembled until you've got all of the pieces. I can't imagine any software engineering approach that doesn't involve being both iterative and incremental. We need both of these techniques in order to make high-quality complex systems.

Experimental

[00:25:12] The fourth of my properties for software engineering is being experimental. Being experimental is profoundly important. This is about carrying out evaluations of something and controlling the variables so that we can learn some lessons. One of my other nerdy-isms is that I'm an avid reader of popular science, so experimentation is very close to my heart. But I think it's profoundly important in the effectiveness and efficiency of organizations. Amazon are a great example of this. Jeff Bezos is on record as saying that the level of risk that an organization takes should scale up with the size of the organization. Amazon goes to the extent of tracking each of their experiments in production. If 50% of their experiments are not failing, they up the level of risk that they're taking, because they need some of the experiments to fail in order to learn. One of the profoundly important attributes of the scientific method is that we learn more when an experiment fails than when it succeeds. It's a superpower of the scientific method.

Kennedy's moonshot and what NASA didn't know

[00:26:21] Being experimental is so important that I want to use another Apollo story to outline it. In 1961, President John F. Kennedy stood up in Congress and made his famous speech in which he said, "We're going to send people to the moon by the end of the decade and get them safely home." At this point, everybody in NASA took a deep breath and panicked, because they had no clue about how to do this. This was in 1961, and this was two weeks after the first American made it into space. That American was called Alan Shepard, and Alan Shepard was a ridiculously brave man. Alan Shepard's space trip was essentially him sitting in a tin can strapped on top of an intercontinental ballistic missile, launched a hundred miles up and then back down. The duration of his trip was measured in minutes, and the risks were enormous.

[00:27:20] At this point, the Redstone rocket that they used to launch the vehicle was commonly blowing up on the pad. So Alan Shepard was taking his life in his hands. He was launched a hundred miles up, and came down on parachutes. Two weeks later, Kennedy said, we're going to the moon. Kennedy had done what leaders do. He had this vision, and he planted a flag and said, "Let's do this remarkable thing. I'm a politician. I have no idea how to achieve this remarkable thing. You engineers, sort it out." He had this vision. He'd come up with this idea. Somebody once asked me, "Do you think that this is the most expensive user story of all time?" And I think it's a good candidate.

[00:28:03] So he'd come up with this vision for what's going on. In this picture, I'm showing you examples of the stuff that, at the point when he made his speech, NASA didn't have answers for. They hadn't really got launches sorted out. If you look at a montage of the American space industry at this point, their launches were blowing up on the pad all of the time. They hadn't done multi-stage launches yet. They hadn't got the splashdown correct. The following mission after Alan Shepard was Gus Grissom, and when he landed back, having also done a suborbital hop, he splashed down in the Atlantic, his capsule sank, and he nearly drowned in the process. Nobody had yet docked spacecraft together, essential if you're going to do the lunar rendezvous approach, the modular approach to systems that we talked about before. Nobody had yet done a spacewalk. You need to be able to spacewalk in case of emergencies, to correct the system, but also to learn what it took to have space suits you could move about in, so the astronauts could walk on the moon. Nobody had done that. None of this stuff was in place at the point at which Kennedy made his speech.

[00:29:19] So where do you start? Under the circumstances, where do you even begin to think about solving problems like this? Well, the first one is that you hire some really smart people. This is a picture of Margaret Hamilton, who was the first software engineer. She was the person that defined the term software engineering. Margaret led the team that was responsible for the flight control systems on the Apollo missions, and she came to realize that the sorts of things that they were doing was engineering. They were thinking about the ways in which things could go wrong, and they were approaching things with a much more disciplined approach to solving problems.

[00:30:01] But that too isn't enough. Here is a picture of the Earth-Moon system. It's important to understand the nature of the problem that you're going to tackle, except this isn't the nature of the problem that they were tackling. This is a kid's storybook picture of the Earth-Moon system. Here is a picture of the Earth-Moon system to scale. At this point in human history, no human being, no human artifact, had been more than one pixel away from the Earth. They were nowhere in terms of space exploration at this point. So the first problem is, how do you get from the Earth all the way to the moon and back again? It's an enormous problem.

Designing the Ranger program as experiments

[00:30:42] You can imagine the NASA engineers sitting in NASA headquarters afterwards trying to figure out what to do. "Oh no. What are we going to do? He said we've got to go to the moon in nine years. How are we going to do this? Oh, my brother-in-law's got a car showroom. Maybe we could get jobs there. No. Let's try and solve the problem. What's the problem? We can't solve it. I know, let's build a spaceship. Let's put three astronauts in it, send them to the moon and see if they come back. No, no, no, this isn't the room, this isn't the place for waterfall thinking. How do we solve this problem? How do we make the problem less risky? I know, why don't we build a spaceship, don't put any astronauts in it, send it to the moon and see if it comes back again. Well, that's better. We might keep our funding for a bit longer. We're not going to kill a few astronauts. That's definitely an improvement, but it's enormously difficult getting something all the way to the moon and back again. It's hugely difficult. Oh, I've got a great idea. Why don't we just send the spaceship to the moon and land on the moon? That is a great idea. That nearly halves the problem. We'd know loads more after we could do that. That would be a really big step forward. What does landing on the moon take? What does landing on the moon mean?" It doesn't have to survive the landing on the moon.

[00:32:19] And that was the job of this program. This is the Ranger program, and the Ranger missions were designing, literally, spaceships as bullets, so that they could carry out target practice. They were going to fire these spaceships at the moon to see if they could hit the moon. That's what this spaceship was for. The first objective, though, the minimum viable product of hitting the moon, is: can you get this thing into orbit? And guess what? It blew up on the pad. It didn't work. So the next mission was also focused on getting the spaceship into Earth orbit. That also failed. It blew up on the pad.

[00:32:57] If you've ever worked in a big project and seen project management working in those sorts of environments, this next one might amuse you. They'd just failed on launch for the first two, and the next mission was supposed to go to the moon, so they went for the moon anyway. In reality, they had learned things from the first two missions, although they were failures, and so they did make it into orbit this time. They did manage to do the burn and shoot the Ranger off towards the moon, and they missed it entirely. The fourth mission was a success. The spaceship got into Earth orbit. They did the burn, it headed off to the moon, and then died on its way to the moon. From Earth-based tracking, they managed to observe it hit the moon, so that was a step forward. The fifth one missed the moon again. The sixth one worked. It got into Earth orbit, headed off to the moon, hit the moon. It got telemetry back. The cameras failed, but they'd still learned more and they'd made more progress. The seventh mission was a success. The eighth mission was a success, and the ninth mission was a success.

[00:34:15] So, NASA in the Cold War: you've got essentially unlimited budgets. You've got access to the finest scientific and engineering minds in the Western world. Can you imagine, if the phone call at that time was, "Hello, this is NASA. Would you like to help us send people to the moon? Because we're stuck." You'd be on the next plane. At least I'd be on the next plane if I got that phone call. So they had access to almost anybody that they needed to solve these problems. Where do you start? You don't start by forming a detailed plan and drawing out Gantt[?] charts and figuring out who's going to be doing what in nine years' time. You start trying to figure out how you're going to learn. You start trying to figure out what tests, what experiments, should I undertake in order to gain more information, deeper understanding, so I'm more likely to get a successful outcome with this mission.

[00:35:05] And that's what the Ranger program was about. This was one of millions of experiments that NASA carried out during this period. The experiments ranged from complex ones, like I'm talking about, to simpler ones, measuring the tensile strength of a particular component of the system, or a bolt, or something like that. The whole Apollo program was designed as a series of experiments, focused on learning. One of the other huge advantages of the modular approach to spacecraft design, and systems design in general, was that you didn't have to have all of the pieces to make progress. The early Apollo missions did not have lunar modules, because the lunar module wasn't finished yet. So they were able to make progress, test out other parts and systems of the spacecraft, and get feedback. This is the minimum viable product for spaceships.

Prediction and illusion

[00:35:56] Being experimental allows us to make progress. It allows us to build on our learning. Here's a simple example of how important experimental technique is to me. In this picture, there are two orange dots. Which one's bigger? I often do this as an audience participation game, where I ask people to give a show of hands. That's not very practical at the moment, so just bear in mind which is your answer. Are the dots on the left-hand side bigger, or are the dots on the right-hand side bigger, or are they both the same size? We can find out the answer to that by carrying out an experiment. If you guessed that they were both the same size, you've probably seen something like this before, and it's an optical illusion that fools our senses.

[00:36:45] Here's another one of those kinds of things. Which line is longer? Is the line on the left longer, or is the line on the right longer, or are they both the same? If you've seen this before, you've probably guessed that they're both the same. In this case, you're wrong. They're not both the same, because I made the line on the right longer on purpose. You can't know the answers to that question without carrying out the experiment. Just guessing is not good enough. Being experimental is about making a prediction and then evaluating the accuracy of that prediction. Whether the experiment passes or fails, we're able to learn from the results, and that's deeply important.

Empirical

[00:37:21] The last of my five principles for software engineering is about being empirical, and empiricism is about learning from reality rather than prediction or theory. Being empirical matters a great deal, particularly in the field of software. We have to make judgments, predictions, about the way in which our software is going to operate in production, but it's very hard to get those right. The only accurate answer is to evaluate these things in production and measure the result. Being empirical matters because it allows us to make evidence-based decisions rather than theory-based or guesswork-based decisions. It allows us to separate myth from reality, and it allows us to move forwards and do research, and to grow and deepen our understanding step by step. Again, this is part of more scientific, rational approaches to problem solving.

[00:38:14] Being empirical matters because we can never be certain of success. We can never be sure that our ideas are the right ones until we get feedback and understand that our ideas have landed, or are effective, or how the designs work. That's the time when we know that this is truly the case. Progress only comes when we risk failure. We must take the risk, and therefore we must work in a way that allows us to collect evidence, data, information, feedback, and then the lessons to learn from production and from the real use of that software. We learn the most when our predictions don't match reality. Again, this is the power of the scientific method: the ability to work in a way where we can make these predictions and then test those predictions and learn whether they're right or wrong. In both circumstances, we're able to make progress.

[00:39:07] Production is always going to surprise us, and if we are taking the kinds of risks that we are talking about, working in an experimental way, then it should surprise us. A good way of thinking about this, I think, is if you think about all of the information that we hold about the software that we're working on: our mental model of what it is that we're trying to achieve, our mental model of the design, the design itself, the source code of our software, the tests that we run against it, the way in which we organize ourselves. Everything to do with the software and the products that we create is really only ever our best theory of what's going on so far. If we're going to work in a genuinely rational way, then what we should be looking for is to evaluate those theories and test those theories, and understand how to get better over and over again.

[00:40:00] This is really the theory of continuous improvement, and it's fundamental to a scientific approach to reasoning. If you think about modern physics, the two cornerstones of modern physics are quantum field theory and general relativity. These are deeply profound ideas that inform our understanding beyond any other human experience in terms of the accuracy of their predictive models, and yet we know that they are incompatible with one another, and so one of them is mistaken. It looks like it's probably general relativity. It looks like general relativity is a theory at a different level of abstraction than quantum field theory, but we don't really know, and there are teams funded all around the world to dig into and try and refine and come up with newer, better theories to answer these questions. We should be thinking in those sorts of terms. We're never really done with software until the software is out of commission. Up until that point, we're always refining and improving and developing our theories of the system, and that is an effective way of working.

Continuous delivery as applied science

[00:41:09] I got into talking about software engineering really via continuous delivery. That's a result of working in continuous delivery, and teaching people how to use continuous delivery to help them with their software development needs, and realizing that there was something going on here. There were some profound reasons why this stuff was working. Continuous delivery is an approach that's grounded in the kind of scientific thinking that I'm talking about. It's based on short, tight, high-quality feedback cycles. It uses test-driven development, in which we create these tiny experiments that populate tiny universes around our software, so that we can control all the variables and evaluate that code. It uses automation to limit the variables, so that we can understand the impacts of changes, control things and get reliable outcomes. It uses hypothesis-driven development, where we make predictions about the ways in which our products will land with our users, and then we can test those predictions in production. And it uses tests as a falsification mechanism, to disprove our systems rather than to attempt to prove them. These are science-inspired[?] ideas. It's also based, as I said before, profoundly on the ideas of feedback, and optimizing for highly efficient, high-quality feedback.

[00:42:32] Fundamentally, though, this is what software development, software engineering, is about. We need to be working in a way where we can make an observation. We can observe whatever it is that we're looking at. We can propose a theory based on that observation. We make a prediction from that theory, and then we carry out an experiment to validate that theory. These predictions, observations and experiments happen at different scales. They can be product level, they can be code level. But nevertheless, trying to close those feedback loops and working in this more scientifically rational way is the way that we get to unlock those benefits that I spoke about earlier in the presentation, in terms of the quality, speed and accuracy of the systems that we are able to create.

[00:43:22] When we talk about software development, we often talk in terms of analogies. We talk about our software development being like the movie industry, or like bridge building, or whatever else. I'm not doing that today. I'm not talking in an analogy. I'm not saying let's be like engineers. I'm saying, "Let's be engineers. Let's start applying the kind of scientific reasoning that leverages human creativity and amplifies it, and apply that to the very difficult problem of software development." Thank you very much indeed for your time today. I hope you have a great day. Thank you. Bye bye.