Thinking Long Term: Preventing Prison For Product Users

Rachel Ganz at UXDX Community: EMEA Series: Product Strategy (Part 1). Video: https://youtu.be/qiZYrmsH_CM

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Arup, Oasys and interacting with the built environment

[00:00:03] Before I get into what I'm talking about, I thought I'd say a little bit about where I'm from. I work for an international company that has a head office in London. I'm based in Berlin, and I have a team which spreads from Montreal to Hong Kong. What Arup does is design and build things, from incredibly complex infrastructure on bits of the New York Metro, the TV tower in Shanghai, the Bird's Nest stadium in China, the Olympic stadium in London: complex infrastructure and building projects. The engineering of the Sagrada Familia in Barcelona, if you know that.

[00:00:56] And here's one of the pieces of iconic design: the Sydney Harbour Opera House, and the structural engineering of the roof, quite a long time ago now. Within Arup is Oasys, and we produce the software that is needed to engineer that sort of structure.

[00:01:23] Who I am: I started out programming a very long time ago, did a master's in human-computer interaction, and got fascinated by human error. I've lived in Gloucestershire, on a narrow boat in London, and I'm currently in Berlin. What I'm concerned about is how we interact with the built environment and how we understand software models of it.

[00:01:54] If you're working in the built environment, you've got things that change over completely different timescales. As anybody who's used the Internet of Things knows, the building will probably last a lot longer than the controls. Generally this is the order: the building lasts the longest, then the infrastructure, the mechanical stuff like fans and central heating, and then you get into the software and the systems, and then the interaction is the thing that changes fastest of all.

[00:02:33] Then when you have a building in a city, you suddenly have a complete set of risks and a complete set of owners. There are the physical risks, if the building falls down, and then there are the social risks, if there are problems with the data and the systems. For example, famously, there have been baggage delivery problems at airports, and that's controlled by software.

[00:03:10] People interact with this stuff at so many different levels and places. Outside a single building there's the environment in which it works, which may be controlled by government, by local authorities, by culture. There are the owners, there are the people who use it, and then there are things like the emergency services that you may not think of as people who have to understand a building and have interactions with it. But whenever you create a building, you have to work out how fire engines can access it so that they can quench a fire. So in the physical world there's a huge amount of interaction consideration.

Stairs, windows, lifts and hidden systems

[00:03:59] Here's a really simple example: a staircase. You may not think of that as a user interaction, because we've known staircases for a long time. About the only thing that can go wrong, apart from you falling down them, is whether you've got to the right staircase and whether you know if you want to go up or down it. To understand that, you have to know what the context of those stairs is in your getting from place to place.

[00:04:28] Slightly more complicated is a window. You have a lot of physical clues about what it does. But with windows nowadays, if your building has air conditioning or central heating, then a window doesn't just open and close. It also affects the environment for everybody else around you, and may have significant effects on heating costs or cooling costs if you open it. A lot of people aren't aware just how complex the balance of their building may be, and a lot of buildings now stop people opening windows because it causes trouble, though it also makes people feel that they're less in control of their environment.

[00:05:19] Even more complex is a lift, especially intelligent, energy-saving lifts that are designed to calculate the most efficient way to manage all the people waiting on all the floors and where they want to go to. And there you are, standing outside waiting for it, and you may have no idea why the lift next door has stopped but it's not going to the floor you want. I don't know if any of you have had this experience. I have had this experience. And again, it's because what is happening behind the scenes isn't visible to you.

[00:05:55] You may only have had a straightforward lift experience in the past. Beyond that, I remember a great story told by somebody who'd come from the Northern Territory of Australia going to Sydney and seeing for the first time this terrible magic room. The doors opened and one set of people went in, the doors opened again, and a completely different set of people came out. That's pretty frightening when you think about it. It is only because we have expectations of what happens in between that we're not scared.

[00:06:30] Those are the simple interactions with a building. Then, as soon as you start... this shows a sewage treatment works, and the controls there. Somebody has to understand this, and understand what the numbers represent and what is important. As soon as you get multiple pieces of data and people interacting over even more complex systems, the challenge of keeping them informed about what they need to know becomes incredibly high. How do you show the system? How do you show the risks? How do you give them context?

Computer models of bridges and invisible forces

[00:07:17] This is what I do, and what the engineers I work with do: they're working with computer models of a building. Here is a computer model of a bridge that is designed for structural engineers to understand how the bridge will behave before they build it, and here's a relatively similar bridge in the process of being built.

[00:07:46] Why did they build the computer model? Well, obviously so that they can take the risk out of building the bridge, by understanding how strong it needs to be, how much it vibrates, what happens in hot weather. One of the issues with global warming is that road surfaces that were designed for one set of temperatures are now being subjected to higher temperatures, and they're not able to cope with them. Rails also. I don't know if you've had the experience that we had in the UK, that the trains had to travel more slowly because the rails had got hotter than they had been designed for, and they just couldn't take the speed of the trains anymore.

[00:08:30] How the bridge moves when people walk on it. How much it costs to build, obviously. But nowadays, as we are all aware of the issues around carbon, or CO2, how much is embodied: whenever you use a piece of steel, whenever you use a piece of concrete, there is a carbon cost to that. And so the more efficiently you can build things, the less carbon you're releasing into the environment.

[00:09:02] Here is the software that I've worked on in the past, and as you can see, it's quite complicated. It's designed to describe invisible forces on representations of real material. There you can see the bridge components. Here are all the different things that they can show as a result of the analysis. It's not an easy read. This program has been around for around 30 years. What it does at the back end is incredibly powerful, and it is currently getting more and more stuff put in the front end as the engineering requirements become higher and higher, and ease of use gets more and more difficult.

[00:10:06] But there are significant problems. Here you can see fairly obviously that there is a different result to the analysis: there's a big blue bar there. But can you see the difference in the building? Nothing really obvious. How do you know what has changed? How can you show a user what is important?

[00:10:41] This is the sort of data that the engineer will actually go and look at. They will pick through these numbers to try and work out what is happening. So it's: how can we assist people to discover where problems are as quickly as possible? It's a 3D model. You can rotate it, you can look at it, but that also means that it can be incredibly difficult to find things. This shows just a screenshot, but imagine, in the complexity of the 3D model, as soon as you turn it round, it becomes hidden. We can put a little bit of AI on it to capture bits that don't behave within an expected range, but how do we know if that is accurate?

How closely does software match reality?

[00:11:39] There are a lot of buzzwords around at the moment with software. Anybody who's mentioned Tesla will obviously have heard of a lot of them. Robotics: yeah, that's going to change the world for the better. Machine learning, obviously. Artificial intelligence, which at the moment is mostly just still machine learning but with a little bit of a flourish [?]. Machine learning is basically statistics; artificial intelligence is machine learning with slightly wider boundaries. Digital transformation. Autonomous vehicles, oh, I love that one. Data mining [?]. They're all magic things, aren't they?

[00:12:23] The key thing with all of them is: how closely does software match reality, and how well do people understand the difference between a model, as we saw of that bridge, and the reality of the bridge? How well do they understand the difference between what a self-driving car sees of the road and what a human driver sees? Because the reality is that software always has limitations, because it has to work on a limited set of data. The data that Tesla or Google uses is vast, but it is not infinite, and the things that can happen out there are actually constantly changing, and the data will always be out of date.

[00:13:22] As well as that, there is the fact that data may be wrong, or the data may be used in the wrong context, which is what's known as dark data. One of the ideas was that we can automatically generate different solutions. These show a whole set of different bridges that have been built by Arup. And then get artificial intelligence to choose which would be the better one. But surely that entirely depends on context. We can set those priorities of using less material, using less energy, reducing carbon, but who has to use it, and where is it situated, and how do people want it?

[00:14:16] And the other key thing is: don't kill anybody. As I said, doctors can kill people one at a time; engineers can kill people in their hundreds if you build a bridge, as people will know who've just seen the refurbished suspension pedestrian bridge in India that just collapsed and killed over 100 people.

People, workarounds and who owns the data

[00:14:44] So maybe software can help stop people making mistakes. Maybe we can create products that reduce error. I could just read this slide, but I think I'll skip it. But one of the problems is that mixing humans and automated systems can cause problems. This actually shows a collapsed building where Arup was brought in to find out what had gone wrong. The extension, the new bit here, was absolutely fine and had been built according to spec. The building that it was attached to had been skimped on during construction, so the way it had been built did not match the way it had been designed to be built, with the very nasty consequences that you can see here.

[00:15:54] That's the thing: you can bring in an automated process, but there are always people around it. And people, let us say, work around situations to find something that works for them. I'm sure none of you have ever done this. None of you have ever been creative on your timesheet, or left something logged in, ticking over to show you're working and capturing your keystrokes while you did something else. But I gather that people do do this, and they find workarounds around every system.

[00:16:44] One of the key things here is: who is responsible for a model of something in software? What is the data, who owns it, and how can we ensure that it is correct? Again, there have been lots of little joyous stories about machine learning on data sets that did not represent the populations that they were serving. But more than that, there have been problems with small things, with driver licensing issuing, where somebody mistyped, and then there have been people who've been issued with two addresses and then have received double sets of bills. Everybody knows of data problems. I'm sure you've experienced them yourself.

[00:17:43] Mostly those problems do not cause people to die, but in the area I work, they can. And so this becomes really important: how do you know that the data you're using is right? For example, people can make mistakes when they're designing something, and this is true of software as well. They can design to the wrong code. For example, you're used to building something for an office in Scotland, or Aberdeen, which is pretty much granite, and then you have to build something in an area that's prone to earthquakes. The codes and requirements are quite different according to that.

[00:18:40] And then there are assumptions, where things haven't been fully communicated. Poor planning, items constructed in the wrong order: I've seen that in software too. And then when people deliberately cheat. Does software actually help any of those things?

What makes a process trustworthy

[00:19:07] Well, one of the things here is the fact that... can you turn your mic off please, Paul? Yeah, sorry about that. What are the tests and the understanding people use to understand whether a process is trustworthy? For software engineers, it's about whether they understand the process, whether it's tested, whether people understand the tests, whether they understand the code. And then there are things like, does it follow good practice? You look at the code and you have some understanding of whether it's been well written, and then whether it's maintained.

[00:20:03] For a structural engineer, again, they want to know how it works, what the inputs and outputs are, and whether they can trust what it has done: whether their model of what the software is doing matches their model of what they'd be doing if they had the back of an envelope, or a piece of paper, or a calculator, or a slide rule. So this set of quality tests is a little different from that sort of quality test. In engineering software, there is the model itself, there is how the model is designed, there is whether the results are reliable, and whether there is too much information for anybody to make a decision on.

[00:21:00] Another sad story. I don't know if you've heard of the Nicoll Highway collapse in Singapore, where four people died. This was to do with the way the software model had been used by the engineer. They assumed that the soil, the clay that the tunnel was being put into, was drained, and drained clay is much stiffer than wet clay. Just imagine a potter's wheel: where is the clay strongest?

[00:21:52] The way engineers analyze soil conditions is that they have various different mathematical models of how the soil behaves. Elastic-plastic, which is that it behaves in a certain way until it reaches a certain level and then everything changes, or Mohr-Coulomb, or others. And you have to understand the right model that you are using for the analysis.

[00:22:29] Those were the mistakes. But the reason I know this is, again, Arup was brought in to find out what had gone so terribly wrong. I'd also like to say that Arup does not produce the software that was used in this case. So there are two things here. One is the engineer using the software did not know his own ignorance, or had been badly trained, and he made significant errors. But how much did the design of the software assist him to make those errors? How could the fact that he'd made the wrong choice be highlighted? Whose responsibility was it?

[00:23:23] Here's an example. For an engineer, in the textbook, something might have less than 10 degrees of freedom. Degrees of freedom, in case you don't know, are the different ways things can move. But a real building can have over a million, because every joint in that building can move in three or six different ways. And then there is a gap between understanding the mathematical model and understanding how it represents the building they're building. How do they control the quality of the data they're getting? And then how do they understand the results? How can you tell where there are problems in your model? How can you tell where your model matches the real world?

New geotechnical software, accessibility and lessons

[00:24:23] This is a piece of software that we released two weeks ago, which I'm really excited about, which is for geotechnical engineering. It's designed to assist you to draw the geotechnical strata, the different layers of soil, to be able to set what material they're made of, to be able to configure the construction stages, and to be able to see the analysis results. In this case, this shows a completed model going through the five construction stages, and you can see how the soil is sinking and bending over the gap.

[00:25:14] Although this does have red in it, this has been checked for color blindness and for accessibility. I just wish to make this point, as accessibility is one of the key issues and one of the key problems, in that approximately one in 12 men is colorblind. Quite a lot of men do not know they're colorblind. If you are just using color to show something as important as danger, or the effect of behavior on part of the bridge, then you just can't afford for them to not see it.

[00:25:59] So what are the key things from this? You have to understand what people do now. To be able to create good software, you have to find out what people already do, what their processes are, and how what you want them to do with software matches their understanding of the real world. Admit what your software cannot do. Understand that there will always be mistakes, misuse and abuse. And make what you have hidden, all your black boxes, as visible and understandable and accessible as possible.

[00:26:47] The problem I have every day is: how do I convey the context, magnitude and importance of the numbers that engineers look at, so they don't kill anybody? There you go.