Lessons Learned Building a Product in Under 4 Weeks that Saved 10,000 Lives

Jennifer Rose at UXDX Community: Rapid Product, UX and Agile and Circular Design. Video: https://youtu.be/7T3y00Z54g0

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Who I am and the product that booked 30 million vaccines

[00:00:00] I'm Jen. I'm going to talk to you, as Rory said, about how to build a company that responds quickly to opportunities, and about saving 10,000 lives in the process. Just a bit of a run-through of what I'll cover today. I'm going to cover who I am, why I'm here, and the three steps that I think are the main takeaways I had from going through this process of building this product, which are: don't wait to be told what to build, create resilient teams that can pivot quickly, and act fast when problems arise. I'm going to get into that a little bit later.

[00:00:36] For now, briefly, I'm Jen. My original background was working for charities in customer service and donor experience teams, and like everyone else I know who now works in product, I fell into it as a career. I started working at a startup as employee number four, which means I was doing bits of everything. After a while an opportunity came up to move into product, which really excited me: moving away from just reporting on problems that our customers were having to having some part in saying what we built next and how we solved those problems. I've now been in product for seven years. I am currently overseeing all things product at New Look. I just started there this year, so I'm in that new job glow.

[00:01:22] But I'm not here to talk to you about that. I'm here to talk to you about my previous role at Accurx, a healthcare startup that builds software for the NHS, and specifically one product of the few that I worked on while I was there. There's one in particular that I'm here to talk about today, and that's because it helped 30 million people get their COVID vaccines. It's actually now being used for COVID and flu every year, but that's probably a story for another day. The timing was really important here as well. Accubook, the name of the product I'm going to be talking about, ended up having a massive impact on the vaccination program rollout and on how quickly GPs and primary care were able to ramp up to millions of patients getting their COVID jabs. You may remember this way back when; we're going to cast our minds back to the wonderful year of 2020.

[00:02:14] This is a screenshot from the product as it was built, almost behind the scenes, where GP users were uploading patient lists, getting texts sent out to patients, and then using it to follow up with those who weren't responding. You may never have heard of it; this might be the first time you're ever hearing about it. But you may have used it as a user. If you got a text to book in for your vaccinations, you will have got that from us, because we did end up sending texts to, I think, about a third of the population, and they used us to get booked in. In fact, if you've ever had a text from your GP, chances are it was from us as well. I say "us" as Accurx. We were the first company sending patients texts to book in for their vaccinations, and I think a few others followed after that.

[00:03:06] As I said, the impact we ended up with was over 30 million vaccines booked through our software, which amounts to over 10,000 lives saved. I think it ended up as more than that, when we were trying to think about how many people didn't get COVID and didn't get into hospitals, et cetera. Talking about impact, it's definitely the most impactful thing I'm sure I will ever work on in my career as a product person.

GPs didn't have the tech for the biggest vaccination effort in UK history

[00:03:32] Let's take our minds back to that wonderful year of 2020. At the end of 2020 the vaccination program was very much on the way, and you may remember headlines like these, if you were in the UK at the time, around vaccines getting approved and ready for rollout. There was lots of back and forth and unknowns about when it was going to happen, how they were getting approved, all those kinds of things. In early November it became really clear that primary care and GPs would play a massive part in the COVID vaccine rollout, and they were going to be the main delivery sites for vaccines, but without much notice. In fact they had to change their ways of working, because they had to start working together in hubs, something they're still doing today because it worked really well, but they had to go through loads of change management.

[00:04:22] We started talking to them about how they were planning to run this alongside their day-to-day service, which they couldn't drop either. There was one quote that kept coming out again and again when we were talking to GPs, and that was "We don't have the tech to do this." They'd never done it before. It was going to be the biggest vaccination effort in UK history, and the technology and processes they currently had at their disposal just wouldn't have cut it, especially at the scale and the speed we were talking about.

[00:04:51] The only option they had was hiring people and getting people to call. This photo might be exaggerating a little bit, but it's not actually that far from the truth. The plan was to use lists of patients and get staff to call them individually to get them booked into the system, or even send letters to book them into the appointment books for their vaccines. And that was the GPs that were actually a bit more advanced. The others would be using pen and paper and calling people one by one. That wouldn't have scaled, because, like I said, we were really talking about optimizing for speed here: how many people can we get vaccinated quickly? It really wasn't going to cut it.

[00:05:31] We knew there had to be a better way, and we were already really well set up for this, because we had been building software for GPs and working really closely with them for a couple of years at that point. We wanted to build something that would speed up the process of getting patients booked in. We knew that was going to be the bottleneck, so we built a product that was able to send texts to many patients at once so that they could self-serve.

[00:05:56] The timelines we had were really tight, and that's something that's not that fun to have in product. When you actually have a timeline and a deadline, it doesn't work that well with agile sometimes. We started talking to practices in November, trying to figure out how we could help and what they knew about what they had to do, and we made the decision to build Accubook on the 13th of November. In terms of timelines, we knew we had to have something live by Christmas. We knew the national vaccine program was really beginning in earnest, and the massive ramp-up of new practices delivering vaccines was happening in the new year, which left us about seven weeks in the middle. We were already a bit panicked about that, but luckily we were set up really well, and we got something live even sooner than that.

[00:06:43] We smashed those targets. We had a demo webinar, which worked, with a bit of manual stuff behind the scenes, on the 5th of December. Then we went live with our MVP, our minimum viable product, on the 10th of December, just five days after demoing that prototype. Five days after that we had booked in over 10,000 patients, so we'd had 10,000 users through our software on the patient side, which of course we were so thrilled with. We were so happy that happened.

[00:07:13] Like I said at the beginning, there are three key takeaways that I always come back to in terms of why we were able to do it so quickly. We learned a ton, and there's lots of stuff we didn't do so well, some of which I'll cover today. The three steps were: don't wait to be told what to build, create resilient teams that can pivot quickly, and act fast when those problems arise.

Don't wait to be told what to build

[00:07:36] I'm going to go into that first one: don't wait to be told what to build. We already were experts in our field. At Accurx they hire doctors to work really closely with the product teams. You can see here Vivek, Satya and Lucy, who were our three clinical leads at the time. Their job is really to make sure that what the product teams are building, or even just thinking about building, will actually solve problems for our users, and they also make sure what we're building is clinically safe. Vivek, there on the left, is a GP, which means he was able to give us immediate insight on those first thoughts about building Accubook and could really quickly validate: "No, that won't work, because this is how GPs do a thing," or "This is the tech you're trying to integrate with."

[00:08:25] That fit really nicely with user research, which was already the backbone of our product process. We were able to just start following our normal delivery process, so it didn't feel like much of a change to our ways of working. We just had to condense it and speed it up a little bit. This is an image from Dovetail, which is the software we use to record and store all of our user research, and it was really nice to have a digital record to come back to later on, so we could say, "What did they say about walk-ins?" or "What happens when the system goes down? Let's go back and have a look." It saved us having to go back and do more research, because we already had some of those insights.

[00:09:06] The thing that changed for us, and that we really had to adapt our ways of working on, was that this wasn't a time when users could tell us what they needed, or when we could ask questions to prompt some of those user needs. When we talked to them, they said, "We're not sure yet, we're going to get told soon," or "We have no idea what the DNA, the did-not-attend rate, will be yet. We're thinking low. We'll copy what we did for flu and learn as we go." So it wasn't something where we were able to pull from those user needs and figure out what problem we had to solve.

[00:09:37] Instead we had to make sure that we were the experts on the topic. Even before we made the decision to build Accubook, we did lots of research into the vaccination rollout program, and you can see here that on November the 11th we published a white paper. This was really key to success, because it meant we could help our users. Instead of having to build something that catered to the hundreds of edge cases and different ways practices could roll out vaccines, we could guide them in best practice and say, "Here is how you use this product." We could be a lot more opinionated about those workflows than if we weren't really embedded. I can still reel off the difference between the gap for Pfizer versus AstraZeneca; that's going to stay in my mind forever. But because we were able to do that, we were able to make hard decisions for our users.

[00:10:25] It paid off for us. We saw numbers ramp up really quickly after releasing the MVP. Yes, there was definitely some stuff that we had to tweak and iterate on the go, but most of the time we could see that the assumptions we were making, because they were so well researched and educated, actually did pay off for us. That was the first big learning that we had.

Create resilient teams that can pivot quickly

[00:10:47] The second one was around teams: creating resilient teams that can pivot quickly. This was a lot of what we already had set up that worked really well. The team structure we have here might look quite familiar. This is how Accurx structures product teams, like many others. Each team is multidisciplinary, which means they're able to work as an independent unit towards their goals. At Accurx we give them lots of autonomy to make decisions on how they work. Do they want a Kanban board or not? Do they want to work in sprints or not? It's really up to them. We judge them on moving metrics, team health and shipping things.

[00:11:26] Alongside that, and this is really why our teams look like this, we gave teams time to be teams. This meant they got to know each other: what their quirks are, how they like to work, which we'll come back to a bit later on as well. We really encouraged our teams to give themselves an identity. It's really important that they knew why they're a team, what their mission was, and how they're going to work together to get there. That might be formulating working agreements or "how we like to work" documents that they could all share and buy into. That one on the left there, Empire Strikes Vax, was one of our cycle names for some of the work we were doing for the vaccinations team, so it's all nice and themed as well.

[00:12:10] Teams also set their own vision, roadmap and OKRs. This was really helpful for them, like I said, to know where they're going, what their mission was, and what metrics we were going to judge them on. It was also a really key way to communicate that to the rest of the company, so everyone else knew this was the most important thing we were working on, and why.

[00:12:32] We may have asked the team to change what they were doing to something new, but we wouldn't also ask them to change who they were doing it with. It doesn't mean that teams always get to decide what they work on. Sometimes, like in this example, a new company priority came up, like vaccinations, or a new time-sensitive thing, and they would be told, "You're going to work on this thing." We may ask them to change that, but we wouldn't also change who they were doing it with. With Accubook, although we had to shift to something new with a couple of days' notice, we did it as a unit, already knowing those people, so we didn't have to spend time getting to know each other as well.

[00:13:11] We also made it really clear to the rest of the company what we were thinking about and then what we decided to do. I was constantly sharing updates in our general Slack channel, as you can see here, telling everyone to come and join our Slack channel if they wanted to know more, making sure everything was really clear to everyone else: what we were doing and why. This was quite important, because it meant that when team members were being asked to do things like interviews, people knew they were working on something that was really time-dependent. It freed up lots of time. It meant I didn't have to shield them from that, because everyone knew what we were working on.

[00:13:44] All of this added up to having a really high level of psychological safety in the team and really open, honest communication. I went around to every team member and said, "Do you want to work on this vaccinations product? It's going to be really intense, with really tight timelines." I knew the answer was genuine when they said they were ready. They considered it, and they said yes, they were ready to go. We had to make a handbrake turn to work on something new, but we did it as a team, and it didn't break us. Like I said, we didn't have to go through learning how to work with each other again. We could really hit the ground running and get things done from day one.

[00:14:19] Like I said, it's how we judge our PMs. It was something in our progression framework: a big part of being a PM at Accurx is leading a high-performing team. If you're interested in learning more about this, by the way, you can google Accurx's progression framework and read through a lot more about it. That was our big second point, around teams and making sure the team worked together really well.

Act fast when problems arise

[00:14:46] The last section is all around acting fast when problems arise, and believe me, we had lots of problems along the way. Even before we started building, we knew the importance of moving fast as a company and how to base those decisions around moving fast as well. This screenshot here is a meeting about the vaccine go/no-go. Senior leaders in the company, our CEO and co-founders, got together and said, "Should we do this thing or not?" because it had lots of implications. It meant taking a team off doing something else. We knew it would pull in loads of other teams as well. It wouldn't just be this team that got affected; it would affect pretty much everyone else in the business. There were about 50 of us at this point, and everyone in some way was touched by us working on this thing.

[00:15:35] There was this go/no-go meeting, and you can see the timing of this, and then not that long after that I got a message from my VP of product, Benji, saying "Vaccine is go." That line in the sand was drawn in that meeting: we are doing this thing. Then we had to take that mentality forward with us: make a decision and act really quickly off the back of it, the same as we had to do when we figured out stuff from our users on the fly. Cool, make a decision, and off we go.

[00:16:06] That same day, straight after the vaccine go/no-go, we did story mapping. This was us that same day, hybrid, some of us in the office and some of us remote, getting into a virtual room and running a story mapping session to decide what would go into our MVP and what would be left out. It was really interesting to do story mapping when you had this type of deadline as well. Versus just "What's the leanest way we can do this?", we also had to think about feasibility for timelines. I'm not going to go too much into what story mapping is here. If you haven't read Jeff's book, I'd really recommend it. It's a great tool for deciding the scope of an MVP, as well as a great communication and stakeholder tool.

Reworking the team's cadence on the fly

[00:16:51] One of the things here, talking about how we had to shift our ways of working: it soon became really obvious that how we were setting things up for our normal product life wasn't going to work here. It wasn't going to cut it for how we were working and how flexible we needed to be. In normal times I was catching up with my tech lead, Mike, twice a week. We worked in sprints and we set weekly goals for the team, so a nice normal cadence.

[00:17:21] But due to the fast-paced nature of what we were working on, this really didn't give us enough flexibility. As the PM, I was talking to our users on support, and I was making sure on Google that I knew all of the things that were coming from the NHS. We were finding new things about guidance every day. That was either validating some of the assumptions we'd made earlier on, or going against them and showing we were wrong, or it was absolutely new information, and there was also information from our users on research calls.

[00:17:51] This is an example of how all of that added up to Mike and me trying to figure out how to rework our processes on the fly. Again, we'd spent enough time together to have that psychological safety with each other, for him to say, "I don't think that session we ran together went really well. Let's figure out how to do that," and we were able to figure out a way of working. We quickly found new ways of working. Again, because we gave teams autonomy, we were able as a team to go off and change all of our ways of working. We didn't have to get sign-off or change to new tools or anything like that. We could just pivot however we wanted to get the job done.

[00:18:29] We changed to daily goals and daily syncs. I think during this time period, up until Christmas, I spent more time with my tech lead Mike than with my husband, still a bone of contention. But we had to make sure we were aligned on all of the important bits for that day. It wasn't good enough to say where we needed to be by the end of the week. For us to keep to those deadlines, we needed to make sure everyone knew what the most important thing was that day. We also made sure we kept an open dialogue like this, so we could review and improve how we were working. This is just one example of the messages that would go back and forth between us: "That worked better. I'm going to go off and do this."

Skipping retros and burning out

[00:19:09] One of the big mistakes I made was starting to skip some of our cadence, some of those rhythm ceremonies that we have all the time. I was really passionate about giving teams time back and making sure they didn't have to context switch as much as possible. What that meant was I said, "Let's skip a couple of retros. We know what we're working on. We'll come back and do a bigger one later on." What this added up to was that we didn't have time to talk about how overwhelmed and overworked the team were feeling.

[00:19:40] We began to burn out. It was becoming clear that people had less energy, and they were starting to feel burnt out by the stress and the long hours they were having to put in. I'm sure you'll remember that time as well. There was nothing to do. You couldn't go out, unless you wanted to go on another Zoom and do a family quiz, so you might as well keep working, especially when it really felt like we were doing something so important. It became really hard to switch off. I remember that feeling well, and I'm sure you all do as well. It wasn't something that was unique to us.

[00:20:09] We really quickly re-established our retros, and this is some of the output from the retro we did. From the off I remember feeling, "Oh God, we should never have stopped doing these." It became clear how much the team were working evenings and weekends, and some of the stuff there that really made my heart sink was a feeling that they couldn't take breaks because they were missing something. Obviously that wasn't what we wanted. We didn't want the team to feel like that, and it was obviously going to be very unsustainable.

[00:20:40] What we also did was make sure we reset those expectations, and again the key here was to act really quickly off the back of that information. How could we make sure the team could take breaks and work in a more sustainable way, while also making sure we delivered that important product for GPs? It was an intense period, and that wasn't going to change. I alluded to it earlier, but we came up with a "how we're working right now" document. Every member of the team shared some helpful info about what times of the day they'd be working and when they'd be uncontactable for breaks. You can see here that last one: "I try and sign off before 5:30. I want to make time for my family."

[00:21:20] This really helped them communicate with each other: "This is when I'm not around, and that's okay." It was okay for some people to say, "I'm actually working more evenings," and that's absolutely fine, but it made sure everyone had breaks and was getting downtime as well. You can see here it then very quickly fostered a culture of sharing breaks and encouraging others to do the same, which for me was really great to see. From feeling like I'd let the team down, because they all felt guilty for taking breaks, we shifted really quickly into a culture of really encouraging that and celebrating each other for doing so. In that thread are a few photos from walks people had, saying, "Look, I'm out, here I am," and here's a pond, I think, in that one. It was really great to be able to solve that problem.

Doubling the support team after go-live

[00:22:06] Then the next thing happened. We went live, which was great, but then our support team became overwhelmed. The intense period went from us to them, though obviously still including us for fixing bugs and things like that. Accubook was probably the most complex product that Accurx had supported, especially because we'd had to be quite strict about some of the workflows, so we didn't have flexibility in some places. We had loads of incoming requests about what we were building and what features would come next, and users were just trying to get set up, which took time. So our support team became a lot busier.

[00:22:41] What did we do? We doubled that team. We went from having people chip in from other teams, since, like I said, it affected everyone, to doubling our support team. Very luckily we were a startup with funding to do this, and we could hire new people in two days, from having that problem to getting people coming in. All that meant was very quickly we said, "Get a job ad up, we need more people in," and by the end of the next week we had some more people trained up, and it wasn't as intensive a period for that team.

Recap and the impact we judged ourselves on

[00:23:13] I could carry on talking about different examples of where we had to deliver quickly, and I'm very happy to answer some in the Q&A as well. But just to recap, our three steps to success, if you want some takeaways from this, are: don't wait to be told what to build. If you're in this scenario where you're having to move really quickly, make sure you know your users really well already. Create resilient teams that can pivot quickly. Keeping a team together that's already high performing is going to mean you move much quicker than forming a team on day one. And three, act fast when those problems arise. Don't wait to talk about what you could do. Make sure you have a culture where you can make decisions really quickly, and autonomy for people to make decisions, so that a problem doesn't remain a problem for a long time.

[00:24:00] The last thing I wanted to share is how incredibly proud we are of what we built. I'm not showing you these to tell you how great it was and how everyone said we were perfect, but because this was really how we were judging ourselves as a team: the impact we were having. What we set out to do was help primary care when it needed it most. A lot of these quotes you can see here talk about how fast they were able to book patients in compared to what they would have had without it, which is absolutely fantastic. That one in the top right even talks about how the product helped them not waste a thousand vaccines. Like I said at the beginning, I don't think I'm ever going to have another product to work on that has as high an impact as this.

[00:24:42] We did have massive impact, hence why I'm still talking about this and riding on the coattails two years later. This is from one week in January 2021, commenting on how through Accubook we helped support the NHS in giving 25,000 years of extra life. Obviously we were one small part, and I'm not going to pretend we are why it was successful, but I'm incredibly proud of being a small part of the bigger picture of the vaccination program, and I'm sure I'll be talking about it for years to come. That was all I wanted to go through. Thanks very much, and I think there's a couple of minutes if anyone's got any questions.

Q&A

[00:25:25] Host: Thanks very much. It was really interesting. You're hitting the nail on the head on a lot of what we talk about at UXDX, like empowered teams, autonomy, things like that. If anybody has any questions, please do post them. We have about a minute, so I'll try and make it quick. Sometimes when people are under pressure and you move to daily goals, that can actually become a burden on the team, because you have somebody tapping people on the shoulder every few minutes going, "Where are we with this? Where are we with this?" How did you make sure, when you went down to that level of granularity of updates, that you weren't distracting the team by constantly asking for updates instead of letting them do the work?

[00:26:12] Jennifer: Good question. I think some of that was the role of the tech lead, having someone who's breaking that down, being able to work with the engineers together to say, "How can we break down these tasks? What do we think is most important?" The second thing is that having the daily goals, in this experience, almost worked as the opposite. Sometimes that daily goal would be on one person. Let's take the example of one of the engineers, Ravi: "Ravi's is the most important thing to get done." What that meant at standup was other people were saying, "Ravi, can I help you with that? Is there anything you need? Do you want me to take that meeting for you?" Again, because the team knew each other really well, it almost acted as the shield or buffer: "I've finished my thing. Ravi, do you need anything before I pick anything else up?" It corralled the team around the most important task. Not always, and it does feel like pressure, especially at the end of the day when you've got your daily task and you're trying to figure out whether it was done or not. But mostly it was a good shield.

[00:27:16] Host: Excellent. Some people are saying thanks for sharing in the chat, and I have one or two more questions, but unfortunately we're out of time, so I might post them into the Slack channel and invite you to answer those questions there as well. Thank you very much, Jennifer.