Should you start with Microservices?

Nix Crabtree, Richard Rodger at UXDX Europe. Video: https://youtu.be/YrTAOMhSpxU

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Round one: the cost of starting with microservices

[00:00:00] Nix: Do you jump all in with microservices or not? Okay. A microservices architecture requires significant architectural and engineering effort up front. Let's say you have 10 endpoints implemented as a microservices architecture from the get-go. That's 10 code bases, 10 data stores, 10 pipelines to build and deploy, 10 automated provisioning pipelines, and that's just to develop them. The architectural effort required is huge. You have the design, the segregation of responsibilities, the boundaries, the orchestrations and the compositions, up front, before you can even start writing any code. You effectively put your developers on hold while the architects endlessly ponder over how best to align all of the arrows in Visio, instead of continuously deploying increments of software of business value and the feedback cycle that goes along with it. That is a lot of wasted time with nothing to show for it.

[00:00:55] Nix: But what is the alternative, Nix, I hear you cry? Bless us with your enlightenment. I'm very glad you asked. These 10 endpoints can begin as a single simple code base. You create a repo, you add a skeleton code base with a ping endpoint, you manually provision a vanilla compute environment which can be anything, anywhere. We're probably at this point four hours in. Nobody cares. You manually deploy a skeleton, code iteration one is done, and all you really need to know to be able to do that is the 10 endpoints to present. The consuming application doesn't care about the implementation, but I'll bet you cash money on the fact that the developers writing that consuming application will love getting their hands on iteration one pretty much when they get back from lunch. That single code base can then grow iteratively and continuously. The architecture is emergent, and with good engineering practices, segregating out those endpoints is simple and straightforward, and the consuming application is blissfully unaware. You do that 10 times and you have a lean, high-quality code base of endless microservices architecture.

[00:02:09] Richard: Many thanks to my learned colleague, but I feel he needs to embrace his inner chaos demon. Yes, you will get something up and running almost immediately, but despite the best of intentions, that messy — if you could call it architecture — that you started with on day one and week one never goes away, because it just gets deeply embedded. And someday, maybe two years later, you decide to have the big rewrite, and that's a whole world of pain. Better to start with a segregation of the system into lots of little pieces.

[00:02:42] Richard: And I dispute the difficulty of getting up and running with a microservice architecture in late 2019. These days you don't need to put that much effort into it. You have things like Kubernetes, you have all sorts of wonderful tools from people like HashiCorp. The only piece that you really have to decide how you're going to build, the little piece of architecture you have to think about at the start, is an abstraction layer for the messages: how do the messages get from one microservice to another? You can use a message bus, or you could use even just a simple little library that hides the fact that you're doing HTTP. So long as you have that little core piece of architecture, which is literally one day of code, you can get up and running and allow different members of the team to work on different things. Typically in a larger enterprise context you don't start with one developer. You're given a team, a team with very heterogeneous skills, and you have to get them all up and productive from day one. By using microservices they don't step on each other's toes, and you keep the production quality safe, and you don't end up stuck, painted into a corner with the wonderful architecture that you invented on Monday morning. I rest my case.

[00:03:56] Host: And we're going to give a quick rebuttal.

[00:04:00] Nix: No. Just no. No. It just — doing simple iterations is definitely the way to go.

[00:04:16] Host: Okay, so now it's over to you guys. I know you have been frantically voting throughout, but now is the official time. Come on, has everybody entered their votes? Not clearly, clearly not yet. However, feeling lost this round. So round one goes to Richard. A sign, a prize for Richard.

Round two: ensuring good design

[00:04:59] Host: So let's start round two. Richard, ensuring good design. If you're doing microservices, it could be, how do you keep everything coordinated when there's lots of little pieces moving?

[00:05:13] Richard: The way that you ensure good design is you let it emerge. It's really, really hard on day one to design a large, complex software system. I've tried and failed many times. You end up back in that rewrite situation. If you were at Kris'[?] talk this morning, he spoke about shimming the interface between the front end and the back end, and that's because the design, the interface that the back end was offering to the front end, wasn't sufficient, so they had to allow the front end engineers to develop their own interface specification, or way of talking to the back end, that was more appropriate for what they needed, responding to the emergent needs of the application.

[00:06:00] Richard: If you accept the fact that microservices have to talk to each other with messages, and you make messages the primary unit of thinking, the primary unit of design, it frees you up to start with a relatively arbitrary set of messages and then, as the system evolves, change those messages into something that makes more sense. Allow the domains to emerge. Now, you're stuck with legacy services and you're stuck with legacy messages, but guess what? Shimming is easy. You don't have to write custom code to shim. All you do is translate between one type of message, the old types, and the new ones, and vice versa. And if you end up in a situation, as Kris did, where you have a 10x explosion of messages to the back end, well, if you're running a large system you're going to be doing canary deployments, so you're only putting 5 percent of your traffic, let's say, through this new messaging shim, and you can tell if that's going to kill the system or not. And because you're using this message-oriented thinking, this message-oriented design, it's easy to tell which messages are at fault. In that way you remove the burden of being a genius, effectively, and you can just, as an ordinary developer, get work done.

[00:07:19] Host: Great, thank you. Over to Nix.

[00:07:21] Nix: Okay, that sounds very boring. Good software design is all about education. It's about bringing people with you on the journey. It's about creating an environment where learning, and putting that into practice immediately, is a natural part of every line of code that you write. I'm not talking about putting people in a classroom and just opening the PowerPoint firehose. I'm talking about a high-trust team which is as accepting of failure as it is of success, and that is a rich and fertile soil which nurtures people along with their skills and their knowledge.

[00:07:56] Nix: Good software is built on some pretty much universal principles. I'm talking about things like SOLID, KISS, DRY, YAGNI, tell-don't-ask. These principles become second nature very quickly when they are continuously practiced. That mindset doesn't develop just on its own. It develops by learning through feedback from others, from your peers. A test-first, pairing approach is a really excellent foundation for this. It requires you to state what you're going to do before you do it, it allows your tooling to create skeleton code for you, reducing basic human errors, and then as a by-product you have a short feedback cycle test suite which tells you if you're going astray. Pair programming is a great partner to this. Apart from the static code analysis tools in your IDE, it is the shortest possible feedback loop, but it has the benefit of being backed up by experience, by domain knowledge and by the team's own working practices, and that provides a rapid learning cycle like no other. You learn something as you're doing it and you put that into practice immediately and you see the reward straight away, and that also builds trust. In a high-trust team they are much more willing to learn from each other and help everyone else become better as well.

[00:09:15] Nix: There's a big difference between knowing you shouldn't and being told that you can't. Trying to stop people doing things wrong through process or gatekeeping is like plugging holes in a sinking ship. They need to learn good design. It's like explaining to a child why they shouldn't touch their freshly iced birthday cake, versus telling them that you're putting it on a high shelf so they can't get to it. One of those scenarios is going to show you just how quickly and quietly they can engineer a precariously balanced furniture tower and put finger marks in the thing you made for them. Thank you.

[00:09:47] Host: Okay, so on ensuring good design: is it around the microservices, the quick, easy replacement, or is it about the teaching and the learning and the coaching? I think this is going one way, unfortunately. I hate my developers, actually. Is there any quick rebuttal that you want to give?

[00:10:11] Richard: Yes, a very short one. In theory, theory and practice are the same, but in practice they're not.

[00:10:18] Host: Okay, so we'll just give another few seconds. Anybody want to give the last vote? Okay, so we're one all. Excellent, keeps things interesting.

Round three: managing microservices

[00:10:31] Host: So the next section that we'll be talking about is managing microservices. It's inevitable that microservices are going to end up in your architecture, but then as they grow and grow and grow it can become very difficult to keep track and to manage them. On this one we'll start with Nix.

[00:10:48] Nix: Thank you. Okay, so managing microservices is hard, and here's why. If you go back to my earlier example, let's say we have 10 microservices. Microservices that don't work as part of a microservices architecture are just services. Typically an interaction with a microservices architecture involves orchestrated calls to one or more of the services. Let's take one microservice in our example and say that actually it calls four others to compose a response. If you're running at scale it becomes way more complex. Let's say, conservatively, you have two running instances of each microservice, and each one of those two share a data store, and we have that set up in two regions. In our example that initial request can come into one of four instances for the initial request, and one of two data stores, and then the orchestrated calls could hit one of sixteen instances backed by eight data stores. That, as you can well imagine, is a lot of logging and telemetry data, and it is well beyond the realms at this point of sifting through the log when something goes wrong.

[00:12:00] Nix: So you've got to build in logging and telemetry from the start, and it can't just be writing information out to log files, and it can't just be one microservice. A microservices architecture is a single organism, and the telemetry and logging has to have that built in from the start. It has to be a single view. So how do we find the right piece of hay in a haystack? Correlation, dependency tracking, and a really good log analytics system. Correlation means that your microservices should accept and pass on some kind of token which identifies that as part of a larger journey. Dependency tracking is all about capturing telemetry data for the dependencies your microservice calls. And then tying those two things together into a single coherent story is the job of a good log analytics system, which means you can quickly and easily pinpoint a particular journey and see all of the data about it in isolation.

[00:13:00] Nix: Dependency management also becomes challenging. One of the tenets of a microservices architecture is the ability to make and deploy change independently from other services, but to achieve this in its pure form is a mind-boggling set of processes and practices. You need to think about API version management as well as binary version management. Business requirements are likely to impact multiple teams, so you need to orchestrate those changes and how they're built and deployed. Breaking changes can therefore result in regression to monolithic deployment. Thank you, finished on time.

[00:13:38] Richard: I think my wonderful, wonderful colleague is suffering from an approach that a lot of developers have when they've started off in traditional environments, which is to put services first and to think in terms of the concrete diagram. Throw all that away. Think in terms of messages first. Understand your system in terms of the message flows: a particular type of message generates another type of message, or two different types of messages. If you think about the system in that way, you can think about analyzing the whole health of the system in terms of those message flows, in the same way that a doctor measures your heartbeat, not the flows of sodium ions through your cells. You understand the system at a higher level of abstraction in order to deal with the issues around management and deployment and complexity.

[00:14:32] Richard: You always start off with a very, very small system with a very small number of services, and the way that your system changes is not by large top-down design, it's by incremental change of the system itself. Any new service, any new message that is implemented by some services, always enters the system by a single deployment of a single service instance, and that never happens without having some covering fire, having some older service that also services that message in some way, perhaps providing less functionality. So you always have this model where you have a version A, and then you have version A and version B at the same time, and then you remove version A and version B, which is the new one, keeps running. In that way you always know if you break something, because you know that the very last incremental change to the system was to deploy a specific instance of version B, and rolling that back gets you back to a good state. In that way you don't need to worry about all this complexity of log analysis or figuring out what's gone wrong, because each individual change is something that you have fully under control.

[00:15:38] Richard: The other aspect which trips people up is the belief that you have to have extremely well-defined schemas, that you have to have very strict typing, essentially, of your messages. I reject that completely and say Postel's law is the way to go: be liberal in what you accept and strict in what you emit. It's the only way to manage running multiple different versions of multiple services and multiple different types of implementation in the system itself. That means that inside a service you are free to determine whether you're going to be strict about what you do, whether you reject any given message, or whether you make a best-effort attempt to keep the system up and running. You don't have to worry about fragility introduced by having an extremely strict schema. If you take those strategies all together, it allows you to keep that complexity under control and allows you to leverage things like Kubernetes, which look after the day-to-day management of the system.

[00:16:36] Host: Great. Any rebuttal?

[00:16:38] Nix: It sounds like just creating small increments of legacy services all the time and then using a strangler pattern to get rid of them, instead of actually doing it collaboratively and making sure everyone's on the same page. Don't introduce people into the system.

[00:16:58] Host: Okay, great. I see people are already starting to vote, so just give another few seconds if you want to complete your voting. Come on. Somebody wants comments. That feedback can be sometimes a little bit heated over speakers. Another 10 seconds. Anything's gonna change? And we're on to the next question.

Round four: technical debt

[00:17:36] Host: Ah, technical debt. Okay, tech debt exists in any system, regardless of whether you do microservices, whatever. Assumptions you make now, they'll inevitably prove to be false, so managing tech debt is critical. Richard, do you want to start?

[00:17:54] Richard: This is the whole point of using microservices. This is the whole reason for bending your way of thinking about things and putting up with some of the minor technical issues that Nix has alluded to. In our system we still have the very first microservice that we built that runs a particular piece of functionality. It's running and it's been running for two years, and it's completely incompatible, pretty much, with the rest of the system. We've come up with much better designs that have emerged from things that we've built later, but it's still running, it still keeps our clients happy, and when we bring in junior engineers they can do a little bit of change on it to fix it up.

[00:18:33] Richard: What's actually going to happen a little bit down the road is that we're going to completely recut that service, build an entirely new one, and then deploy both of them at the same time, and put a few new customers through the new service. Once we're happy that everything is working, we're going to take the old code, which is full of technical debt, badly written, not efficient at all, and we're just going to throw it away. rm -rf. We're just going to delete that entire code base. What's happened there is that no developer has got an emotional connection to that code. Nobody has decided that it's theirs and they're going to defend it against newcomers. We don't have to unravel the bad design that we've put in place there.

[00:19:13] Richard: Yes, there's a little bit of the database schema that needs modification, but that's actually easy, because one of the rules around microservices that I like to follow is that you don't have to have separate data stores for each microservice, but pretty much, even if you're using one database, stick to one table per microservice, and also shield that database by having a messaging layer that allows you to talk to the database. So again you can go back to that message translation tactic which I spoke about earlier. This also lets us build new services rapidly, allow junior engineers to build the services, because even if they make a mess, even if it's not efficient, even if they haven't done things properly, even if they've created a message API which is not quite right, which is inefficient, we know down the road that we can throw it away. When somebody starts working for me I always say, don't get attached to your code, because I will personally be deleting your code in three months' time.

[00:20:08] Nix: My learned colleague is drawing a parallel with the use of the plastic water bottle. It's disposable, we don't care how it's made, we just throw it away and get a new one, and we all know where that's got us. So let's talk about tech debt in terms of two things. Let's talk about how it impacts your customer: bugs, or emergent features. I've said it before and I'll say it again: good software engineering principles are fundamental to preventing tech debt in your code base. If you don't need it, don't put it in. If you're building for what you think you might need, you have code that is already doing nothing but complicating things, or even worse, it's doing the wrong thing and now you have to maintain it. That's tech debt. When requirements change, as we expect them to do in an agile mindset, you have code that's wrong, that nobody asked for, that you have to rip out or change to meet the requirements, and that's tech debt caused by your tech debt.

[00:21:06] Nix: There's another aspect of this called contagion. Contagion is a term I first saw used by Bill Clark of Riot Games, League of Legends fame. Contagion is viral tech debt. If someone writes something badly, even if you're going to throw it away, especially if it's someone senior, other developers will follow that implementation. Maybe they straight up copy and paste it. That may even contaminate the applications that interact with your application, and suddenly one piece of bad code starts springing up all over the place, and it's really hard to track it down and get rid of it. There's also tech debt that slows down your developers: obvious things, lack of automation, clunky pipelines, inefficient release processes, so on and so forth.

[00:21:53] Nix: But in the short time I have left I want to talk about one thing. Do not touch the shiny things. Do not even look at the shiny things. Do not listen to the sweet siren song of the shiny things. Start with vanilla tech, tech you already know like the back of your hand if possible, or at least which has been used successfully in your organization, and start with the smallest and most basic version. Change only the configuration you need to, and half do, and then leave it alone. If you're not automating those little tweaks they will come back as tech debt. Thank you.

[00:22:33] Host: Okay, do we have a rebuttal?

[00:22:36] Richard: I actually think that the viral tech debt piece — it's not a rebuttal at all, I actually want to acknowledge that viral tech debt is hugely, hugely significant. What's his name, Bill Clark? You introduced it to me. Forget about everything I said, this one is super important. If you're a senior developer and you slack off and you don't follow your own processes and the quality standards that you've set for yourself or expect from your team, that will infect everybody else in your team. I think if there's one takeaway from this particular debate it's that idea, which I've found really, really useful, and I feel really guilty about stuff on my own code base now. Sorry, that's it. That's a super important point which I have to acknowledge. Thank you very much.

Closing arguments

[00:23:19] Host: Great. Okay, so let's do the final time. Do we have any — we have closing arguments. I don't know, I'm just finishing off this. Okay, so if everybody can finish their voting. I've noticed a pattern: whoever speaks second wins.

[00:23:40] Nix: Or it's whoever's speaking about architects.

[00:23:42] Host: Yeah, whoever is speaking about engineering stuff. Okay, so we are actually 50/50, so in each question we are two and two. So what we're going to do is a quick closing argument.

[00:24:06] Nix: It's actually fair to say that Richard and I agree on many of the things we debated, and it was really interesting to play devil's advocate to each other. I think we both have a passion for doing things well, continuously improving how we do them, and bringing people with us on the journey, which is great. However, in the context of this debate, Richard is my sworn enemy, seeking to defile the good, to befoul the proper, and undo all of the great and hard-won things that you and I have strived to build over the years. It is therefore with a heavy heart and a strong will that we must, in good conscience, reject his machinations and rally to the golden flag of proper development. Mount your unicorns, valiant people, and ride with me across the utopian rainbow fields of the greater good, for all noble developers everywhere. Thank you.

[00:25:19] Richard: Despite my learned colleague's eloquent admonitions, they are a thin veneer over an ideology which will lead you astray, very, very far into the desert, where you will die of starvation. The only way to build sustainable things, the only way to build things that are robust and actually work, is to put aside the conceit that our human intelligence is good enough to do the job, and embrace the way that biology and nature has done it. Evolutionary generation of design is a much stronger way to end up with a system that does what you want. You may not entirely understand it, you may have a deep level of discomfort with the fact that nobody in your organization actually understands the hundreds of microservices and how they interact, but yet those types of systems are the only types of systems which can ultimately be robust and which can ultimately let you move really, really quickly, because again we go back to this idea of just making small, safe changes all the time. Thank you.

[00:26:26] Host: So we'll do the last voting. Nix is currently in the lead. Has everybody put in their votes? I think I'm going to have to call this one: Richard, congratulations. Unicorns for the win.

[00:26:48] Nix: I even have rainbow shoes.

[00:26:50] Host: But we do have time for a few questions.