The Challenges And Pitfalls To Avoid When Shifting To Microservices

Dana Lawson at UXDX Europe. Video: https://youtu.be/m6vWWM5Y61A

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Why listen to me about microservices

[00:00:00] Hi, I'm Dana Lawson, VP of engineering at GitHub. Today, and hopefully you know why you're watching this, we're going to talk about the challenges and pitfalls of shifting to a microservices architecture. I know, I know, you've probably heard it a million times: "Microservices are the future, monoliths are dead." I'm sure there's going to be some other new boutique architecture (do we call it boutique? We're calling them boutique today) that somebody is going to come and say, "Hey, we need to move to that." But let's focus on the boutique one that we've all probably encountered, and hopefully the reason why you're watching this video. I'm going to share with you some of the challenges I personally encountered when making the shift. And luckily enough for me, I've had more than one opportunity to try and get it right. So how about we just dive in?

[00:00:50] First of all, why the heck is Dana Lawson talking to me about microservices? A little bit about me. I've been in technology for 21 years. I know, that's really hard to tell, but it's true: 21 years. I've gone from being called a sysadmin, to an SRE, to a DevOps engineer, to a software engineer, to a products engineer, to who knows what type of engineer. At the end of the day, I've seen it all. Well, I think I have; maybe not all, but most of it. I've had three monoliths that came with the edict of "turn them into microservices": two in Ruby, one in ColdFusion. You heard that right, folks. One was ColdFusion.

[00:01:32] And then four CTOs, over that timeframe, telling me the benefits of microservices. Now, I'm not saying those CTOs read Hacker News, but I've had four CTOs come in and say, "We're going to microservices." And I'm like, all right. And I've probably manifested over a hundred problems by diving in there without some of the practices and principles we're going to talk about. Hopefully you'll avoid them and not do what I've done, because it really sucked.

The goal versus the bowl of spaghetti

[00:02:01] You've probably got the edict, like I said, from somebody above, or maybe even from yourself, that said, "You know what, we're going to go from our full-stack, single-application architecture, our monolith, to this beautiful diagram of clearly represented services and products." One of the goals of microservices, as we all know, is to really have the sophistication and the walled garden between the different parts of the customer experience, the user experience. Because really, we engineers can get in our own way.

[00:02:32] When we think about shifting to microservices, maybe the first thing I always said was, "Technical debt, it's going to screw us," or "Capacity is going to run out." So we come up with this really flat, beautiful architecture that allows us to move fast. Well, that's the goal. And then reality hits, and it actually looks more like this. No, that is not my bowl of spaghetti and meatballs. That is an actual, true dependency graph of a microservices architecture, because that's the reality of microservices if you don't avoid some of these common pitfalls.

[00:03:09] I know it's going to sound like common sense, and that's why they're common pitfalls. But the reality is, we get shortsighted sometimes when we're close to things. Really, avoiding the beautiful bowl of pasta dependencies is about thinking about how you organize, how you're going to orchestrate, how you're going to maintain your live site, and then, once again, how you're going to continuously improve. Those are the benefits of microservices. It's not just because of technical debt and capacity. If that's the reason you're making the shift, don't do it. You don't need to.

[00:03:39] I was joking at the last UXDX with Chris Slowe from Reddit, when we were doing a panel on microservices. The reality is, once you have a monolith, you're probably always going to have a monolith. It's a question of how much of the monolith you don't need, and where the extraction points have to happen. Because you're never going to get to an architecture like the one you saw. Well, maybe, I don't know. If you do, call me; I'd love to see your presentation.

Four practices to counter the pitfalls

[00:04:07] We're going to go over these three, actually four, key practices that counteract the pitfalls I've encountered. I touched on them a little bit. The first one is design, constraints and collaboration. Before you even start writing a line of code or changing your services to fit this new architectural pattern, you really need to think about how you're organizationally designed, and how your teams communicate and collaborate. Are they operationalized? Are they used to being on call? Do they have a sense of ownership and agency? One of the benefits of a microservices architecture is that you can push accountability down to the developers, and hopefully increase not only their productivity and experience, but your customers' experience.

[00:05:02] Then, we often end up not having a plan for deploying all these services. With one monorepo, one monolith: cool. You knew that you commit to master and it goes in, and boom, which was probably a challenge in itself. But now, if you don't think about how you're going to do this pragmatically with your microservices, you're defeating one of the huge goals of it. That's a huge pitfall: not putting energy into how you're going to create your whole development, automation and lifecycle. You're at a user experience and DevOps virtual conference; DevOps is hopefully in your ethos. If not, this is something that really should be, as you make the shift. If you don't change your internal engineering practices, you're not going to get any of the benefits of this new architecture.

[00:05:52] Then, live site operations: they're going to change. Now you're not watching one thing and waiting on your 500s. You're watching a whole bunch of services, which is awesome. But if you don't orchestrate it in the same manner as your deployment pipelines and your DevOps cycle, if you don't put the thought and practice into how you're going to monitor it and have that in place, you're going to have a really hard time. In fact, I guarantee your mean time to recovery and your SLAs and SLOs will go down as you introduce a microservices architecture, if you do not put the emphasis on how you're going to maintain and watch it.

[00:06:29] And finally: hello, continuous integration, innovation and improvement. Microservices allow you to deploy rapidly and to experiment. Maybe you have an objective for growth. In a monolith, it's really hard to make quick UI changes. Why do we move to microservices? One of the benefits is to be able to have an experience compartmentalized, so that you can change pieces out, you can measure, you can record. If you're not thinking about how you're going to continuously integrate and iterate, then you're once again defeating the purpose, because it doesn't help with productivity, and it does not help delight your customers. And that's what it's all about. So let's dive in.

Organizational structure and Conway's law

[00:07:20] Why is organizational structure one of the pitfalls, the first pitfall? I know you all know Martin Fowler, who has written the bible on microservices, and a lot of his talk is about Conway's law. Conway's law, in a nutshell, is that we organize and design systems the way we show up in the world. If your teams right now are structured in a way that really produces a monolithic architecture, and you have the development practices in place to do so, and you don't think about how you're structured, you're going to inadvertently create microservices that have the same pitfalls as the current operators of your products today.

[00:08:08] I really encourage you to think about this, and go and train your teams. When you're looking to split a large application apart, we tend to focus on just the technology layer, leading to UI teams, server-side logic teams, and database or data teams, with these lines that separate them. Even simple changes can take a lot of time: you've got to go talk to that person, and you've got to talk to that person. So think about how the organizational patterns are going to empower you to think about the boundaries.

[00:08:40] A smart team will optimize around the things that make sense. Conway's law in action really improves the way we think about our microservices; you can't have one without the other. Martin Fowler goes into detail on how he thinks about building the architecture, but it's so true: you cannot just go and do the tech. You have to do the organizational patterns to be able to promote it.

[00:09:07] What we want to get to is more of a world that looks like this: full, comprehensive ownership. We want full-stack teams, with the business capabilities they own compartmentalized. Not in a way where there's no communication, but so you can limit those cross-team dependencies. You're not going and talking to the data team or the UI team when you need to make a change; it's all within your team. Microservices help that be a thing, because that's really, once again, why we're doing this. Why are we doing this? Not because of capacity, not because of tech debt, but so that we can improve our customers' experience: mean time to recovery, availability, clear constraints, so that we can experiment, iterate and hopefully delight. Because believe me, there are a million companies trying to do what you're doing right now.

[00:09:54] So take the time and make sure you're organized the way you want those services organized. You want full-stack teams that build, own and operate? You need to change the way your engineering teams work. This is actually probably the hardest thing when we think about going into a microservices architecture: the culture. Now you have teams that have ownership. Now you have teams that really have to think about ingress all the way to egress, and egress all the way back into ingress, and that whole communication and transaction time. That's probably the hardest one: not preparing them, and not thinking about how they're going to operate and communicate.

[00:10:30] So don't avoid it. Don't just take what you have now structurally. If you're an engineering manager, think about how you organize. Is it the workflow that's going to set you up for success? If not, you need to change it. Sorry, engineers hate reorgs, but reorg or you'll have a hard time. Because if not, you're going to end up with something like this: "Hey, we've got all these microservices, but I can't even do anything." So what's even the point? Microservices can harm a culture, because now your developers have so much friction, and we don't want to harm them. It's supposed to increase productivity, so they can continue to build features and iterate for the customers. So think about it.

Workflow automation and the build pipeline

[00:11:15] Now, as we've restructured our teams, we've avoided a pitfall. We also need to avoid the next pitfall: workflow automation. Hopefully you're listening to this talk, and you're at this conference, because you believe in development operations and you believe in user experience. We also need to think about that in how we work internally: our software development lifecycle and delivery cycle.

[00:11:36] In a traditional monolithic application, there's a single build, a pipeline, and out comes the executable application. All the different development work feeds into this pipeline. If a high-priority bug is found, we have to go and redo the whole build: fix, integrate, test and then republish. Maybe if you're SaaS it's not as bad, but if you have a monorepo, we all know you're on the hook for the deployment. Somebody introduced something that you don't even know about, and we've got to go all the way back to the beginning. One of the reasons we think about microservices is because we want to avoid that. We want to be able to quickly iterate and fix.

[00:12:13] Following a microservices philosophy, there should never be a long release train where every team gets in line and waits. In fact, at GitHub we do have a piece of the monolith left, because, like I said, once you have a monolith, you'll always have a monolith; it's just a question of how big it is. And everybody always bellyaches about having to get in line. In a microservices architecture, it's awesome. It's like: cool, I'm going to go into the service provisioning pipeline, it's all interconnected, boom, new service, I can do it, I've got monitoring. And hopefully you've built a repeatable pattern: it can be merged, tested and deployed, and, as you see, it goes all the way through.

[00:12:50] Some of the challenges are: now you have a team that is totally responsible and independent, so they own their own build pipeline. This is why, if you don't think about your build pipeline, your CI, your CD, how you go from "I did the code" to code to cloud, you're not going to produce the outcomes you want. I encourage you to have a unified, automated pipeline to build and deploy services that is not hidden, so that every team knows how the system works. This will allow multiple languages and frameworks to be introduced, because you've thought about it and built those patterns out.

[00:13:24] One of the pitfalls here, though, is opening the floodgates too wide with microservices [?]. We have a tendency to think, "Cool, now we can use any tool in our toolbox." But really, I would say a pitfall to avoid is not limiting it to two, three or four. What do you really have in your ecosystem? What can you operationalize? What can you build and produce in a repeatable pattern, so you don't end up with dependencies you don't understand? Because once again, if you don't think about these things ahead of time, you're not going to have the outcome you desire. So put the energy and effort into building the CI/CD pipeline, and limit choices at first.

[00:14:03] One of the guiding principles we had at InVision when I was there, when we built out our microservices, was: you can pick whatever language you want, but here's the toolbox of the ones we've already built automation and workflow for. If not, you're going to have to build the automation and workflow yourself, because we want that repeatability. Not in the sense of a monolith, but still the repeatability. So think about the toolbox, think about your CI/CD, think about your DevOps. Because if not, once again, we'll go back to that comic, and we don't really even improve our deployment experience, and we haven't built the circuit breakers in place that allow us to iterate and have a great experience in the event of failure.

Monitoring and preparing teams for on-call

[00:14:48] My third point in practices: if this, again, is all new, build up the culture and the muscle of application performance monitoring, your DevOps monitoring. I don't care if you're using Datadog, Prometheus, Sentry, New Relic, whatever flavor or version. Once again, make it highly visible, make it understood, build out the groundwork for most of your microservices and how they're used, and really ensure that teams are built around it. When we think about organizing ourselves and having a culture of operations so that we can produce microservices, we also have to think about how those teams are going to be enabled. We want to take the ethos of "build, own and operate", which Amazon took and put on steroids, and really apply it. I'm not saying you have to follow every tech giant's rule book on how to do it, but the principles behind a microservices architecture are really mean time to recovery and that customer experience. They go hand in hand.

[00:15:55] The energy and effort that you've put into your deployment pipeline has to be equally there for the orchestration of your performance. What's your Apdex? Apdex is the industry standard for application performance indexing. You want to have a delightful experience; you've gone and removed your monolith; maybe you were having capacity problems, so now you want it faster. Well, how are you going to watch it and make sure? So take the time, build it out, and think about how you're really going to watch the system. In some places in the application you can put things like circuit breakers, you can put in endpoints, you can think about the contracts, hub-and-spoke models, and how communication lies. What type of database technology do you need? Have it totally encompassed so that you can monitor and watch it.

[00:16:43] You're changing your teams a lot. One of the biggest pitfalls is not preparing the culture for these changes. When I was at, I think it was New Relic, another pitfall I encountered was not thinking culturally, hello, at a monitoring company. Now you're putting everybody on call. When you had a monolith, you might have had an SRE group on call, or a DevOps team as the window into the site for interruptions. Now you have a whole bunch of microservices, and you've organized your teams so they build, own and operate. Are you empowering them? Are you building the culture? You don't want to just blindside a whole bunch of production engineers and put them on call without preparing them, because what's the point? Once again, you're taking away one of the benefits of microservices and creating a pitfall, by not enabling the team.

Continuous improvement and the Octocat

[00:17:38] And then finally: why are we doing it? Continuous improvement. One of the pitfalls to avoid, and I would say the biggest one, is not thinking about how you're going to improve the system. You've put in all the work to organize your teams, to build dynamic workflows, to be able to monitor it, but you didn't think about how you're going to interchange these things. Microservices give you the ability, in the event of failure, to always have a delightful, or almost delightful, experience, just like we're seeing with Mona, GitHub's famous Octocat.

[00:18:07] If we think about all of her amazing accessories as independent microservices, this allows us to constantly change and iterate and think about how a customer may interact with your product. Maybe Mona wants a new hat; you can go and easily change that hat [?]. Maybe she represents Canada with her amazing red and white. Cool, now we can go and change that, and we have an awesome Octocat, but now an even more awesome Octocat, because of the microservices architecture [?], having avoided the pitfalls, and with your whole software delivery lifecycle being seamless.

[00:18:52] So don't make the same mistakes that I did. Don't neglect to organize your team. Go think about how people communicate; empower them, give them agency. Build the design into the microservices, into the CI/CD and the workflow. It doesn't matter if you create a whole bunch of services: if you can't orchestrate them, deploy them and maintain them, you're taking away one of the benefits. Think about your monitoring; have a monitoring strategy. Empower, enable, and train. Train, train, train. Don't just think you're going to put in a new piece of software to watch the site without people understanding what it means.

[00:19:27] And then finally, build in that muscle of iteration. It comes back to planning and iterating. If you have the architecture and the frameworks right, you should be able to go and make those changes. Because if not, you've just hit all of the pitfalls. Don't do it; keep the monolith; convince your team that adding more capacity will keep you going. No, don't do that. We've all done that. Take the time, do the design, and incrementally implement. Because if you do, you're going to win, and you're going to win at building the best microservices architecture of all time.

[00:20:01] Hopefully you learned a little bit, and I'm sure you're going to hit some of these pitfalls. I encourage you all, if you haven't, go read Martin Fowler's book. It's highly, highly recommended. He goes into even more detail on some of the ways you can, one, avoid these pitfalls that we've all hit, or at least that I've hit many times over, and two, influence and really create the strategic plan to make your product one of the future. Thanks.