A Journey From Complexity: Reducing Technical Debt

Knowledge / Inspiration

A Journey From Complexity: Reducing Technical Debt

Continuous Delivery
UXDX Europe 2018

What is complexity?
• History of complexity ( Backend, Backend to frontend, Frontend and back again )
• Microservices curse ( how the frontend start aggregating )
• GraphQL aggregating
• Microfrontends approach ( break down complexity )
• Serverless to rescue ( reduce complexity )

Hi everyone. Thank you for the introduction. So I also changed my title in the meantime of the presentation. What I'm here to talk about today is “Complexity”. In general complexity and in web development. So first, let me introduce myself and time, it's just to reiterate I’m Fabrizio Fortunato, lead front-end developer at RYANAIR LABS, and this is my Twitter and GitHub handle and also you find my blog where I generally speak about web development.

So let's start, “sing to me of the man, Muse, the man of twists and turns driven time and again, of course once you have plundered the hallowed heights of Troy”, this is how Homer starts Odyssey and these are I want to star our Odyssey, Our journey through complexity. So start with the meaning of the word complexity. Complexity means the state or quality of being intricate or Complicated, it comes it derives from Latin complexes. And if we look at the usage over time, this is by Google, What you can see is that in the 1950s, the word starts being like something really being overused and I don't know if it is a coincidence, but in 1954, the first programming language was released, FORTAN. So I do believe is developer's fault actually of this overuse of the word complexity. So the first driver complexity in web development is business logic. Business logic, is that part of the program, which encodes the business rules? It dictates how our business object interacts with one another. So we can represent it generally to a flowchart. So, in this case, I want to search for the flights. I'm going to input my dates. If there are flights available I will display the flights otherwise, I'd suggest different days. And this is a simple business logic that we can encode.
So now, after the introduction, we are ready to go to the first step of our journey. And we start off with the history of complexity because I believe it is important to know our past in order to understand the present and predict the future. Web development started brought in 1991. That's when the first website came along. Now. I ran through the data from 1991 to 2000, the average page size was 14 kilobytes. We weren't talking that much about complexity. The only thing that we had was dancing baby GIF and like that, that’s was web development during that time.
The next second from 2000 to 2010. What was marked was the release, for example, of CSS2, we started paying more attention to the UX and by this, I mean, generally designing glossy buttons. That's what we doing. So the average web page went from 14 kilobytes to 100 kilobytes. In the next decade, so the current decade from 2010 till now, what was marked this decade for was for the release, the coming of new modern JavaScript Frameworks. Backbone at first, then angular js. Also, HTML5 came along during this decade. The Ryanair website also went through different transformations, 3 actual websites during this time, and this is because all these changes were fuelled by an increase of complexity in web development and we needed some tools to actually handle this complexity. But this didn't reflect well with performance. So what we started was in 2010, our web page was around 470 Kilobytes till we arrive at today with an average page is 1.5 Megabytes. Now, I know, I'm kinda correlating the complexity with average page size, which is not entirely true. But for a web application, it is actually a good metric to look at it. And just to I want to stress out why this is important like page size because let’s experiment. This one. You can find it in Google if you Google impact calculator. So let's say that our website loads in 3 seconds and we have an average of 1 million users with an order average of $150 with a conversion of 2%. If we managed to decrease our speed, our page load increase our speed. To decrease our page load time by one second. This is the potential revenue that you're looking at, 1.3 million. And this is not just some kind of game, Amazon reported this and that if they will slow down their website by 1 second, they will lose roughly around 1.6 billion per year. So that's why it is important.

So during this talk, I talk about these different areas of complexity in web development and how we can try to reduce this kind of complexity. We start off with Services complexity. So the interaction between front-end and back-end. Historically complexity was only part of the backend. Back-end was the monster and front-end was just the responsibility was just displaying things or making glossy buttons, right?

Then the communication was very simple. One to one. So this kind of back-end is what we refer to as the monolith. Now, I know this is not the kind of Odyssey that I was speaking about. So the monolithic architecture when started to be let’s say destroyed or shifted into microservices, back in 2011, it started. So what it means is that the backend such using a collection of different services, loosely coupled and independently deployable. Our application became a collection, a suite of different surfaces. But if we can rephrase the law of conservation of energy, “complexity cannot be destroyed”. It can only be changed from one form to another. So what it meant is that now the application or the communication between all these micro-services was done in the front-end. In our case, we'll have to, for example, deal with flights, accommodation and car hire. And, we can also see this. Like, if you look at the average total number of requests over time, you can see that it increase also over time. What we are trying to experiment with in order to reduce this complexity, is the concept of back-end for front-end. So basically is putting back the aggregation what it was supposed to be. What we gain by using this concept is that we are reducing complexity in the front-end because we are shifting back to the back-end. And, remember complexity on the front end, the one who pays it is your users.

We can unify data models between the different microservices. And also, if you have different clients, you can reuse this business logic. You can reuse these Services across your, for example, Android application, or iOS application. A perfect match for this one is GraphQL. GraphQL is a query language for your API and what it does. It models your business domain, as a graph, you define a schema and it uses a very powerful, declarative data fetching. So the client directly specifies what is necessary for him. That's how I look at our basket services. This is what we are developing now on GraphQL. So this is an example of the schema. We have a basket with some total, the items inside the basket, we call it components and the total, for example, as an amount, currency and if it includes taxes. So let's have a look, at how we can use GraphQL. So what we can do is what we defined in GraphQL is queries. In this case. I want to get my basket and I'm only interested in the total. We wanted to display only our total here in our queries. In another example, in this case, instead, we want to also display our items inside. So what we do is we just append it to a similar query, just append that we want also our items in a basket. So this is the power of declarative data fetching. You always specify exactly what you need. There's no over or under fetching. We were discussing the backend shifted into microservices, but what about the front-end? Well, the front end by nature, it's generally a monolith. And that's mainly for performance reasons. The other thing is that different frameworks don't talk very well to each other like React, Angular. And they don't really work together that well. What this led to is a high exposure to technical debt similar to what the back-end was experiencing before shifting into microservices.
So, give you an example of this, this is there on our website is a single page application built in angular js. And we're talking about more than three hundred thousand lines of code. That's only the front-end with 75 routes, which translates roughly into 75 pages and we're not talking about content pages. These are heavy applications with heavy business logic pages. So what we realize is that scaling up teams and software is art. When you have more than ten distributed teams working on the same code base, this is art. Since there is no clear division between the teams. Everyone is responsible. So in the end, no one is responsible for actually these changes.

So to sum it up. We can't write software fast enough with the monolith. So what's left for us to break down monolith? But so where we can start, let's look at your page as an example. So we have our home page, our Flex select page and our products page and what we can do is split them up into different applications. So each page is its own application. This is what I call micro pages.

In this sense. What is a micro page is an independent page that can be developed and deployed independently. They are loosely coupled so they only know what is next. What is the next page and probably the previous page. The although even if they are loosely coupled it doesn't mean that we are inventing the wheel over and over in the pages. So we are using a set of share components but we can use them across the pages. So, these are basically all the characteristics that the microservices have applied to the front-end. It also gives you a way to break down your monolith iteratively. You don't need to release all the pages once you can take Page by Page and start introducing these micropages. So it's basically the concept of single page application per page, which I know it feels you're under using the power of Single-page applications, but this is necessary in order to reduce your surface of exposure to technical depth. So the next one is design complexity. Over the course of the years, we look at the different designs that came along with different approaches to design. So it’s started off five years ago, with responsive web design based upon the concept of fluid grids or media queries for hiding showing content and also flexible images. What we thought was responsive web design basically, we designed for desktop and then we also scaled down to mobile. When we realize that the mobile traffic started increasing and the scaling down, was difficult, we reversed this approach. We started using mobile-first. This design is an example, in rooms of Ryanair.com. That's what it is. But this was actually good enough for us, and that's because this way, we basically doubled our complexity. We had two completely different interactions. Two completely also design between mobile and desktop.
So that's why we started using an Adaptive solution. What does it mean is that we develop only for mobile and only for desktop. What it gives us is two independent flows, two different applications, which are very performance-oriented for that specific device? Okay.

So, the next step, or last step actually in complexity is infrastructure complexity. Now I know that when speaking about front-end, you're generally, not involved, not responsible for the infrastructure, but I do believe that our code is running somewhere, and we need to know how this code is running. So, what we'll look for here is how we can decrease complexity in our infrastructures also.

Let me introduce you to Serverless. We define Serverless as any application or service which relies on third-party services, like for example S3 for object storage or Google Cloud Functions for running your code in the cloud. Why Serverless is very good in our case, because not only we are reducing complexity in our teams, but we are also offloading the blame to someone else.

And moreover, as I was speaking, a front-end developer shouldn't be responsible for, actually looking after server, should be responsible for caring about using interfaces or writing user interfaces, this is an example of all the topics that a front end developer needs to know during a normal day of development. I want to be Serverless to be on that list also.

Example of how we use Serverless in our area. It's, for example, for hosting our solution. We use S3 to host our files and then Cloudfront, which is Amazon’s CDN to serve these. And what is our responsibility just to upload the file that you will be doing for an FTP. And then Amazon takes care of the rest, scaling up and everything. And then another example is for Lambda function. So for functions in the cloud in this case is Brotli compression. Brotli is generally a better compression algorithm for the web. So generally performs better than GZIP. This is the list of the browser that supports it now. The usual offender is always there. And so we need a way to actually support this fallback.

Can I ask you a question, raise your hand? How many of you use Brotli in production. No one. A couple of people. Okay. What if I show you how actually easy it is with serverless to start using Brotli. So it's just a few lines of code in here. We are intercepting the request. We are checking the headers and if the browser supports Brotli, we send the Brotli compressed file. Otherwise, we fall back on others.

Why this is important. Why? This means a lot to us because just by doing this with a few lines of code. We are reducing our application footprint by 23%. So to sum it up all of these practices, all this journey from complexity and I want to present you today is what we call Ryanair 3.0 and we actually just released it last week just in time for UXDX, I would say. And so this is our first step is our first page towards the micropages architecture. We started with the flight select.

So what it means Ryanair 3.0? It is an adaptive solution. It's going to use the micropages approach and also leverage a lot of Serverless. Well, what we started doing especially in Ryanair 3.0 is paying special attention to Performance. Which translated into going from a 2.2 MB of the current solution to actually 572 KB and this translates into a first meaningful paint going from around 15 seconds to actually six seconds. Another metric that we're looking at is the current flight select with an online mode on a synthetic metric takes around 10 seconds while the new flight select takes around 1.5 second.
Okay, so as I promised you, we want to make our user happy again. And this is the first step, but I do believe that we are still far away from it occurring. And that's because reducing complexity in web development is a never-ending journey. So I want to leave you with the outcome razor “Entities are not to be multiplied beyond necessity”. So keep it simple.

Host: Thank you so much. Ryanair must have an awful lot of technical debt, but it sounds like you're managing it in really incredible ways.

Fabrizio: Yeah that is lots of work actually going, and lots of people actually handling all of these, since now it's a very collective effort.

Host: Yeah. So we do have some time for some questions. We have a mic that is ready to be passed around. Please put your hand up. If you have some questions, we have one here in the front.

Audience 1: Hi, that was really interesting. Two questions really. What way are your teams set up? Are people in cross-functional teams. And the other one is, would these pretty drastic changes be applicable to every company or just because you have such a scale of users you've made some huge, huge decisions there. Would you recommend that to everybody? Or is it just because it's so many users.

Fabrizio: Okay. So let's start with the teams. Like generally, we have more heterogeneous teams, so you will have front-end developers, back-end developers and all the different figures on the team. And so we believe that the team should be autonomous and it can take a feature from development to production. These things that we use, these tips, these steps from complexity. I think part of it, like, you can definitely do what we recommend to start using Brotli for example, or another case would be GraphQL for unlinking your aggregation. But other things like adaptive or the micropages approach, are specific for our use cases. So, the Adaptive, in the end, you're gonna handle two applications. So it brings you a different kind of complexity that you have to manage. While the other one, the micropages, we wanted a way, iterative way to break down a monolith. And this is what we came up with.

Host: Excellent. Thank you. Any more questions? One in the back.

Audience 2: Hi you said, you support your micro pages with shared components. How do you manage shared components so that the complexity is transferred into them? And they’re kept easy to use and don't become specific to something.

Fabrizio: That this is something that we always debate between the teams and so, starting off with visibility. So, everything needs to have visibility of these components in order to understand how it works. And so creating all the tooling around actually to manage those components is important to release them, to publish them etc. After that, like how we kept it simple is generally we don't start from a generic but we start from the specific. So one of them, for example, one of the micro pages that are those showing, it's in there inside the micro page that we develop one of the common components and then when we see that another page needs it or another flow needs it then, we try to extract it. So it's kind of like instead of going from the general starting from the general, we start from the specific and then go to the general.

Host: Thank You. Any more questions? Oh, you go.

Audience 3: I was wondering if by taking a micro pages approach that you could possibly incur a higher performance penalty from pulling down a payload, a large payload for every single app for every page. And if so, and if you did experience that, did that sway, your choice of framework to something that was a much smaller bundles size as a result. And how did you approach that?

Fabrizio: Okay. So the first thing is, yes, we actually consider that there's gonna be a hard refresh between the pages. So we'll have to deal with it. And what we're looking at is, for example, start intensively using server-side rendering. That's, that's one thing that hopefully, not to make any promises, that maybe by the end of the year, we're going to have something for it. And besides that yes, the reason was the following, it's easier to actually split our application and then try to put it back together rather than starting another time from a monolith approach.

Audience 3: Great. Thank you.

Host: Still any questions find here?

Audience 4: So you talked about serverless technology and the fact that development shouldn't need to care about the server hosting of their application. I guess there has been traditionally, a different train of thought that developers need to know and be part of the DevOps team and have DevOps members within teams because sometimes there are issues that you only see in production or production-like environments that can be very difficult to find or to debug. Do you have a similar experience now that you have moved to a serverless? How do you debug in production when you don't have those environments to look after those things locally.

Fabrizio: Yeah, this is something that we started a year and a half ago. We started roughly two years introducing DevOps also in the teams. So let the teams take more responsibility rather than throwing the ball to another team, to the infrastructure team. And what we came up with was that a lot of the front-end developers wouldn't have the necessary knowledge or skills to actually look after scaling services or writing configuration for your Linux machines Etc. So that's why what we worked on is to try to find out what is the simplest approach that we can take to put our teams out in production. That's why the decision about the serverless and regarding the environments like using, we use AWS, probably you seen in the example, and it's very easy to replicate production-like environment across different your accounts.

Host: Thank you. Any more questions from the floor?

Audience 5: I was wondering about your policy to design just for mobile and desktop. We did the same recently prioritizing those two viewports, but then we came to testing, the tablet design wasn't acceptable, which meant the tasks had to go back into the team to be reworked which caused frustration all around. Have you experienced the same and how did you address that?

Fabrizio: Something similar not actually the same. Because generally tablet traffic is not that significant to actually justify solution only for tablets. So that's why we're still debating if using the desktop or the tablet version for the tablet in this case. But generally tablet traffic is not that significant that we have looking to create a tablet specification.

Audience 5: Thank you.

**Host: **Excellent. I think that's all the time we've got for questions now, but if you all give a researcher a big round of applause. Thank You.