Going Serverless Panel

Nik Rahmel, Laura Varga, Sébastien Fabre, Eddie O'Rourke, Sebastian Hesse at UXDX Europe. Video: https://youtu.be/8tAtEko7e8k

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions

[00:00:00] Eddie: Thanks everyone for coming along. Before we get into the meat of going serverless, and how easy or otherwise it has been, I might ask each of the guys to talk a little bit about their own roles, respectively with BBC, Walmart Labs and K15t. Nik, if I can ask you to go first, maybe.

[00:00:18] Nik: Sure. Hello, I'm Nik. I'm a senior software engineer at the BBC, specifically iPlayer, so that's our video-on-demand platform in the UK. I think that's it.

[00:00:31] Laura: Hi, I'm Laura. I'm an engineering manager for Walmart Labs. I started as an engineer on the Jet.com platform. I helped build the platform that was running Jet.com, then Jet.com got acquired by Walmart, so now, a few years later, I am helping build the new platform for Walmart.com.

[00:01:02] Sebastian: Hi everybody, I'm Sebastian, and I'm working for K15t. We make apps for Jira and Confluence; you probably know those products. They have a marketplace full of apps, and we provide such marketplace apps. One of our main products synchronizes Jira data between different Jira instances, and we've been using serverless functions in production for about two years now. That's one of the main things here.

[00:01:30] Eddie: Thanks, guys. Last but not least, my name's Eddie. I work for Codec Ireland. We are a full-stack Microsoft solutions provider. I look after our software development practice, which is basically providing services to customers locally and internationally around lean, agile software development and all things DevOps.

Why go serverless

[00:01:50] Guys, to kick off, in terms of going serverless, the first question really is: what was the key driver or motivation that made you look at going serverless, as opposed to, say, conventional infrastructure? Maybe I'll start with you, Nik, on that one.

[00:02:03] Nik: Sure. I think for us there are two main key points. The first one is experimentation, on a technical side, not the A/B tests that we talked about earlier. It allows our engineers to spin up a function for innovation reasons, or to try out something where we don't know yet whether it will scale, very quickly and flexibly. We know exactly how much it's going to cost, we know exactly when it's going to run, and it's only going to run while we do this innovation.

[00:02:43] The second bit, which we've started over the past year or so and which is just starting to roll out now, is that because serverless functions are so compact and small, they allow us to be a lot closer to the user. We host mostly on AWS, and AWS allows us to run Lambda serverless functions in their points of presence. Amazon has 11 PoPs in the UK, I think, and it just allows us to be a lot closer to the user. Rather than having to deploy the whole component in the one data center in London or in Manchester or wherever, we can have all the small functions all over the country.

[00:03:24] Eddie: Thanks, Nik. Same to you, Laura.

[00:03:26] Laura: Okay. First of all, as you said, rapid experimentation: that's a very good use case for serverless. And if you look at serverless as functions in an event-driven architecture, that is another very good use case, which we have implemented. If you want to play around with an event-driven architecture, then serverless is a good place to start.

[00:04:03] Sebastian: Thanks. I can support what the others have said, basically. When we receive webhooks from Jira, mostly Jira for our app, we don't know how many webhooks we'll receive, how often people are changing their issues, how much we need to synchronize. So our decision in the beginning was that we only want to run our code when it's really necessary. And as we can get spikes, like thousands of webhooks within two seconds, for example, that's something I think serverless is a really good fit for.

Has it reduced cost

[00:04:39] Eddie: Okay. Nik mentioned cost. I think a lot of people look at serverless as maybe one way of driving down costs, not just in terms of compute consumption but the cost of managing instances and associated infrastructure. What's your experience been so far? Have costs reduced, and have you been able to measure that?

[00:05:04] Nik: I think overall it's important to keep the balance. While we do innovation, the first use case I was talking about, cost has definitely gone down, because we don't need to pay for the big servers anymore. As these use cases or experiments become more and more production-ready, and we have more and more traffic on them, at some point there will be a threshold at which it becomes uneconomical to stay on serverless, and we look at going back to running classic components. But as Sebastian said, it depends as well on the traffic patterns. If you have spikes of thousands of requests within a second and nothing for the rest of the hour, then it might still be more economical to stay on serverless. So it depends. But I think for us it's been good. At the BBC overall we have two and a half billion serverless executions per month, so someone must be seeing value in it.

[00:06:08] Eddie: Small scale. Thanks.

[00:06:12] Laura: All right. Price-wise, it depends so much on the use case. If you're a startup and you just want to run something and take something out to market really, really quick, go for it. That's probably the fastest thing you can do. But as you ramp up your usage, always be careful, because it will become expensive. So my advice would be to always track the cost. It's not a given that serverless will be the cheapest option at any point in time.

[00:06:46] Sebastian: Yeah, there's a certain threshold. And what you also need to consider is that you not only have the Lambda functions, which have a certain cost, but you also need to consider all the other services that you're using: databases, maybe caches, whatever. Everything you need to buy in, because on serverless functions you can only run a certain amount of code, of course. You cannot run your own database cluster or whatever, so you need to buy in everything else. That's what you also definitely need to consider here.

Empowering developers, and knowing what's running

[00:07:21] Eddie: Okay. I'm curious to know about other opportunities that have arisen as a byproduct of going serverless. A lot of people feel, well, this is great, it's giving more power to my developers. Some people might say it's a negative that it's now very easy for them to get access to infrastructure. Has that been your experience, or have you recognized other opportunities as a result of going serverless? Maybe I'll let Laura go first for a change.

[00:07:50] Laura: Sure. If you don't keep an eye on cost, and if you are not aware of everything that's moving around, you will get in trouble, because application developers, if they're not charged back, will spend all the money they want. That's a given. It's not a question of will it happen, it's a question of when will it happen. At the same time, you do want to enable them to experiment quickly and get results quickly, so that's a good thing. But you have to establish certain boundaries there.

[00:08:37] Eddie: Okay, thank you.

[00:08:39] Sebastian: I think the cool thing is that you can get running pretty quickly, and the maintenance costs that you would have with a usual server, updating any kind of libraries, etc., or the system falling apart, you don't need to worry about. However, if you use it a little bit more, the developers also need to understand what's actually happening with those single functions. When we discuss it with other teams and they ask how it actually works with those functions, that's often something you need to be aware of before starting to apply it. The basic concept is quite easy: you have functions, and you don't need to consider anything else or maintain anything. But the whole concept means you have so many instances of your code running, and you have limited options to handle them, and that makes it a little bit more complicated for the developers as well. So easy to start, yes, but then it gets more and more complex.

[00:09:44] Eddie: Okay, thank you.

[00:09:46] Nik: I think on realizing options, I'm going to give one example of something that started out as a serverless function for us in iPlayer, which was label extraction. For each new episode that becomes available on iPlayer, and it's about 200 episodes a day, we can't really predict what exact times they will become available, because we don't know how long the encode is going to take, whether they're captured off air, or live programs, and all that. So originally, with 200 episodes a day, serverless was a great fit for that. It was innovation.

[00:10:22] But as we moved to supporting it full-time, to support the product, the issue that Sebastian was talking about became an issue. People don't really know where the workflow comes in and how the whole picture fits together. Magically, sometimes it would be labeled, and then if the episode got modified later on, after it had been made available the first time around, the information wasn't up to date anymore. So we changed the whole architecture for that reason, to update everything all the time, and have a less event-driven workflow and a better-understood approach that we're all comfortable with. So that was something that enabled us to get started really quickly, and it's now become a real thing, I guess.

Serverless and DevOps

[00:11:12] Eddie: Okay, thanks, Nik. Sebastian, I'll go to you for the next question. We've managed to get through ten minutes without mentioning DevOps, but I know it's relevant for your business, because you've got so many customers and there's pressure in terms of time to value. I'm curious, in terms of your use of serverless: is this part of a wider DevOps strategy for getting code from dev into production, and if so, how has serverless helped you, or has it made a difference either way?

[00:11:47] Sebastian: I think in principle, DevOps and serverless are two basic, separate things. Of course you can use both and have a better process for bringing value to your customers. But I would say in general, you should definitely apply some sort of DevOps process, and whether you then use serverless functions or containers or regular servers doesn't really matter. In the end, for us it's cool that we have those serverless functions, because if you want, you can have updates nearly in a second. As soon as you deploy or upload a new artifact for your functions, it gets applied immediately to all the following requests to your functions. That's a pretty cool thing, so the deployments can be very fast. You can also have canary releases, of course. Anyway, I think it makes the process faster, but in principle, from my point of view, they're two separate things.

[00:12:49] Eddie: Okay, thanks. For you, Laura, is that question relevant in terms of how you work?

[00:13:00] Laura: In my eyes, DevOps is there regardless of whether we're talking about serverless or a platform that runs microservices or runs containers or whichever. It comes with some integration and support for application developers to see their logs, to set up alerts, to actually deploy their software in a reliable, continuous fashion. This should happen regardless of whether you go for serverless or decide to go with some other solution. At the end of the day, they should get the benefit of the same tooling around the solution.

[00:13:40] Eddie: Okay, thank you.

[00:13:45] Nik: For us, I think DevOps was a stumbling block, a roadblock, when it came to introducing serverless functions. As Laura said, we want to use the same tooling for DevOps for serverless functions as we did for classic components, but because you have less control over the environment in which the function runs, you don't necessarily have that control. So we had to build our own tools to get the logs out of Amazon into our own logging platform, to be able to look at them. And our continuous deployment platform wasn't set up to deploy Lambda functions when they were first released. Obviously our developers were very keen to get started on them, since they're so powerful, so we had to not use our in-house deployment system for a while and use the Serverless Framework. So there was a lot of hand-cranking involved to be able to align the tools for serverless and serverful, if that's the word.

Other challenges: comfort, observability and fit

[00:14:50] Eddie: Okay, that's a nice segue into the next question. Other than the DevOps roadblock, what have been the other big challenges in terms of implementing serverless?

[00:15:04] Nik: Besides what I just said, I think it's engineers being comfortable with it. It's a new paradigm, and users need to know what they're doing. I think there are still a lot of people who are more comfortable with the classic workflows and supporting them. We don't have a separate serverless ops team versus classic component teams. Everyone just tries to pick the solution that's best for them, and if they're more familiar with the more expensive one, they will probably pick the more expensive one first, which might not be ideal for the business, but it might be better for the ops side of things.

[00:15:44] Eddie: Sure, okay. So Laura, at Walmart, what have been the biggest challenges?

[00:15:48] Laura: All right. The biggest challenge for us when using functional programming and event-driven programming is actually the observability side of things. If the system is simple, if you have one to three functions and they're tied together by some events, that's fine. We ended up having 2,000 to 4,000 microservices, and it's nearly impossible to reason about them and understand what each one of them, each small web service, does. That's where you need things like Zipkin or Jaeger or OpenTracing, something that will allow you to understand your system by following the requests that flow through it. That was the biggest challenge for us: keeping a hold on our understanding of our systems.

[00:16:44] Eddie: Okay, thank you.

[00:16:46] Sebastian: Besides having a complex architecture, I would like to add another point. For example, we had the problem that we wanted to provide a REST API, and we tried to do this using serverless functions. However, in the end we gave up. We already had a REST API written in Java, and we wanted to migrate that to serverless functions, but in the end we said, okay, that's not really a good way to accomplish it. So you might also need to think again about your use case for serverless functions. In the end we said, okay, we have separate services, usually running on EC2 instances, and they serve the REST API, and everything else, like our synchronization process and flow, runs on Lambda functions. So that's more of a technical addition to what you have already said.

Security, compliance and governance

[00:17:44] Eddie: So from the IT and also the organizational point of view: Laura, your business is obviously retail, one of the biggest retail names in the world. In terms of things like security, regulation, compliance and governance, has going serverless introduced any kind of challenges in that area, or has it been very similar to the old ways?

[00:18:03] Laura: Right. We have an interesting use case, in that we're not using AWS, because we don't want to, and so we're limited to maybe Azure Functions or Google Cloud Functions. But really, what we're doing at Walmart is building our own platform on top of the cloud providers, so that we become agnostic of the provider, and then we offer either functions, sorry, OpenFaaS, as we implement OpenFaaS, or we do Knative, stuff like that. So the security part comes in-house, kind of. For example, with Knative we'll run Istio, and we'll have in-flight encryption. So that's how we take care of it.

[00:19:09] Eddie: And in terms of relevance at the BBC?

[00:19:13] Nik: I'm just trying to think. We do process a lot of user data, obviously, like what people have seen, and we deal a lot with children in the audience as well. But I think the technical principles behind it don't differ too much. Access control, system boundaries and all that apply the same as they do for classic components. I can't think of any example where the choice of architecture has really made a difference to the considerations we have there.

[00:19:51] Eddie: Okay, that's good to know. Sebastian?

[00:19:53] Sebastian: Basically the same. The good part of those serverless functions is that you can't attack a server which isn't up and running at the time. So that's the good part from the technical side. But you still have the same challenges that you have with a usual application. Your application needs to be secured as well. If you introduce any kind of security vulnerabilities, they can come through serverless functions or through containers or whatever. If your application code doesn't handle it well, you still won't catch them.

[00:20:26] Nik: Actually, that just reminded me of something. You were talking about containers, and I guess you can call that serverless as well. When we deal with containers, in one way it becomes much more of a black hole where we get the source from, especially when we build off containers or images that other people have provided on the internet. It's like, I'm just pulling this random code. So maybe that is a consideration here.

Cloud-native services and vendor lock-in

[00:20:53] Eddie: Are you finding, since going serverless, that you're using more services that are native to whatever cloud ecosystem you're using, or Auth0, or authentication products? Are there more opportunities there than before? Is there a relevance there?

[00:21:18] Nik: I think for us it's again more of a roadblock, because we're used to the tools we have from the old world, and we have to introduce new tools. Like I said earlier, we used the Serverless Framework initially, which was a new way of working for us and a new way of doing deployments. And some internal libraries for authentication don't work anymore, and we have to deal with the vendors: "Hey, how do we make this more efficient? How can we work with this better?" So it's certainly changing the tools that we need to use.

[00:21:52] Eddie: Okay.

[00:21:54] Laura: I'm pretty sure you need to start using API gateways, application gateways, basically all the ecosystem that the vendor provides around serverless, because otherwise you don't have any visibility and you don't have any control over anything. So it is, in a very good way, a way to do vendor lock-in.

[00:22:33] Sebastian: We were talking about challenges with external services, right? Okay, good. One point I would like to add here is that if you use these serverless functions and you're using external services, you need to make sure that they can also handle the spikes that you are experiencing with your Lambda functions. If you have 1,000 Lambda functions running at the same time, because they now get called due to some sort of event, and you're calling an external service, you must make sure that this service can handle this load as well, and can also scale up to the number of calls that you're making. That's something you might only realize when it happens, if you haven't tested it properly before. But it's definitely a big thing you need to consider with serverless functions. Or you set some sort of capacity limit on your functions, but then, on the other side, why would you use Lambda or serverless functions?

[00:23:36] Eddie: Okay. Laura, you mentioned vendor lock-in. In your case, you're locked out of one vendor, presumably because Amazon is a competitor to Walmart, so maybe I won't ask you the vendor lock-in question, because you're left with two, which is Microsoft and Google. But Sebastian, has this been an issue at K15t? Is there a concern now, where maybe the perception was, "Well, I have my instances, they're portable between different vendors"? You're primarily on AWS?

[00:24:07] Sebastian: Yeah, we are using that.

[00:24:08] Eddie: Has there been any concern about being too locked into that vendor?

[00:24:12] Sebastian: We're not using different vendors at the moment, so for us the discussion didn't really start about that. And I think it's also fine to stick to one vendor. The cool thing, if you do it the way you're doing it, is that you can switch between vendors, of course. If one goes down, you could take the other one. But things can happen anyway. You can prepare for that, of course, but the question is how much work you want to spend here, because it also costs you a lot to make sure you're not in vendor lock-in.

[00:24:55] Nik: Yeah. When did we start moving into the cloud? 2014, I'm going to say. I'm just going to throw a number out there, I don't know. And it's been Amazon ever since. Actually, I'm lying: there are some legacy Windows applications which have now moved to Azure, and we have started adopting Google Cloud as well. So our development teams have tried to choose different vendors, but for us and our team, yeah, we're pretty locked in. Is it going to be a problem? It depends how Amazon behaves and how our contract with them is going to look in the future. I'm not looking forward to any changes, but it might become a problem.

[00:25:37] And with serverless it's a bigger problem than it is with classic components. If you have the RPM of your microservice, you can push that onto any Linux box that runs anywhere, be it Azure, Google Cloud or Alibaba Cloud. But with your serverless functions, all you have is code, just code. You need to set up the infrastructure around it, and that's going to be a lot of work.

Future plans

[00:26:06] Eddie: Very good. I'm conscious we have less than five minutes left, so I'll just ask a final question to wrap up. I'm curious to know your future plans, based on the experience you've already had with serverless. Do you see this being adopted retrospectively into existing infrastructure, or going forward being adopted as almost a default way of deployment? Any thoughts about that?

[00:26:31] Nik: I think for us, the Lambda@Edge thing, and I'm sure other platforms have an equivalent, is something that we've seen success with over the past couple of weeks of rolling it out, and we probably want to do more with that. So in terms of getting things closer to the user, and making use of the full ecosystem that we get from Amazon, there's probably more serverless to come. We're also looking at Docker and containerization, and making things more efficient that way in terms of resource sharing. Which way we're going to end up going, I can't say, but servers are definitely here to stay. Serverless is just an addition for the more unpredictable loads or the more distributed loads, I guess.

[00:27:22] Eddie: Very good, thank you.

[00:27:26] Laura: Right. For us in Walmart, Walmart probably uses all the technologies out there in the world, so we'll probably still use Azure Functions, we'll probably still use Google Cloud Functions. But in the platform that we're building, we want to offer Knative, and that's what we're going to do. And as for whether you should go serverless or not, it's really case by case. It very much depends on your workload, and on what you want from your application. So really, it's case by case.

[00:28:00] Eddie: Okay, thank you.

[00:28:01] Sebastian: I'm basically saying the same here. You need to know what you want to do with your system; it depends heavily on the workload. What I think is pretty cool, if you start using serverless functions, is that you can basically also pick between all the different services, and they interact nicely with each other, at least that's what I know from AWS. So you can say, okay, I have my functions here, and now I want to extend it with some analytics or machine learning, and I just connect those different dots, or services, and they play nicely with each other. Then, at least in theory, you can pretty easily extend your app and your capabilities. So I think that's a really interesting and good point about serverless functions. But again, it depends. The first point should be thinking about your use case, and whether it really makes sense to use serverless functions.

[00:29:01] Eddie: Okay, thank you. Excellent answer. So on behalf of UXDX, I'd like to thank all three of you for your time today.