The advantages of functional programming with Clojure

Heloisa Carbone at UXDX Community: Functional Programming and Aligning Stakeholders. Video: https://youtu.be/z0cHtOfXIx4

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Why a company would choose a functional language

[00:00:01] Heloisa: Hello. Thanks so much. Hello everyone. I'm Heloisa. I'm a software engineer at Nubank, and today I brought a very different topic for this environment, let's say: the advantages of functional programming with Clojure. This is not going to be a very technical talk. I want to approach this in a more business way. So how would you choose a programming language for the company? Why would you choose a functional programming language for a company? And I want to focus on Clojure, which is the language I have been in touch with for the past three years. It's a language that I think nowadays is probably my favorite, and I think it's something that the industry should take a closer look at, because it has been helping a lot.

Functional programming concepts

[00:01:01] To understand those advantages, there are a few things that we need to know about functional programming. As you might know, there are a lot of different languages nowadays, but most of them follow different paradigms, which are different ways to solve problems. They have different approaches, and normally we choose a programming language to try to solve something that that language might be better at. When we are talking about functional programming, we also have a few guidelines and concepts of implementation, and the languages that follow this paradigm have some benefits and use it to have a better approach to testing and debugging.

[00:02:03] What do we seek with functional programming? The first thing is that we seek services that are going to be totally immutable. We won't have variables changing over time, because in that way we are totally able to have control over the code. For example, nowadays one of the most used programming languages in the market is Java, which is an object-oriented programming language, and it doesn't follow the immutability principle. So sometimes it's really hard for you to debug problems when you're having them, because you don't have control over what's happening in the whole process of the service. With functional programming we try to have that as much as we can.

[00:02:55] Also, as the name says, functional programming: the idea is to try to split our problems into functions. In what way can I split my problems to put them in small functions that are going to be totally testable? Because in that way we're going to have high coverage of the scenarios that our service is going to apply to. We also try to have pure functions. We might understand pure functions as mathematical functions: if you have an x that is something, the result is always going to be exactly the y that you have. So you are totally able to predict what this function is doing and the return that it is going to give you based on the input that you have. Those concepts really help us to have a highly concise idea in the programming language.

[00:04:01] We also seek function composition. We normally think about problems as a composition of functions that we might apply to the problem. And what is different from other paradigms as well is that normally in functional programming we seek recursion over loops when we need to iterate over a collection. We don't have for or for-each situations; we always use recursion over the collection. We also prefer expressions over statements. Expressions we might understand as the example of the mathematical function, in which you always have a result for the input that you gave. Statements, which are the things that we try to avoid, are functions that don't have a result. For example, if you have a function that makes many calls in a step-by-step way, and you don't have anything being returned, just a void, this function is a statement.

[00:05:01] In functional programming we try to have only expressions in our way of doing things, because that way it's easier for us to know exactly what's happening during the whole execution time. And there are a few things that we try to avoid, and they are totally related to the things that I mentioned. We avoid shared state as much as we can. We don't want to have variables being mutated, changed over time, and used without control. This is something that we truly try not to use. If there is a scenario in which you need it, you are totally able to do it, but it's not really recommended.

[00:05:50] We also try to avoid side effects. For those who don't know exactly what side effects are, side effects are everything that goes outside of a service. For example, an error being thrown, a print to the screen, a connection to a database, an HTTP call to another service and so on. All those things are side effects. Of course, our services will need to have side effects, but with functional programming we at least try to separate them in the best way that we can, because in that way we will have a more accurate service being created.

The history of Clojure

[00:06:28] After telling you all those concepts that we try to have with functional programming, just to clarify the idea a bit, I want to deep dive more into Clojure: what the difference is between Clojure and the other languages, why it would be nice to implement it, and how to implement it in a company. To start, I think it's nice just to know a little bit about the history. Knowing the history of a language is not normally something that is really common, but in this case I think it's really important, because it makes up the foundation of what Clojure is.

[00:07:03] It is a really young programming language. It was released in 2007 by Rich Hickey, and it was designed to be a general-purpose, practical functional programming language which would be suitable for any professional whose host language was Java. And how would we do this? Clojure is built on top of the JVM, which is the Java Virtual Machine, the environment that runs Java, for example. Having a functional programming language running in this environment allows not only someone who wants to learn this language to use it, but anyone who has the JVM already configured and an infrastructure created. So it's really easy for you to start using.

[00:08:00] The idea is to bring functional programming languages closer to the market, because nowadays, and this has been changing in the past few years, it's still a small number of companies that use functional programming languages. Mostly because in the past functional programming was only an academic idea. Normally we just used functional programming languages at college, in courses when we wanted to; it wasn't even in the syllabus of the degree, for example. So this idea of having Clojure close to a Java environment brought an easier way for people to try to use it without having to do it by themselves, because it was going to be much easier. This was one of the fundamental ideas in the conception of this programming language.

[00:09:01] It is also open source, so you're totally able to have access to the code and modify it if you want, and collaborate on it. And it follows the Lisp dialect. The Lisp dialect is the grammar of the functional programming language. Lisp was the first functional programming language, created in the academic world, and it was a programming language created to solve problems with lists; that's why the name Lisp. It has a really different style of grammar, because it has many parentheses, and Clojure follows this idea.

[00:09:45] At first sight it's really weird to see how the code is structured, but it's really easy for you to understand how things are done, and it's really simple. As long as you understand how things are put there, you're totally able to read it and program, and it is really, really fast to teach others to use it. Rich has made some modifications to this Lisp dialect, just to make it more understandable, with some additions, but for the most part it follows this dialect, which is really good, because there are many other programming languages that follow it as well.

Why a company would choose Clojure

[00:10:29] Now I want to get to a use case of how we would use Clojure and why a company would choose to use Clojure. What are the benefits and advantages that would make it choose such a different and not so usual programming language? The first thing is that Clojure is really objective. You don't have to have many lines of code to have something running and doing stuff. Java has a really verbose programming style; Clojure is totally different, and it's much more direct and to the point. It's really fast to have things running, so this is a really good advantage of it.

[00:11:17] The other thing, which I think is one of the most important, not only for developers but also for the company itself and at the business level, is that we have outstanding test coverage with this language. What do I mean by that? As we have those ideas implemented, pure functions and being able to predict everything that is happening, you're totally able to have high coverage of unit tests, integration tests and so on in the service. You might have almost full coverage in a single microservice. In that way you're not only guaranteeing the quality of that service, but also the quality assurance of it.

[00:12:04] Sometimes we see companies having separate quality assurance teams to validate and see if what has been done is okay. When you have a language that by itself has the principle of doing that as its main status, this might change the flow. For example, at Nubank we don't have a separate team to do quality assurance. We only have the services being tested by themselves. The developers are able to do the full coverage of testing much more precisely and directly, without much effort. So it's easier, and very good for the company, which doesn't have to spend money to have more people working in different chapters. This is something really, really nice.

[00:13:07] Another thing is that, as it is built on top of the JVM, as I mentioned, it has full interoperability with Java. We are totally able to use all the libraries available to Java, all the things that are already done. You have direct interoperability. You're totally able to just import it there and use it; you don't have to do anything difficult. So this is quite useful as well. It also has a strong community, and it's very easy for you to get tutorials, understand things and ask questions, because the whole community is really receptive and able to talk to you if you need. So this is quite good.

How to use Clojure in a company

[00:13:52] And how would we use Clojure in a company? I think in this part we can take a look not only at Clojure but at any programming language that a company would choose: how it would use the language itself. The first thing, in my opinion, if you want to use a microservices architecture, as it is the most used today and has the most reliability, is to decide the language and use a single language for most of your microservices. I would say to have around 90% of microservices using the same language. In my case, Clojure would be the best scenario, because in that way you're totally able to have a single infrastructure being created and evolve it over time.

[00:14:47] You don't have to decide, "Oh, now I'm going to use Java, now I'm going to use Python, now I'm going to use another functional programming language." When you have many different languages, it's really hard for you to have control over the infrastructure. You're not able to automate many things. When you have a single programming language, this is much easier, and you might evolve the whole scenario as a single thing. So this is something really good to do: choose a language and follow it for the whole idea, only using a different one when it's extremely necessary.

[00:15:26] Another thing is to have a predefined architecture for the microservices that you're going to build. Imagine a scenario in which the company has scaled so much that you have more than 100 microservices being created. If you don't have a defined architecture, it's really hard for you to be able to do maintenance, to move people when you need to, and do those things. For example, with Clojure, the diplomat architecture is an architecture that is really nice to use. I will talk more about it in a moment. But this is something that I think is really useful for all the developers, to be able to work in a more precise way and a faster way as well, because you don't need to take time understanding what's happening, as all the microservices follow the same structure.

[00:16:28] And also, when you have this whole infrastructure created through a single programming language, we might be able to include continuous delivery, which is the idea of coding, testing, doing the validations and putting it directly into production. I will talk more about this as well, but all those things are things that you might do to have it running at the company.

The diplomat architecture

[00:16:51] When I talk about the diplomat architecture, this is a known architecture. Hexagonal architecture is a very well-known architecture for microservices. It follows the idea of ports and adapters. It has the idea of an anti-corruption layer to avoid breaking changes in services. The diplomat architecture is pretty much the hexagonal architecture with two levels of business logic in the part of the context. What I want to say is that it has one more step inside this idea of hexagonal architecture.

[00:17:30] How would I do it in Clojure? I would have this structure of a project, in which I would have the diplomat, which is the part authorized to negotiate on behalf of this service. Everything that needs to be negotiated: things that are arriving at the service, HTTP calls that are arriving, HTTP calls that I'm sending to other services, events and the messaging part, if you use it. All those things would be handled by this diplomat part, and we would have input and output to those ports. The ports are all the connections that the microservice has to the outside world, to other microservices and infrastructure.

[00:18:19] We would also have a predefined external schema, a definite contract for those ports, and we would also have an internal schema, which is our internal model, the model that the context knows. They are different because we need this anti-corruption layer, which are the adapters, to change from the external schemas to the internal schemas, to guarantee that these scenarios wouldn't break if something changes inside the context of our service.

[00:19:00] We would also have this more internal part, inside the internal model, which is the controllers. This is the first level of business logic, and it would not only have the adaptations being performed there, but would also have all the interactions with other things. If I need a connection to a database, or to ask for things, this is where I would do that interaction. So here the side effects would all be concisely implemented, because, as I mentioned at the beginning of the presentation, we try to avoid side effects. So this would be the place where we would put all the side effects, to be able to handle them in an integration test way. And to finalize, I would have the pure logic, which is exactly the idea of having a function created extremely pure: with an x that I gave, I would have exactly the y that I was expecting. The idea of pure mathematical functions.

Continuous delivery

[00:20:09] And the continuous delivery would be done with the coding of the microservice. Then I would do unit testing and integration tests. Integration tests, for those who don't know, I know that I talked about it, try to predict all the side effects that the service is generating. As we try to avoid side effects, we at least need to have control over them. If I want this service to be connecting to the database and updating a table, I want to be able to check that this was done precisely. And if I'm sending an event to another microservice, I want to be able to guarantee that this message has been created and sent. So the integration test is a test done to validate those side effects, and unit testing is a more unitary style, in which you're able to test function by function.

[00:21:15] When you have everything done, you're able to put it in a repository, for example GitHub, which would have a CI running. It would run all the tests that this microservice has, validate contracts, and have the whole infrastructure already configured there. In that way we would be able to have someone other than yourself, another coworker, validate and see if things are okay and in the code conventions that you follow. After approving, we would build the microservice and deploy it, and after that it would be delivered and available to everyone to use.

[00:22:00] So this is the idea of continuous delivery. It's quite useful to do, and with Clojure it's really easy as well. With any programming language it's easy, and if you follow that idea of having your infrastructure with a single programming language, it helps everyone a lot: the company and the developers.

Onboarding engineers: mob programming and buddies

[00:22:27] And as I'm talking about a very different programming language: when you decide to use something like that in a company, how do you onboard people? Because it's difficult to hire people who know it, right? But as Clojure is a very simple language, it's easy to onboard engineers. There are a few things that I think might be used for any programming language as well that help to do it. The first thing is self-service online courses. Nowadays it's really useful for anyone who wants to learn; it's something to start with, and it's a good thing.

[00:23:09] But I would say the most important thing, when you want onboarded engineers truly doing hands-on activities, is that I think mob programming is the best thing a company could do to teach engineers anything that they need. How does it work? It's similar to pair programming. I'm not sure if you have heard about mob programming, but it's very similar to pair programming, in which you have a group of engineers doing a single task together, and they normally switch places. After five to 10 minutes they switch places on who is coding, and all of them give ideas on how to solve the problem, how to do it.

[00:23:56] When you have a new programming language to teach, this is a very good idea, because you might have someone from the company who already knows it very well. So there is a single person who knows Clojure, but all the others are able to solve problems. They give their ideas of how they want to solve the problem, and the person who knows the language teaches them how to do it in Clojure, and they program it by themselves. So they do hands-on activities, and this allows people to truly understand what's happening, and it's really useful. I think it's even very nice to do it within the teams themselves, because it shares the knowledge and speeds up the process of learning much more than a single self-service online course.

[00:24:51] Another thing that I think is really useful is to have a buddy. Sometimes, when a person is new to the company, it's really hard to ask others questions. When you have a predefined person to whom you're able to ask the questions, it's easier to ask them. Even if it is a simple question, you're not going to be so ashamed of asking. So this is something I think is really useful to do as well. And also have this person integrated into the team from the very beginning, because then they're going to do pairing with the others, do mob programming with them, and in this way, with tasks and hands-on activities, it's going to be much faster.

Summing up: learning curve, community and documentation

[00:25:38] Just to summarize a few ideas about Clojure and why I would choose Clojure to be used in a company. The first thing is that it has an accelerated learning curve. It's a really easy programming language, because it has a simple grammar, and it's really fast for you to learn how to do it and start developing a service by yourself. I can say from experience: for example, when I did an exercise to enter the company that I'm at today, I had two weeks to choose a functional programming language and make a program. I chose Clojure, because I didn't know any functional programming language at that time, and I was able to do it in a week, with a RESTful API and so on. So it's really fast, even if you're by yourself. This allows us to learn a lot in Clojure and evolve with time, and this is something really, really good.

[00:26:44] It also has an amazing community and documentation. You're totally able to find many examples over the internet and ask questions if you need, and the tutorials are quite good. Different from other languages, I think Clojure has extremely well-done documentation, because you don't have lengthy descriptions. You have the descriptions, but they are direct to the point, and you also have so many examples that it's much easier for you to take a look and see, "Oh, it's exactly this that I need," not only from the idea. So this is quite good.

[00:27:23] If I have to recommend something, I would say to start with this magnificent book, which is Clojure for the Brave and True. It's really simple and didactic, with a very smooth flow of thought, so it's quite nice and a good starting point. And may Clojure be ever in your favor, and functional programming in your life. Thank you so much for watching this presentation. Now I'm open to questions if you have any. Thank you.

Q&A

[00:27:51] Host: Excellent, thank you very much. I really enjoyed it. I had a question, but you were answering it, particularly around the side effects, because that's my biggest problem with functional programming. It's perfect in theory, but then the real world comes and starts trying to mess with it.

[00:28:10] Heloisa: Yes. This is something that I think is really hard to predict in any programming language. But with Clojure, if you use a good architecture, it's easier for you at least to put it in a single place and do the testing over it, to make it more precise.

[00:28:32] Host: I guess I'll throw out one question, but if anybody else does have questions, please do post them. One of the criticisms I've heard of hexagonal architecture is that it adds a lot of overhead. Instead of having maybe a very simple thing, now you've got files everywhere. It just adds a lot of boilerplate and a lot of complexity. What would be your comeback to that?

[00:28:58] Heloisa: I think you do have to create many, many files, and sometimes you lose control of how many things you're doing, and whether you are truly following the idea of the hexagonal architecture itself. But I think the good thing is that at least you have exactly the same names for things. For example, if I need to move an engineer from one team to another, because that project is much more urgent than the ones we have here, it's much easier if you follow the same architecture than if anyone just decides what to do. So even though you have all those files being created, and sometimes they don't make much sense, sometimes it's better to do it that way than to have everyone doing what they want in a simpler way, because sometimes what's simpler for you might not be as simple for the other person. So having a predefined architecture might help as well, even though it has many files being created.

[00:30:13] Host: Excellent. No, it's a good counterargument. Well, that brings us to the end of our time. I just want to say thank you once again. That was a really interesting chat, and I hope everybody out there enjoyed it as well. Thank you.