On Monoliths and Microservices RSS

When we began the development of our new Online Shop otto.de, we chose a distributed, vertical-style architecture at an early stage of the process. Our experience with our previous system showed us that a monolithic architecture does not satisfy the constantly emerging requirements. Growing volumes of data, increasing loads and the need to scale the organization, all of these forced us to rethink our approach. Therefore, this article will describe the solution that we chose, and explain the reasons behind our choice. Monoliths In the beginning of any new software development project, lies the formation of the development team. The team clarifies such questions as the choice of the programming language and the suitable framework. When talking about Server Applications (the subject of this article), some combination of Java and Spring Frameworks, Ruby on Rails or similar frameworks are the common choices for the software teams. The development starts, and, as a result, a single application is created. At the same time, a macro-architecture becomes implicitly chosen in the process without being challenged by anyone. Its disadvantages come slowly to the surface during further development: It gives rise to a heavyweight Micro Architecture

The scalability is limited to Load Balancing

The maintainability of the system often suffers in the case of particularly big applications

Zero downtime deployments become rather complicated, especially for stateful applications

Development with multiple teams is inefficient and requires a lot of coordination ...just to name a few.

The Big Monolith

Of course, no new development starts as a “Big Monolith”. In the beginning, the application is rather slim, easily extendable and understandable – the architecture is able to address the problems that the team has had up to this point. Over the course of the next months, more and more code is written. Layers are defined, abstractions are found, modules, services and frameworks are introduced in order to manage the ever-growing complexity. Even in the case of a middle-sized application (for example, a Java Application with more that 50,000 LOC), the monolithic architecture slowly becomes rather unpleasant. That especially applies to applications that have high demands for scalability. The slim fresh development project turns into the next legacy system that the next generation of developers will curse about. Divide and Conquer The question is, how can this kind of development be avoided, and how can we preserve the good parts of small applications inside the big systems. In short: how can we achieve a sustainable architecture that will after the years still allow for efficient development. In software development, there are a lot of concepts around structuring the code: Functions, Methods, Classes, Components, Libraries, Frameworks and etc. All of these concepts share just one single purpose: help the developers to better understand their own applications. The computer has no need for classes – we do. So, as all of us, software developers, have internalized these concepts, a question comes to mind: why should these concepts stay limited to just one application? What prevents us from dissecting an application into a system of several, loosely coupled processes? Essentially, there are three important things to keep in mind: Conway’s Law: The development starts with a team, and in the beginning, the application is very understandable: at first, you have a “need” for just one application.

Initial costs: Spinning up and operating a new application only appears to be an easy task. In reality, you need to set up and organize a VCS repository, build files, a build pipeline, a deployment process, hardware and VMs, logging, monitoring tools and etc. All of these concerns are often accompanied by certain organizational hurdles.

Complexity in operations: Big distributed systems are clearly more complex to operate than a small load balancer cluster. If we let things slide, we will generally not end up with a system consisting of small, slim applications, but with a big clunky monolith. And the biggest problems reveal themselves when it is already too late. Normally, it is initially rather clear if a particular application has to be scaleable, and if a rather large code base is underway. It is then the case that when you come around the above mentioned obstacles, you either have to solve them, or live forever with the consequences. At OTTO, for example, it did pay off to establish four cross-functional teams very early in the process, and therefore to prevent the development from belonging to just one team. As a consequence, four applications originated instead of just one. As we already had to operate a big system in the past, the complexity of operations seemed a solvable topic for us: in the end, there is little difference in operating 200 instances of a monolith or a similar number of smaller systems. Initial costs of spinning up the new servers can eventually be overcome through standardization and automation. Since we are not referring to cloud services, you indeed have to do some groundwork at least once in order to set up this automation. But then, you can benefit from all this work for a long time in the future – a cost that pays off. Scalability So, how can we design a system made out of small, slim applications instead of a monolith? First, let’s remind ourselves in which dimensions applications can be scaled. Vertical Decomposition The vertical decomposition is such a natural approach that it can be easily overlooked. Instead of packaging all of the features into one single application, we disassemble the application into several applications that are as independent as possible of each other from the very start.

Vertical Decomposition

The system can be cut according to business sub-domains. For example, for otto.de we have divided the online shop into 11 different verticals Backoffice, Product, Order and so on. Each one of these “verticals” belongs to one single team, and it has its own Frontend as well as Backend and data storage. Shared code between the verticals is not allowed. In the rare occasions where we still want to share some code, we do so by creating an Open Source Project. Verticals are therefore Self-Contained Systems (SCS), as they are referred to by Stefan Tilkov in his presentation about Sustainable Architecture“. Distributed Computing A vertical can still be a relatively big application, so one might want to divide it even further: preferably by splitting it into more verticals, or by disassembling it into more components through distributed computing, where the components run in their own processes and communicate with each other via, for example, REST.

Distributed Computing

In doing so, the application is not only split vertically, but also in a horizontal way. In such an architecture, requests are accepted by services, and the processing is then distributed over multiple systems. The sub-results are then summarized into one response and sent back to the requestor.



The individual services do not share a common database schema because that would mean tight coupling between the applications: changing of the data schema would then lead to a situation where a service can no longer be deployed independently from the other services.

Sharding A third alternative for the scaling of a system can be appropriate when very big amounts of data are being processed, or when a decentralized application is being operated: for example, when a service needs to be offered worldwide.

Sharding

However, as we are not utilizing the concept of sharding at this moment, I am not going to go into any further detail at this point.

Load Balancing As soon as the single server cannot process the load anymore, load balancing comes into play. An application is then cloned n-times, and the load is divided via a load balancer.

Load Balancing

Different instances of load-balanced applications often times share the same database. This can therefore lead to a bottleneck, which can be avoided by having a good scalability strategy in place. This is one of the reasons why NoSQL DBs have established themselves in the software world in the recent years, since they can often deal with scalability in better ways than relational DBs.

Maximum Scalability All of the mentioned solutions can be combined with each other in order to reach almost any level of scalability within the system.

Sharding. Vertical Decomposition, Distributed Computing and Load Balancing

Of course, this can only be means to an end. When you do not have the corresponding requirements, the end result can become a little bit too complex. Thankfully, one can take small steps in order to bring themselves towards their target architecture, instead of starting with an enormous plan from the very start. As an example, at otto.de, we started with four verticals plus load balancing. Over the last three years, more verticals came into being. Meanwhile, some grew to be too big and bulky. So currently, we are in the process of introducing microservices and extending the architecture of individual verticals in order to establish distributed computing. Microservices The term microservice has grown very popular recently, especially when talking about the system that is being sliced into finer, more granular domains. The term microservice has grown very popular recently. Microservices are an architectural style, where a system is being sliced along business domains into fine-grained, independent parts. In this context, a microservice can be a small vertical, as well as a service in a distributed computing architecture. The difference to traditional approaches lies in the size of the application: a microservice is supposed to implement just a few features from a particular domain, and should be completely understandable by a single developer. A micro service is typically small enough, such that multiple microservices could be run on a single server. We have had good experience with “Fat JARs”, which can be easily executed via java –jar <file> and, if needed, can start an embedded Jetty or a similar server. In order to simplify the deployment and the operations of the different microservices on a single server, each server runs in its own Docker container. REST and microservices are a good combination, and they are suitable for building larger systems. A microservice could be responsible for a REST resource. The problem of service-discovery can be solved (at least partially) via hypermedia. Media types are helpful in situations that involve versioning of the interfaces, and the independence of service deployments. Altogether, microservices have a number of good properties, some of which are: The development in a microservice architecture is fun: every few weeks or months, you get to work on a new development project instead of working on an oversized old archaic system

Due to their small sizes, microservices require less boiler plate code and no heavyweight frameworks

They can be deployed completely independently from one another. The establishment of continuous delivery or continuous deployment becomes much easier

The architecture is able to support the development process in several independent teams

It is possible to choose the most appropriate software language for each service respectively. One can try out a new language or a new framework without introducing major risks into the project. However, it is equally important not to get carried away by this newly emerged freedom

Since they are so small, they can be replaced with reasonable costs by a new development project

The scalability of the system is considerably better than in a monolithic architecture, since every service can be scaled independently from the others Microservices comply with the principles of agile development. A new feature that does not fully satisfy the customers cannot only be created in a fast way, but it can be also destroyed as quickly. Macro- und Micro-Architecture The term software architecture traditionally implies the architecture of a single program. In vertical or microservice style architecture, the definitions like “Architecture is the decisions that you wish you could get right early in a project” is hardly relevant anymore. What part is hard to change in microservice style architecture? The answer is not the inner components of an application anymore. The difficult things to change are some of the decisions that have been made about the microservices, for example, the ways they are integrated into the system, or the communication protocols between the involved applications and etc. Thus, we at otto.de are drawing a difference between a micro-architecture of an application and the macro-architecture of the system. The micro-architecture is all about the internals of a vertical or a microservice, and is left completely in the hands of its respective team. However, it is worth it to draw some guidelines for the macro-architecture, which can be defined by the interactions between the services. At otto.de, they are the following: Vertical decomposition: The system is cut into multiple verticals that belong entirely to a specific team. Communication between the verticals has to be done in the background, not during the execution of particular user requests. RESTful architecture: The communication and integration between the different services takes place exclusively via REST. Shared nothing architecture: There is no shared mutable state through which the services exchange information or share data. There are no HTTP sessions, no central data storage, as well as no shared code. However, multiple instances of a service may share a database. Data governance: For each data point, there is a single system in charge, a single “Truth”. Other systems have read-only access to the data provider via a REST API, and copy the required data to their own storage Our architecture has begun going through a similar development cycle like a lot of other architectures out there. At the moment, we are standardizing the manner in which the microservices will be used at OTTO. Integration So far, I have elaborated a lot on the extent to which a system can be divided. However, the user is ultimately at the heart of all our development, and we want our software to be consistent, feeling all of a piece. Therefore, the question becomes: in which way can we integrate a distributed system, so that the customers do not realize the distributed nature of our architecture? Hyperlinks The easiest kind of web-frontend integration of services is the use of hyperlinks.

mono01

Services are responsible for different pages, and the navigation takes place via the links between these pages.



AJAX The use of AJAX is also strikingly obvious for the purposes of reloading parts of the website via Javascript and integrating them on a particular page. The use of AJAX is also strikingly obvious for the purposes of reloading parts of the website via Javascript and integrating them on a particular page.