Episode 50

In the previous episode, we started a journey through layers of abstraction of modern back-end web application. As a response to unexpected request, a Hobbit object is going to an adventure, and must travel safely across the technology stack. We seek an answer to the question what happens between Java code and the physical machine. Perhaps, even further.

Our Hobbit leaves the, yet familiar, plain of web framework and goes deep down under the Misty Server Mountains.

Riddles in the Dark: The Server

The code sits atop of web framework, and web framework sits atop of web server. That was mostly true until recently. Now, with Spring Boot, it is common that instead of deploying packed application to the server, there is a fat jar that contains embedded server inside an application. No configuration, no deployment, just running a single jar. Simple solution for simple problems, but of course it’s no silver bullet and might not fit everywhere. How does the server and the application fit together?

There are different tiers of servers, as I wrote some time ago. The largest that we care about are application servers, like Glassfish, JBoss or Weblogic, that implement entire Java Enterprise Edition specification including EJB, JMS, timers, aspects and such. Then goes web servers like Tomcat or Jetty that implement a subset of it, namely servlets and JSPs. And finally, there are simple HTTP servers like Apache or Nginx to serve static content, not really related to Java. Web servers are most relevant, since they are relatively lightweight and provide environment for our web framework to live in. Other strict JEE standards that require full application servers are currently in decline.

Web container, aka servlet container, constitutes a part of web server. Its implementation is called Catalina in case of the Tomcat web server. The role of servlet container is to manage servlets lifecyle, provide entry points for different events related to it, take care of security, map requests to proper servlets. Servlet container manages HTTP sessions, filters, wrappers, access to disc resources, context and scope. At least, that was the idea. Nowadays, most of this responsibility is delegated to web framework, which is actually a single big servlet. And big servlets know how to take care of themselves. Does this mean that servlet standard is useless now? No. It just changed its target user group. Instead of being the gateway between web application developers and server developers, it now connects web framework developers and server developers.

Our Hobbit is now part of object implementing HttpServletRequest, passing the official interface between web application with all its foundations and the servlet container. The concrete implementation of it guides him to proper network socket, beyond the mountains. Far to the east from here lies the gloomy Mirkwood.

Flies and Spiders: The Java Virtual Machine

Web server, with all its burdens, is a Java program, meaning it is executed on Java Virtual Machine. And what is JVM then? Well… it’s a generic program or, looking from operating system point of view, a process. But let’s not go too far yet. JVM is a specification, much as servlets. Concrete implementations of the standard are, for example, HotSpot from Sun Microsystems or JRockit from BEA Systems, both consumed by Oracle at some point. What does exactly mean “to be executed on JVM”? We shall consider it right now.

In compiled languages, like C++, we have source code and compiler for particular platform which produces a machine code that can later be executed directly as a program. In interpreted languages, such as Python, we have source code and interpreter for particular platform, that interprets the code on the fly to make things happen.

Java platform is a bit of both. We have source code, compiled to bytecode prior to execution. Then we have a virtual machine that takes bytecode and acts partly as an interpreter and partly as yet another compiler, depending on situation. The process is called just-in-time compilation and aims to maximize advantages of ahead-of-time compilation and interpretation while minimizing their disadvantages. Basically, executing earlier compiled code is faster, but it has initial overhead to perform the compilation itself. Interpreting is generally slower, but can take advantage of run-time knowledge, that is not available before. Finally, we can dynamically recompile bytecode to yield better results. When JVM is up for some time, it accumulates statistics about code being executed, and perform adaptive optimizations to further speed up the execution.

The idea of another layer of abstraction here is to have portability and unified safety net over code execution. Error handling is the same on every operating system. The performance overhead is now mostly mitigated by numerous internal optimizations of JVM and the advantages are considerable – less errors, less crashes, less to worry about and faster development. At least in web applications world.

Up to this moment, our hobbit was living in a universe of Java sources and bytecode, but now we are crossing the threshold to another, less abstract world. We are now talking machine code, which means an actual, physical processor instructions. But wait, there is still a long way before we get there.

Barrels Out of Bond: The Container

As we follow the Hobbit after crossing the gates of machine code, a large blue whale emerges right in front of us. It carries numerous wooden barrels on its flat back, each one resembling another. It looks at us from above as if it wanted to say: “Witness me petty wanderers! I am powerful, I am smart, I have companions and we are taking over the environment of enterprise applications”.

Operating system level virtualization, commonly known as software containers or just containers is rapidly gaining popularity since Docker release in 2013. “Wait, stop, get back”. Says our companion in confusion. “We were not yet into operating systems, but we were talking about virtual machines just a chapter ago and about containers two chapters ago. I’m not following…”. Unfortunately, the more layers of abstraction we have the sooner we run out of good names. Container can be part of web server to contain web application. Also, container can be a part of operating system to contain running process. Similarly, virtual machine can simulate execution environment for bytecode and virtual machine can simulate execution environment for… another virtual machine.

The idea of software containers is to take advantage of isolated user spaces, a feature that became available in 2007, known as cgroups in Linux. Cgroups allow to run multiple isolated process trees and file systems on single operating system, each of them able to consider itself root. Container is a run time instance of an image consisting of layers (he, he, he…). Base layer indicates expected host native operating system, and further layers are instructions on how to add required services like Java, web server, appropriate libraries and packages or environment variables. When running containers, we only need single operating system underneath them, as opposed to running full virtual machines where each contains its own copy of entire operating system. Containers can save us order of magnitude of disk space and startup time, allowing for much more efficient resources utilization. Also, we have full execution environment, with anything we need, contained in single entity. It can be run on development machine or in production with the same result. At least in theory.

We can do virtualization in many ways, we can even put containers inside full virtual machines and there are rare cases where it might make sense. In essence, we are mapping API of virtual operating system to the API of real one (can we tell what’s real in this virtual madness anymore?), including network interfaces for our little adventurer.

A Warm Welcome

The Hobbit’s movements suddenly became stiff and mechanical and his gaze appears somewhat absent. He climbs the whales back and positions himself in one of the barrels. All that in some kind of trance, seemingly without realizing what is happening. Majestic creature turns surprisingly swiftly and heads down the river, towards the Lake Town. It’s getting dark, and flashes of fireworks begin to light up the distant sky. People, full of joy and blissful ignorance, are starting to celebrate New Year’s Eve.

Without slightest realization, what’s coming next week…

image source