10/30/18

Ian Eyberg

Many people have heard of the security advantages of unikernels. These commonly get boiled down to "it’s a smaller attack surface." Unfortunately, that does little justice to the true security implications of running your software as unikernels. So let’s revisit what the unikernel four point security model actually is:

What is a single process system and why is it important? To answer that question we must answer the question of what a multiple process system is and why that is important. A long time ago in a galaxy far away we used to have computers that consumed the space of entire walls. They cost $500k in today’s money and they had this peculiar business problem. How do we run multiple programs by multiple users without each user having their own computer? That would at the very least, be too expensive. So thus in 1969 which is not quite 50 years ago from the time this article is being written Unix was given birth. Later on many many years later there was a Finnish student named Linus, you might have heard of this guy, that unleashed Linux to the world. During the first dot-com boom and later bust it was extensively adopted.

Then something interested happened. A few companies including ones like VMWare sprung up and reinvigorated a old concept called virtualization essentially sub-dividing the resources of a given server into multiple ones. What’s interesting was not the birth of this $50B by market cap company, but later on in 2005 a smaller bookstore chain started offering server capacity - later known as ec2 in the form of vms. This launch of ‘cloud computing’ allowed massive new companies to grow such as Uber, AirBnb, Twitter, etc. What was curious was this new breed of company weren’t used to spinning up just a few databases or a few webservers - they were spinning up thousands or tens of thousands of VMs. Already devops practioners from these companies had started to embrace concepts like isolating an individual program to a single vm. Why? Mainly just trying to manage all of that is saner that way. Also, it’s a very common concern that if you want to eek out the performance from a given server you don’t want something like a database sitting on the same thing as a webserver - as if those companies could fit both on a single server to begin with anyways.

Later on those involved in working with containers continued this practice with even more isolation of a single program per unit.

Without even realizing it engineers had already started to adopt practices that *looked* like a single process system but inherently wasn’t. In the process they added tons of complexity and tons of layers.

A single process system is just that - one process running in a vm without the capability of running another process. Notice that I said “in a vm” because this design concept doesn’t make a ton of sense if it’s not in a vm. Even the IoT/embedded folks who utilize unikernels tend to deploy on top of hypervisors if for no other reason for management purposes.

Now, If you look at and analyze any sort of RCE attack on linux you quickly come to the conclusion that practically all of them require the concept and capability of running other programs on the same vm. Let’s examine a few recent ones. Back in March of 2018 there was the infamous DrupalGeddon. A simple form sanitization bug was spotted that allowed arbitrary remote code execution on every single Drupal deployment out there - of which there are millions including banks, government institutions and health providers. If you scan through some of the exploits you’ll find on github every single one of them try to spawn a shell and slurp down new code. An owned system is fairly useless to an attacker if they can’t run their code on it and by code we’re talking about running another program as a separate process.

What’s crazy is months after this attack there were many reports of Monero crypto-miners infecting through these bugs. The kicker here was that the first thing they would do is scan the system looking for other crypto-miners that had already infected the system!

That's now the state of cloud security. We literally have bots fighting over resources because it's so freely available. Not sure Philip K. Dick could have conceived of that one.

What's the point? If you take it at face value that AWS or Google Cloud is "safe enough" - safe enough to host a large percentage of well known tech companies software and if you take it at face value that you deploy virtual machines to these cloud environments than by nature deploying unikernels to these cloud systems are inherently dramatically more safer than deploying Linux that has implicit support enabling the running of other 3rd party processes - ones you did not intend to run.

What does it mean to not have a user? Why have a user at all? If you look at some of the marketing that has come out of the container ecosystem their proponents eschew the concept of having so-called "snowflake" servers - that is if a server is misbehaving you shouldn’t be in the practice of ssh’ing in and trying to figure what is wrong - chances are it's your application filling up the disk, it's your application spiking the cpu, it's your application gobbling up the ram, etc. The moral of the story is that you should debug your application locally. Also - production servers are not your dev environment - let’s hope they aren't anyways.

The real back story here, however, is that we migrated from Unix a long time ago when we started deploying our applications on the cloud. The unix user permission model broke a long time ago with the introduction of cloud computing. It’s been a long time since you could pop onto a sun box in the mid 90s, see a half dozen users running their various programs and send a wall msg to everyone. Today for better or for worse that is all abstracted away on a different plane - AWS IAM or GCE IAM. Once again - this is because we are just using way too much software on way too many servers nowadays.

Then if you look at how a lot of devops people are deploying their applications - they could care less if they were deploying as root or a machine user or something else. Once again - for better or for worse this is now the state of things. You’ll have older security curmudgeons decry running things as root and using this logic almost any docker/kubernetes based installation should be arguably failing every single security assessment out there for their failure to adhere to this model but honestly this model has been broken for quite some time now anyways. Once again - the cloud broke the unix model.

No shell? That removes an entire class of shell code exploits. Of course the security world is so insanely bad today that you don’t need to carefully craft nop sleds and find and construct elaborate rop gadgets to get what you want done. No, now today all it takes is a single curl request to pop a shell on a remote server. I kid you not - this is exactly what happened with Equifax. However it’s worth discussing why shells are bad to begin with from a security perspective. Now before the haterade piles on let’s get this out of the way. My IDE is tmux && vim - I live in the shell. I have mutt and irssi installed for crying out loud - yet that is my dev environment - it’s not a production server and there is a very clear difference between the two.

Here’s the deal, attackers don’t necessarily care at all about which bug or which exploit they manage to breach your server with. Just look at the tooling that some app sec teams or SOCs use. The goal of the attacker is to land a beachhead on a server that gives them access to what is usually a very open internal network. Networking segmentation products aside this is very very common. There are a *lot* of companies that think they can simply hide all of their internal software applications behind a VPC and call it a day but that's just the firewall fallacy in full effect. If I root a web app server there’s a very good chance I can now freely talk to your database. Or perhaps maybe I am now looking at a kakfa queue - it doesn’t really matter. The point is is that now I’m in and now I’m going to enumerate it and see what else is there. There’s probably a decent percentage of servers that have github keys laying around for 'ease of deployment' or what about all those ssh keys that I've seen lay around in prod deployments cause "we don't use passwords". Let me just examine that bash history...

The list just goes on and it’s all because the shell is inherently built for running multiple programs. Think of every single command you can type into the shell - those are all different programs running. I know this is an overly redundant statement to make here but sometimes it’s hard for people imagine what we are proposing here because it is such a foreign concept.

At the end of the day a hacker wants a shell. Don't give it to them.

Ok let’s talk about this one as this is the one that most people point at. Yes this is true. Most unikernels kernel portion weigh in at tens of thousands of lines of code while Linux is 15M lines of code - 7-9M depending on your count is drivers. I often, in demos with people, will show them that their new VMs become roughly the size of their application since the other portion is so tiny. There is no reason for it to be big - as a VM appliance we don't need support for 20 different ethernet adapters and 30 different audio drivers. It needs to talk to the disk. It needs to talk to the network. That's about it.

Keep in mind that "modern" distributions like Ubuntu aren’t just the 15M lines of code in the kernel but also the rest of the operating system which can be ~50M lines of code. Fedora can reach up into the stratosphere at 200M lines of code. There are some unikernels that focus on 'provably correct' implementations and they are small enough to do so - it clearly gets progressively harder if you want to do something like that as your LOC count goes up. Most unikernels statically link their libraries but the ones that can dynamically load there isn’t much there. Maybe ten - maybe none.

Listen - I’m not going to lie and say that unikernels will solve all of your security woes - they obviously won’t however they have very clear security advantages and in the world in which we live in of 2018 where even the most respected tech companies are facing severe security issues every single day that are resulting in extremely large financial repercussions it’s worth taking a look at. As of this writing Uber has settled for a $150M suit against them for a data breach they tried to cover up. Facebook is facing a 1.6B fine and Google also tried to cover up a security issue which they ultimately decided to shutter an entire social product.

It's hard to deny the security benefits of unikernels and security is just one of their benefits.