I am working on automating the bootstrapping and provisioning of a zoo of cloud servers and had to decide (again) what path to choose. I could build my own customized image and go from there or use a vanilla out-of-the-box Linux image and set it up during bootstrapping.

In the past, I chose the first path, because it allowed to quickly start new servers from one custom image (for example with PHP + Nginx), but on every system upgrade or change I had to fix running servers and create a new image for every introduced change. Which was a hell of work, especially if I had to keep all images in sync between cloud regions and maybe even cloud providers.

So I decided this time to do it in a different way. I now use vanilla, out-of-the-box Linux images (Ubuntu 12.04 LTS in my case) and during bootstrapping I provisioning the image with the software and settings I need.

But there are also several ways to do that and to make a long story short, I did not do the shell scripting way, but did the enforcing-a-state way.

This whole enforcing state thing took me a while to get, because you do not write shell scripts which run commands to install software, but you define “there should be an Nginx and PHP5 on this machine installed and running”.

For doing this state-enforcing business, there are two 800 pound gorillas in the ring called Chef (from Opscode) and Puppet (from Puppetlabs). I did research, read books and experimented with both and it really seems that Chef, which is very Ruby-focused, seems to win over Puppet, which exists some years longer. Especially with the current (or past?) hype about Ruby Chef became the number one choice for many DevOps.

In the end I tried hard to get Chef running in a more productive environment and I just constantly hit limits or weird behaviors and got more and more frustrated. In general and that is of course my personal opinion, Chef seems to be very hard to install and to master for one reason: to endorse the expensive training courses of the company behind Chef and their commercial hosted-server offering.

So I turned desperate and already reserved 2 weeks in my calendar just about “hammering Chef in my brain” when a colleague recommended me another project called Salt.

Salt is “just” 18 months or so old, written in Python (big pro for me!) and the best thing I was working with since very long. My team and I switched now fully to Salt and at least one of us says every day “Salt is so cool, I love it!”.

Salt is like a mix of Chef/Puppet (defining states) and an easy way to communicate with machines directly (like with an MQ). The big difference to Chef is the architecture: the slave (called minion) does not pull for changes every bunch of minutes, which can cause weirdness, but has a standing connection to the master which allows instant changes and commands.

Here are the things I think make Salt the “better” Chef/Puppet, at least for us:

standing connection from master to minions using ZeroMQ instant changes and command, can be even used as SSH replacement very scalable due to ZeroMQ very secure due to PKI easy gathering of minion statistics immediately see if a minion is down capability of enforcing states through YAML state files YAML is more readable than Ruby capability of executing remote commands on one, all or a selection of minions very fine grained tagging possible using “grains” installation of a Salt server within seconds! excellent documentation, even with tutorials very active and friendly user group just a handful of project owners allow fast updates very active open source community no holding back of features due to commercial reasons very rapid adding of features and issue fixing codebase is Python, very modular and understandable and allows own bugfixes(!)

As summary, I am mostly impressed about the documentation, the easy installation and the absolutely mind-blowing support of the core dev team around Thomas Hatch. We also already filed issues on Github and fixed bugs and pull requests are quickly considerated and added to the official codebase.

I expect that more, especially Python, developers start using Salt and will never look back to other solutions. Within 2-3 years Salt surely becomes to Python what Chef became for Ruby and I am very much looking forward to that.

My team and I will continue to contribute to Salt and I very much recommend it to every one who has the need to automate server provisioning or remote execution on a bunch of servers. Start with the tutorials at http://docs.saltstack.org/en/latest/index.html and write me your opinion!