Open sourcing our infrastructure · 2013-11-08 09:00 by Archive

About a year ago, we began to seriously scale up our infrastructure, moving from one overworked server to currently 22, with 13 of these being dedicated filter download servers.

We started with one server with a dozen different functions, and a range of scripts that kept it all together. While this works well for a single server, it’s a bit hard to scale, so we moved to a configuration management tool, Puppet, which is also used by Mozilla, Wikimedia and many others.

Puppet manifests are basically code, so we’ve been planning to open source all of that from the start. Yet until recently, we’ve been worrying that putting our server configuration in the open would make us vulnerable to attackers, so we wanted to make some changes that wouldn’t give away the real server names and functions first. But by now, we trust our infrastructure enough to open source everything as-is. So we did that. Like most of our repositories, it’s also mirrored to GitHub.

As with our other projects, contributions are very much appreciated – there’s a ton of things that need doing. Getting started is pretty easy, the README explains it in depth. In a nutshell: You can easily set up local virtual machines that are set up just like the production servers and use these to work on the Puppet manifests. Then you can submit a patch for review, and once it’s accepted we’ll deploy your changes to the production environment.

Feel free to drop by in our IRC channel if you’d like to work on something, here’s a quick overview of what we want to do in the near future:

There’s still one multi-purpose server that isn’t set up via Puppet – we want to migrate it’s services to new single-purpose servers.

We want to set up some new services, like a CI server.

We’re in the middle of renaming all servers based on their purpose – renaming servers is a fairly simple task to get started with.

We would like to use an External Node Classifier to make sure various server-specific information isn’t spread out in a dozen different configuration files.

Commenting is closed for this article.