A buzzword

The Information Systems industry, as an almost inevitable side-effect of dealing in innovation, suffers from a disease of buzzwordification. As new concepts are developed for discussion, new words are needed. As those ideas become popular or trendy, the definitions of words tend to get away from those who coined them. In the worst case, they can diverge to the point of near-meaninglessness. While this can make for some fun bingo games , it also sows an unfortunate amount of miscommunication and resultant confusion.

In this article, I wish to thoroughly tackle “DevOps”, a buzzword close to my heart. If you don’t work in web operations, and you haven’t been looking at software job postings of late, you may never have seen it, but if you work on the web, you likely will before long. If you have seen it, you may or may not understand what it means. You can read the Wikipedia article, but I don’t think it’s terribly illuminating. But even if that makes sense to you, it is a partial and fairly unopinionated treatment. I seek here to be both fuller and more opinionated in laying out my personal understanding. I hope also to be a little less confusing.

As divergent definition is part of the buzzword game, I will seek to address some of the common understandings in turn, and to order my discussion in terms of progressively broader and grander visions of the idea. It is my hope that this will make the divergence seem relatively easy to follow. Let’s begin:

Agile, Automated Infrastructure

If you see a posting for a “devops engineer”, this is what the recruiter or hiring manager was likely thinking of: An expert in certain “configuration management” tools such as [Puppet or Chef] and Vagrant (or more recently Ansible and perhaps Docker) that allow for the automated setup and maintenance of servers (and their services). These servers may be found in a traditional data center, but more than likely, you’ll also find attached a more widely known buzzword—one that has inspired a Microsoft catchphrase and an upcoming R-rated comedy— The Cloud!

“Nobody Understands the Cloud! It’s a F#!king Mystery! — Jason Segel, “Sex Tape”, Summer 2014

(While the cloud is not fundamentally necessary for this “devops”, it provided an important change of norms. What previously needed to be done by physically moving and touching hardware could now be done by API call. These tools expand on that idea.)

History

To the credit of the job-posters above, the earliest history (as recorded by Damon Edwards) is on the side of this definition. At the strictest level, the word “devops” began as a shortened hashtag for discussion of the first DevOpsDays conference organized by Patrick DeBois in Ghent, Belgium in October 2009. This conference, in turn, arose from two major inspirations:

John Allspaw’s “10+ Deploys Per Day: Developer and Operations Collaboration at Flickr” presentation at O’Reilly’s 2009 Velocity web performance conference. but before that Andrew Clay Shafer’s proposed “Agile Infrastructure” session at the Agile2008 Conference (which literally no one but DeBois attended)

And in this word soup, the roots of this understanding are pretty clearly present.

The Problems

By this narrow interpretation of DevOps, the goal is basically to eliminate a few common problems with developing and running software as a service:

Developers’ computers are set up differently than the machines that run the software in production (and also different then other developers’ machines). This can lead to arbitrary behavioral differences and difficulty in debugging. Production machines are configured and tuned over time, in a way that is often not well-documented and therefore not repeatable for new machines. This makes scaling by adding new machines very hard. Quickly testing (and iterating on) changes to server configuration is untenable due to a) unknown starting state, b) the inability to readily duplicate so as to not test on live traffic, and c) lack of tooling for automated comparison of results

The Solution

All of these problems are eliminated by the approach known as “Infrastructure as Code” allowed by the sorts of tools listed above.

By defining and changing the configuration as code (or data), setup and maintenance of machines is imminently repeatable and inherently documented. Further, by treating configuration as code, operations staff can benefit from development tools and practices like revision control systems, code review processes, and community/open source libraries.

On top of laying out an initial state that can be spun up on demand (and tinkered with to test things out), the infrastructure code defines a desired state that can be enforced and converged upon over time. In this way, configuration management systems like those above (and the forerunner CFEngine, whose physicist creator Mark Burgess provided this formulation back in 98) can act like a computer’s immune system, maintaining homeostasis in the face of configurational drift, regardless of source.

These systems can be kind of complicated (or at least hard to master), and with their close interrelation (and potential integration) to more traditional operations infrastructure systems concerns—like monitoring, alerting, log-aggregation, networking, and database administration—as well as new concerns created or invigorated by the rise of cloud computing—such as autoscaling, zone failover, and general partition tolerance, it is not terribly surprising that many companies are looking for experts in these things and calling them “devop” or “DevOps engineers”. Different schools of thought, however, suggests that DevOps should fundamentally not be thought of as a job title, but rather a powerful idea for the general operation of IT organizations, and indeed any organization with IT concerns. We move on to those schools…