Puppetizing and Deploying Emacs

ThoughtWorks employ me as a Continuous Delivery consultant, not actually as a Lisp hacker. Apparently Lisp hackers are in less demand than continuous delivery consultants.

What sort of world do we live in? Not an optimal one I reckon.

Anyway, this means that I spend way too much of my time writing Puppet scripts and using Vagrant virtual machines. But sometimes the lisp hacking and the continuous delivery expertize come together. For example, I've been doing some stuff with Elnode and other Emacs servers that will benefit from being packaged in a Vagrant VM.

I find Puppet very dissatisfying, but it's better than trying to provision boxes in other ways. It is declarative for example. That's a good thing. But it's requires much mundane verbosity in the form of repeated tedious directory structure. That's a bad thing. Package managers like, apt , yum and those... are great but packages for them are necessarily less portable than puppet and harder to write.

Emacs can be started as a daemon, so we can host code in it quite happily. But it's hard to deploy to. So I've written some puppet to make it easy:

class emacs { group { "emacs" : ensure => "present" } user { "emacs" : gid => "emacs" , home => "/home/emacs" , } file { "/home/emacs" : ensure => "directory" , owner => "emacs" , group => "emacs" , mode => "u=rwx,g=rx,o=rx" , } package { "curl" : } $emacs_dist_url = "https://github.com/downloads/nicferrier/heroku-buildpack-emacs/emacs.tgz" exec { "emacs-tarball" : require => Package[ "curl" ], path => "/usr/bin:/bin" , command => "curl -Ls $emacs_dist_url -o /tmp/emacs-24.tgz" , creates => "/tmp/emacs-24.tgz" , user => "emacs" , group => "emacs" , logoutput => true } exec { "emacs-dist" : require => [File[ "/home/emacs" ], Exec[ "emacs-tarball" ]], path => "/usr/bin:/bin" , cwd => "/home/emacs" , command => "tar xvzf /tmp/emacs-24.tgz" , creates => "/home/emacs/emacs/bin/emacs" , refresh => "rm -rf /home/emacs/emacs" , logoutput => true, user => "emacs" , group => "emacs" , } file { "emacs-initd" : require => Exec[ "emacs-dist" ], source => "puppet:///modules/emacs/initd" , path => "/etc/init.d/emacs" , owner => "root" , group => "root" , mode => "u=rx,g=rx" , } service { "emacs" : require => File[ "emacs-initd" ], ensure => "running" , enable => "true" , hasstatus => "true" , } }

NB: I wish puppet accepted the fact that install tarballs is such a common exercise and provided direct support for them.

This installs a usable Emacs 24 daemon and starts it with a daemon script that is also installed.

Daemon Init

Daemon scripts for Emacs have kicked around for a while, ever since Emacs could be daemonized, which was a while ago. I've taken an existing one and adapted it a lot. My one can install packages and call commands (which you need to be able to do to stop and start servers for example). It also bootstraps itself with an Emacs init file.

The initd has it's own repository on github.

Starting Emacs like this makes it possible to package elnode, for example, like this:

class elnode { exec { "install-elnode" : require => Service[ "emacs" ], command => "/etc/init.d/emacs install elnode" , } }

It should also make it possible to package emacs packages more like other apps.

For example, one could treat an elnode instance as a service, but it would need separate start and stop scripts in init.d. That wouldn't be hard of course, they can just:

case $ 1 in start) /etc/init.d/emacs command elnode-start "0.0.0.0" 8001 ;; stop) /etc/init.d/emacs command elnode-stop 8001 ;; esac

It does seem tedious to have to write this script over and over. Perhaps puppet could write it somehow or perhaps Emacs packages should provide scripts like that.

It also seems that it might be fun to give puppet a package handler that supported emacs packages, so you could actually say:

class some-emacs-thing { package { "elnode" : provider => "emacs" , repository => "http://marmalade-repo.org/packages/" , } }

Maybe that could even do all the work of spinning up the emacs server if it wasn't already there. There is precedent for this, Ruby and Python both have package extensions to puppet to make this stuff work.

It's something I'll investigate if people make enough noise at me.

Maven. The best bits. For Emacs.

Starting Emacs as a daemon and putting packages in it is only the beginning of the evil things I've done in the name of Emacs deployment. Perhaps the epitomy is to implement maven in EmacsLisp.

For those who don't know, maven, is the worst tool ever designed. Ok, ok: in my opinion. But it's not just me.

It does have one cool thing though, the separation of packages from repositories. This has now been copied in a lot of packaging tools. Emacs' ELPA has it in theory, you can specify an HTTP archive such as GNU's ELPA archive or Marmalade or you can use a plain old directory to install packages from.

In practice, file archives haven't been very useful because there weren't many tools to utilize them.

So I have written elpakit.

elpakit is a tool to help with packaging Emacs packages. But also to allow you to make a temporary archive of them.

(elpakit "vagrant/packages" '( "~/work/elnode-auth" "~/work/emacs-db" "~/work/shoes-off" "~/work/rcirc-ssh" ))

Presuming those directories contain dev trees of Emacs packages then elpakit will build the packages and make a package archive in vagrant/packages .

This is the beginnings of package based project management for Emacs. It seems like the list of packages you use to make a private archive could become your project definition, you could track drift from git repos across the project, manage the archive as a whole, pushing and pulling it to places, co-ordinate branching across the parts of the project.

Like other stuff I'm doing what is fun about this is that we're inventing it. It's a brand new time for Emacs. Come and help out!