Introduction

I cooked up something at work that I think is going to be awesome for building the project that I work on a day to day basis. Here's the basic problem, in a different post, I'll expand on these in more depth:

we do continuous deployment.

we maintain and publish multiple branches.

our builds take a non-trivial amount of time (4-6 minutes depending on hardware,) and will continue to get longer.

our documentation toolkit, Sphinx, lacks concurrency for some steps, which means builds take too long and leave most of a contemporary computer idle. Furthermore, given our use of topic-branches it can be hard to get work done during a build.

So there are a couple of notable hacks that I've come up with, over the past few months that help:

duplicate some of the initial work so that different output formats can build in parallel (using Make's job control) at the expense of disk some space.

using a source proxy, (i.e. copying the source content into the build directory and building from this copy so that the actual source files can change during the build.)

These changes are simple and amount to some really minor changes to commands and Make files. This next fix required a non-trivial amount of code, but is really awesome: