A little more than a year ago I released whenjobs which was an attempt to create a practical language for automating complex “business rules”. The kind of thing I’m talking about is managing the many diverse steps between me tagging a libguestfs commit with a version number and a fully tested tarball appearing on the website. Or the hundreds of steps that go into 100 OCaml packages being updated and rebuilt for Rawhide.

Whenjobs wasn’t the right answer. Goaljobs [very early alpha] might possibly be.

What I need is something which is flexible, can deal with failures (both hard and intermittent), and can be killed and restarted at any point.

The first observation is that make is nearly the right tool. It’s goal-based, meaning that you set down a target that you want to have happen, and some rules to make that happen, and this lets you break down a problem from the larger goal (“build my program!”) to smaller subgoals (“compile this source file”).

program: main.o utils.o cc $^ -o $@

The goal is “ program is built”. There are some requirements ( main.o , utils.o ), and there’s a recipe (run cc ). You can also kill make in the middle and restart it, and it’ll usually continue from where it left off.

Make also lets you parameterize goals, although only in very simple ways:

%.o: %.c cc -c $< -o $@

Implicit in the “:” (colon) character is make’s one simple rule, which is roughly this: “if the target file doesn’t exist, or the prerequisite files are newer than the target, run the recipe below”.

In fact you could translate the first make rule into an ordinary function which would look something like this:

function build_program () { if (!file_exists ("program") || file_older ("program", "main.o") || file_older ("program", "utils.o")) { shell ("cc -c %s -o %s", "main.o utils.o", "program"); } }

Some points arise here:

Why can’t we change the target test to something other than “file exists or is newer”?

How about “remote URL exists” (and if not, we need to upload a file)?

How about “Koji build completed successfully” (and if not we need to do a Fedora build)?

How about “remote URL exists” (and if not, we need to upload a file)? How about “Koji build completed successfully” (and if not we need to do a Fedora build)? What could happen if we could add parameters to build_program ?

Goaljobs attempts to answer these questions by turning make-style rules into “goals”, where goals are specialized functions similar to the one above that have a target, requirement(s), a recipe to implement them, and any number of parameters.

For example, a “compile *.c to *.o” goal looks like this:

let goal compiled c_file = (* convert c_file "foo.c" -> "foo.o": *) let o_file = change_file_extension "o" c_file in target (more_recent [o_file] [c_file]); sh " cd $builddir cc -c %s -o %s " c_file o_file

The goal is called compiled and it has exactly one parameter, the name of the C source file that must be compiled.

The target is a promise that after the recipe has been run the *.o file will be more recent than the *.c file. The target is both a check used to skip the rule if it’s already true, but also a contractual promise that the developer makes (and which is checked by goaljobs) that some condition holds true at the end of the goal.

sh is a lightweight way to run a shell script fragment, with printf-like semantics.

And the whole thing is wrapped in a proper programming language (preprocessed OCaml) so you can do things which are more complicated than are easily done in shell.