Nomad is one of the newest HashiCorp projects. It is a tool for managing a cluster of machines and running applications on them. It is similar to Mesos or Kubernetes but it is more simple. As Consul Nomad is a simple binary that may be launched without any dependencies. As a many Go lang projects there are no differences for server or client installation and there are many arguments may be passed through command line for changing behavior. Instead of Getting Started I was inspired by docker-consul project that simplify deployment process of Consul and separate server nodes from client nodes.

Main concept

I like Docker. It is one of the most influential projects now. And all the things (not all, ok) may be packed into Docker and can be launched easily. Three Docker images are needed for our needs. First, called its nomad , should be base image with minimum differences between image and original binary. Second should be a server agent and third should be client. Therefore if more freedom is needed base image is preferable. But if up and go is wanted server and client images saves your time.

Building base image

Docker images are builded from Dockerfile. Lets create one of them. Choose OS image as base layer. I prefer Alpine for lightweight software distribution.

FROM alpine:3.2

Define Nomad version and its signature for security reasons.

ENV NOMAD_VERSION 0.2.3 ENV NOMAD_SHA256 0f3a7083d160893a291b5f8b4359683c2df7991fa0a3e969f8785ddb40332a8c

Update Alpine packages, certificates and glibc.

RUN apk --update add curl ca-certificates && \ curl -Ls https://circle-artifacts.com/gh/andyshinn/alpine-pkg-glibc/6/artifacts/0/home/ubuntu/alpine-pkg-glibc/packages/x86_64/glibc-2.21-r2.apk > /tmp/glibc-2.21-r2.apk && \ apk add --allow-untrusted /tmp/glibc-2.21-r2.apk && \ rm -rf /tmp/glibc-2.21-r2.apk /var/cache/apk/*

Automatically download, unzip, check hashsum and make Nomad executable.

ADD https://releases.hashicorp.com/nomad/${NOMAD_VERSION}/nomad_${NOMAD_VERSION}_linux_amd64.zip /tmp/nomad.zip RUN echo "${NOMAD_SHA256} /tmp/nomad.zip" > /tmp/nomad.sha256 \ && sha256sum -c /tmp/nomad.sha256 \ && cd /bin \ && unzip /tmp/nomad.zip \ && chmod +x /bin/nomad \ && rm /tmp/nomad.zip

Three ports are needed to expose.

EXPOSE 4646 #HTTP EXPOSE 4647 #RPC EXPOSE 4648 #SERF

And last, define entrypoint.

ENTRYPOINT ["/bin/nomad"]

Next, create Makefile

build: docker build -t aatarasoff/nomad:0.2 .

Build image with make command and push it to Docker Hub.

Now standalone launch is possible.

docker run -d --net host --name nomad \ -p 4646:4646 -p 4646:4646/udp \ -p 4647:4647 -p 4647:4647/udp \ -p 4648:4648 -p 4648:4648/udp \ -v /var/run/docker.sock:/var/run/docker.sock \ aatarasoff/nomad -dev -bind=<your_binding_ip_address>

Test it. Create example Nomad job.

docker exec nomad nomad init

And run it.

docker exec nomad nomad run -address=http://<node_ip>:4646 example.nomad

Check that redis container is launched on the node.

Clustering

Standalone is good but not usable in real projects. Nomad has server nodes and client nodes. Server nodes use raft consensus algorithm for leader election. So, odd number of bootstraping servers is needed. Nomad support config files in hc1 format. It's similar to JSON but more readable. Create it for server type.

# Increase log verbosity log_level = "INFO" # Setup data dir data_dir = "/data" # Enable the server server { enabled = true bootstrap_expect = 3 }

bootstrap_expect = 3 means that three available servers are needed before server agents start.

Pack this config to new docker image.

FROM aatarasoff/nomad:0.2 ADD ./config /config/ VOLUME /data

And pass its through entrypoint args:

ENTRYPOINT ["/bin/nomad", "agent", "-config=/config/config.hc1"]

Build and ship image as it is described early. After that launch three containers on different machines.

docker run -d --net host --name nomad \ -p 4646:4646 -p 4646:4646/udp \ -p 4647:4647 -p 4647:4647/udp \ -p 4648:4648 -p 4648:4648/udp \ -v /var/run/docker.sock:/var/run/docker.sock \ aatarasoff/nomad-server -bind=<your_binding_ip_address>

And link them into cluster:

docker exec nomad nomad server-join -address http://<node2_ip>:4646 <node1_ip>:4648 docker exec nomad nomad server-join -address http://<node3_ip>:4646 <node1_ip>:4648

Make similar actions for building client image. Create client config file:

# Increase log verbosity log_level = "INFO" # Setup data dir data_dir = "/data" # Enable the client client { enabled = true }

Build it and push it and launch it on client machines.

docker run -d --net host --name nomad \ -p 4646:4646 -p 4646:4646/udp \ -p 4647:4647 -p 4647:4647/udp \ -p 4648:4648 -p 4648:4648/udp \ -v /var/run/docker.sock:/var/run/docker.sock \ aatarasoff/nomad-client -bind=<your_binding_ip_address> -servers http://<node1_ip>:4647

Check that client nodes is available:

curl <node_ip>:4646/v1/nodes

And run example job as it was running in standalone.

Project is available on GitHub: https://github.com/aatarasoff/docker-nomad

Images are available on DockerHub: