Since I last properly wrote about Docker a lot has changed. Docker have introduced some new command line tools which allow for easy orchestration of Docker instances, clusters and container management. These are;

Docker Machine — Allows you to easily deploy Docker instances to a lot of different platforms.

Docker Compose — A replacement for Fig.

Docker Swarm — Native clustering for your Docker instances.

Out of these three technologies, Swarm is not really suitable for production use so I won’t go into it in too much detail in this post.

Docker Machine

Rather than downloading the pre-compiled binary I decided to use the Homebrew formula (this assumes you have Cask installed);

# Make sure everything is up-to-date brew update brew doctor brew cask update brew cask doctor # install docker-machine brew cask install docker-machine

This will install docker-machine;

docker-machine -v docker-machine version 0.1.0 docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM

I already have VirtualBox installed so lets create a machine called “Testing”;

docker-machine create — driver virtualbox testing INFO[0000] Creating SSH key… INFO[0000] Creating VirtualBox VM… INFO[0006] Starting VirtualBox VM… INFO[0006] Waiting for VM to start… INFO[0038] “testing” has been created and is now the active machine. INFO[0038] To point your Docker client at it, run this in your shell: $(docker-machine env testing)

docker-machine comes with a few commands which will help you connect using the locally installed docker client;

docker-machine env testing export DOCKER_TLS_VERIFY=yes export DOCKER_CERT_PATH=/Users/russ/.docker/machine/machines/testing export DOCKER_HOST=tcp://192.168.99.100:2376 docker-machine config testing — tls — tlscacert=/Users/russ/.docker/machine/machines/testing/ca.pem — tlscert=/Users/russ/.docker/machine/machines/testing/cert.pem — tlskey=/Users/russ/.docker/machine/machines/testing/key.pem -H=”tcp://192.168.99.100:2376

Thats it, I now have a Virtual Machine launched and ready for me to start using Docker;

docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM testing * virtualbox Running tcp://192.168.99.100:2376

As with any new installation, lets run a “Hello World”;

docker $(docker-machine config testing) run busybox echo hello world Unable to find image ‘busybox:latest’ locally 511136ea3c5a: Pull complete df7546f9f060: Pull complete ea13149945cb: Pull complete 4986bf8c1536: Pull complete busybox:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.

Status: Downloaded newer image for busybox:latest hello world

Finally you can SSH to the Virtual Machine using docker-machine ssh machine-name;

docker-machine ssh testing Boot2Docker version 1.5.0, build master : a66bce5 — Tue Feb 10 23:31:27 UTC 2015 Docker version 1.5.0, build a8a31ef [email protected]:~$ uname -a Linux testing 3.18.5-tinycore64 #1 SMP Sun Feb 1 06:02:30 UTC 2015 x86_64 GNU/Linux [email protected]:~$ cat /etc/*release NAME=Boot2Docker VERSION=1.5.0 ID=boot2docker ID_LIKE=tcl VERSION_ID=1.5.0 PRETTY_NAME=”Boot2Docker 1.5.0 (TCL 5.4); master : a66bce5 — Tue Feb 10 23:31:27 UTC 2015" ANSI_COLOR=”1;34" HOME_URL=”http://boot2docker.io" SUPPORT_URL=”https://github.com/boot2docker/boot2docker" BUG_REPORT_URL=”https://github.com/boot2docker/boot2docker/issues" [email protected]:$ exit

Great, so I now have a Virtual Machine running on my local computer, what more is there?

docker-machine is designed to be used with the following public and private cloud providers (more are being added all of the time);

Amazon EC2

Microsoft Azure

Digital Ocean

Google Compute Engine

Rackspace

SoftLayer

OpenStack

VMWare vCloud Air

VMWare vSphere

Lets use docker-machine to launch a Digital Ocean droplet. To do this you will need to generate a Personal Access Token, you can do this by following these instructions. Once have the token launch the droplet as follows;

docker-machine create → — driver digitalocean → — digitalocean-access-token cdb81ed0575b5a8d37cea0d06c9690daa074b1276892fc8473bdda97eb7c65ae → dotesting INFO[0000] Creating SSH key… INFO[0000] Creating Digital Ocean droplet… INFO[0004] Waiting for SSH… INFO[0071] Configuring Machine… INFO[0120] “dotesting” has been created and is now the active machine. INFO[0120] To point your Docker client at it, run this in your shell: $(docker-machine env dotesting)

(and no, that is not my Digital Ocean Personal Access Token, it’s just a random string)

So what just happened? docker-machine accessed my Digital Ocean account via the API and launched the following Droplet;

OS = Ubuntu 14.04 x64

RAM = 512MB

HDD = 20GB SSD

Region = NYC3

These defaults can be over-ridden by providing more options, run docker-machine create — help for a full break down of the options along with example arguments.

Once the droplet had finished booting, docker-machine connected to the droplet via SSH installed, configured and started the latest version of Docker.

So, we now have two machines launched, one locally and one in Digital Ocean;

docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dotesting * digitalocean Running tcp://45.55.134.248:2376 testing virtualbox Running tcp://192.168.99.100:2376

lets run the “hello world” again, but this time use the droplet which has just been launched;

docker $(docker-machine config dotesting) run busybox echo hello world Unable to find image ‘busybox:latest’ locally 511136ea3c5a: Pull complete df7546f9f060: Pull complete ea13149945cb: Pull complete 4986bf8c1536: Pull complete busybox:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.

Status: Downloaded newer image for busybox:latest hello world

and finally SSH into the machine;

docker-machine ssh dotesting Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0–43-generic x86_64)

System information as of Sat Mar 21 07:24:02 EDT 2015

System load: 0.43 Processes: 72 Usage of /: 11.4% of 19.56GB Users logged in: 0 Memory usage: 12% IP address for eth0: 45.55.134.248 Swap usage: 0% IP address for docker0: 172.17.42.1

Graph this data and manage this system at: https://landscape.canonical.com/

[email protected]:~# docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE busybox latest 4986bf8c1536 11 weeks ago 2.433 MB [email protected]:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b8a83077d858 busybox:latest “echo hello world” 4 minutes ago Exited (0) 4 minutes ago kickass_almeida [email protected]:~# exit logout

Finally, you can stop and remove machines using docker-machine stop machine-name and docker-machine rm machine-name, be careful when using rm as it does not prompt you if you are sure;

docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dotesting * digitalocean Running tcp://45.55.134.248:2376 testing virtualbox Running tcp://192.168.99.100:2376 docker-machine stop dotesting docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dotesting * digitalocean Stopped tcp://45.55.134.248:2376 testing virtualbox Running tcp://192.168.99.100:2376 docker-machine rm dotesting docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM testing virtualbox Running tcp://192.168.99.100:2376

So thats a quick overview of docker-machine. As you can see, it is a a really convenient way to bootstrap Docker server instances across many different providers and tear them down all using a single command from your local machine.

Docker Compose

Docker Compose started life as Fig which is something I have written about before in a previous post, the currently release doesn’t add too much in the way of new functionality, but it does start laying the foundations for working with docker-swam, click here for the full release notes.

Like docker-machine I installed it using a Homebrew formula;

brew install docker-compose ==> Downloading https://homebrew.bintray.com/bottles/fig-1.1.0.yosemite.bottle.1.tar.gz ################################################################## 100.0% ==> Pouring fig-1.1.0.yosemite.bottle.1.tar.gz ==> Caveats Bash completion has been installed to: /usr/local/etc/bash_completion.d ==> Summary /usr/local/Cellar/fig/1.1.0: 186 files, 2.2M

So using docker-machine lets create a Docker server instance to use docker-compose with;

docker-machine create — driver virtualbox compose INFO[0001] Creating SSH key… INFO[0001] Creating VirtualBox VM… INFO[0007] Starting VirtualBox VM… INFO[0008] Waiting for VM to start… INFO[0041] “compose” has been created and is now the active machine. INFO[0041] To point your Docker client at it, run this in your shell: $(docker-machine env compose)

as docker-compose doesn’t interact directly with docker-machine we need to tell the main docker client the details of the server instance which has just been launched;

$(docker-machine env compose)

this command injects the environment variables needed for the docker client to connect to the server instance, to see these you can just run docker-machine env machine-name on it’s own;

docker-machine env compose export DOCKER_TLS_VERIFY=yes export DOCKER_CERT_PATH=/Users/russ/.docker/machine/machines/compose export DOCKER_HOST=tcp://192.168.99.100:2376

From here, it is just like Fig, apart from fig.yml file should now be called docker-compose.yml, I had a fig.yml file from a previous post still on my machine;

web: image: russmckendrick/nginx-php volumes: — ./web:/var/www/html/ ports: — 80:80 environment: PHP_POOL: mywebsite links: — db:db db: image: russmckendrick/mariadb ports: — 3306 privileged: true environment: MYSQL_ROOT_PASSWORD: wibble MYSQL_DATABASE: wibble MYSQL_USER: wibble MYSQL_PASSWORD: wibble

It launches two containers and links them together along with mounting the ./web folder in the NGINX container. The directory structure of the folder I am going to be running docker-compose from looks like;

tree -a . ├── [russ 356] docker-compose.yml └── [russ 102] web └── [russ 67] index.php

1 directory, 2 files

To start with I pulled the images which are going to be launched, you can ignore this part, it just makes showing whats going on in this post more straight forward;

docker-compose pull Pulling db (russmckendrick/mariadb:latest)… Pulling web (russmckendrick/nginx-php:latest)…

Now the images have been pulled down it’s time to launch the containers;

docker-compose up -d Creating example_db_1… Creating example_web_1…

We now have two running containers;

docker-compose ps Name Command State Ports — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — example_db_1 /usr/local/bin/run Up 0.0.0.0:49154->3306/tcp example_web_1 /usr/local/bin/run Up 0.0.0.0:80->80/tcp

You can also open your browser using;

In my example I see a phpinfo() page.

Once the containers are running you can connect to them using docker exec;

docker exec -it example_web_1 bash [[email protected] /]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.2 1.5 115200 15360 ? Ss 13:59 0:01 /usr/bin/python /usr/bin/supervisord -n root 16 0.0 3.2 382876 33624 ? S 13:59 0:00 php-fpm: master process (/etc/php-fpm.conf) root 17 0.0 0.2 110016 2096 ? Ss 13:59 0:00 nginx: master process nginx nginx 18 0.0 0.5 110472 5568 ? S 13:59 0:00 nginx: worker process webserv+ 19 0.0 1.5 383132 16284 ? S 13:59 0:00 php-fpm: pool mywebsite webserv+ 20 0.0 0.8 382876 8848 ? S 13:59 0:00 php-fpm: pool mywebsite webserv+ 21 0.0 0.8 382876 8848 ? S 13:59 0:00 php-fpm: pool mywebsite webserv+ 22 0.0 0.8 382876 8848 ? S 13:59 0:00 php-fpm: pool mywebsite webserv+ 23 0.0 0.8 382876 8852 ? S 13:59 0:00 php-fpm: pool mywebsite root 95 0.0 0.4 91540 4740 ? Ss 13:59 0:00 /usr/libexec/postfix/master -w postfix 97 0.0 0.6 91712 6508 ? S 13:59 0:00 qmgr -l -t unix -u postfix 200 0.0 0.6 91644 6232 ? S 14:05 0:00 pickup -l -t unix -u root 234 2.3 0.2 11748 2968 ? S 14:07 0:00 bash root 250 1.0 1.1 110012 11616 ? S 14:07 0:00 nginx root 251 0.0 0.2 19756 2212 ? R+ 14:07 0:00 ps aux [[email protected] /]# exit exit

Finally you can stop and remove the containers, and the Docker instance;

docker-compose stop && docker-compose rm — force Stopping example_web_1… Stopping example_db_1… Going to remove example_web_1, example_db_1 Removing example_db_1… Removing example_web_1… docker-machine rm compose docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM

Docker Swarm

Before going any further, the documentation warns ….

Note: Swarm is currently in beta, so things are likely to change. We don’t recommend you use it in production yet.

Now that’s out of the way lets use Homebrew to install docker-swarm;

brew install docker-swarm ==> Downloading https://homebrew.bintray.com/bottles/docker-swarm-0.1.0.yosemite.bottle.tar.gz ################################################################## 100.0% ==> Pouring docker-swarm-0.1.0.yosemite.bottle.tar.gz /usr/local/Cellar/docker-swarm/0.1.0: 4 files, 8.7M

As we already have docker-machine installed I will be using it to locally create the cluster, first of all we need to launch an instance and run the swarm container;

docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM docker-machine create -d virtualbox local INFO[0001] Creating SSH key… INFO[0001] Creating VirtualBox VM… INFO[0006] Starting VirtualBox VM… INFO[0006] Waiting for VM to start… INFO[0039] “local” has been created and is now the active machine. INFO[0039] To point your Docker client at it, run this in your shell: $(docker-machine env local) $(docker-machine env local) docker run swarm create Unable to find image ‘swarm:latest’ locally 511136ea3c5a: Pull complete ae115241d78a: Pull complete f49087514537: Pull complete fff73787bd9f: Pull complete 97c8f6e912d7: Pull complete 33f9d1e808cf: Pull complete 62860d7acc87: Pull complete bf8b6923851d: Pull complete swarm:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.

Status: Downloaded newer image for swarm:latest 63e7a1adb607ce4db056a29b1f5d30cf

As you can see, I got a token when the container launched 63e7a1adb607ce4db056a29b1f5d30cf I will need this to add more nodes, but first we will need to create a Swarm master;

docker-machine create → -d virtualbox → — swarm → — swarm-master → — swarm-discovery token://63e7a1adb607ce4db056a29b1f5d30cf → swarm-master INFO[0000] Creating SSH key… INFO[0000] Creating VirtualBox VM… INFO[0006] Starting VirtualBox VM… INFO[0006] Waiting for VM to start… INFO[0038] Configuring Swarm… INFO[0043] “swarm-master” has been created and is now the active machine. INFO[0043] To point your Docker client at it, run this in your shell: $(docker-machine env swarm-master)

We then need to connect the Docker client to the swarm, this is done by adding — swarm to the $(docker-machine env machine-name) command;

$(docker-machine env — swarm swarm-master)

Now lets add another node;

docker-machine create → -d virtualbox → — swarm → — swarm-discovery token://63e7a1adb607ce4db056a29b1f5d30cf → swarm-node-00 INFO[0000] Creating SSH key… INFO[0000] Creating VirtualBox VM… INFO[0006] Starting VirtualBox VM… INFO[0006] Waiting for VM to start… INFO[0039] Configuring Swarm… INFO[0048] “swarm-node-00” has been created and is now the active machine.

We now have a 2 node cluster called “swarm-master”;

docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM local virtualbox Running tcp://192.168.99.100:2376 swarm-master virtualbox Running tcp://192.168.99.101:2376 swarm-master (master) swarm-node-00 * virtualbox Running tcp://192.168.99.102:2376 swarm-master

Using docker info gives more information about the cluster;

docker info Containers: 3 Nodes: 2 swarm-master: 192.168.99.101:2376 └ Containers: 2 └ Reserved CPUs: 0 / 4 └ Reserved Memory: 0 B / 999.9 MiB swarm-node-00: 192.168.99.102:2376 └ Containers: 1 └ Reserved CPUs: 0 / 4 └ Reserved Memory: 0 B / 999.9 MiB

Great, so what does all of this mean?

Lets pull some images down;

docker -H 192.168.99.101:2376 pull redis docker -H 192.168.99.102:2376 pull mysql

Notice how I have pulled redis on “swarm-master” and mysql on “swarm-node-00”, I can now make sure that containers only launch on a node where the image is available;

docker run -d — name redis1 -e affinity:image==redis redis af66148bbbc8dcd799d82448dfd133b968d34eb7066a353108bf909ea3324a58 docker run -d — name mysql -e affinity:image==mysql -e MYSQL_ROOT_PASSWORD=mysecretpassword -d mysql 70b2d93d6f83aa99f5ad4ebe5037e228a491a4b570609840f3f4be9780c33587 docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 70b2d93d6f83 mysql:latest “/entrypoint.sh mysq 3 seconds ago Up Less than a second 3306/tcp swarm-node-00/mysql af66148bbbc8 redis:latest “/entrypoint.sh redi 2 minutes ago Up 2 minutes 6379/tcp swarm-master/redis1

another example would be ports being used on a node, lets pull my NGINX & PHP image on both nodes;

docker -H 192.168.99.101:2376 pull russmckendrick/nginx-php docker -H 192.168.99.102:2376 pull russmckendrick/nginx-php

Now lets launch the a container and bind it to port 80;

docker run -d -p 80:80 russmckendrick/nginx-php 2d066b2ccf28d2a1fa9edad8ac7b065266f29ecb49a8753b972780051ff83587

and again;

docker run -d -p 80:80 russmckendrick/nginx-php 40f5fee257bb2546a639a5dc5c2d30f8fa0ac169145e431c534f85d8db51357f

Nothing special there you say? Well, normally, when trying to launch the second container you would have gotten the following error as you can not bind two containers to the same port;

docker run -d -p 80:80 russmckendrick/nginx-php FATA[0000] Error response from daemon: unable to find a node with port 80 available

However, in this case as Docker is aware of what is running on the nodes wothin the cluster including which ports are in use. Docker via Swarm simply launched the container on “swarm-node-00” and it knew that port 80 was already in use on “swarm-master”;

docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 40f5fee257bb russmckendrick/nginx-php:latest “/usr/local/bin/run” 4 seconds ago Up Less than a second 192.168.99.101:80->80/tcp swarm-master/elated_einstein 2d066b2ccf28 russmckendrick/nginx-php:latest “/usr/local/bin/run” 8 seconds ago Up Less than a second 192.168.99.102:80->80/tcp swarm-node-00/drunk_mestorf 70b2d93d6f83 mysql:latest “/entrypoint.sh mysq 26 minutes ago Up 26 minutes 3306/tcp swarm-node-00/mysql af66148bbbc8 redis:latest “/entrypoint.sh redi 29 minutes ago Up 29 minutes 6379/tcp swarm-master/redis1

All of this was done with no prompting or special commands, it helpfully just got on with it

As you can see, docker-swarm is still very much in-development and there are some deal breakers, like containers not being able to talk to each other across nodes. However, with the news that socketplane.io (they produce a container based software defined networking solution using Open vSwitch) is joining Docker I don’t think it will be too long before this problem is resolved.

Finally, lets remove the running instances;

docker-machine rm local swarm-master swarm-node-00

That’s it for now, expect a follow up post in the next few months as these tools are updated.