So I’ve been working a lot with docker lately, but I always hit a wall when I try to do multi-host syncing of volumes. I’ve long suspected that the solution would be to use Amazon EFS, which was very recently made generally available. In the following, I present a solution to creating distributed docker volumes on AWS.

In this post I make use of the aws-cli and jq, which can be installed on Ubuntu 16.04 by running

apt install awscli jq

First, make sure that you have a role that allows access to EFS, and create a AWS EC2 machine which uses that role. Since we’re making a cluster, creating an auto-scaling group with that role seems to be the way to go.

Make sure that ContainX’s netshare plugin is installed and running. On Ubuntu 16.04, this amounts to doing:



wget

dpkg -i docker-volume-netshare_0.18_amd64.deb

echo 'DKV_NETSHARE_OPTS="efs"' > /etc/default/docker-volume-netshare

systemctl enable docker-volume-netshare

systemctl start docker-volume-netshare apt-get update && apt-get install -y nfs-commonwget https://github.com/ContainX/docker-volume-netshare/releases/download/v0.18/docker-volume-netshare_0.18_amd64.deb dpkg -i docker-volume-netshare_0.18_amd64.debecho 'DKV_NETSHARE_OPTS="efs"' > /etc/default/docker-volume-netsharesystemctl enable docker-volume-netsharesystemctl start docker-volume-netshare

Since we’re on an instance that has a role which allows it to access EFS, no further configuration of netshare is needed.

We then create an EFS, using the aws-cli. This can also be done using the console, but showing how to do it in the cli is easier. Make sure you’ve run

aws config efs.preview true

first, since the cli for EFS is not yet stable.

To create the file system, we do

aws efs create-file-system --creation-token docker-volume-test

The creation token is to make this command idempotent, i.e. trying to create two file systems with the same token is an error.

The output of this command includes the file system id, which is what we use in netshare to mount the file system as a volume. However, since we are command line hackers, we’ll fetch the id directly from the cli

aws efs describe-file-systems | jq -r '.FileSystems[]|select(.CreationToken == "docker-volume-test")|.FileSystemId'

This returns the Id directly, which will be something like

fs-f3efc785

To be able to mount this EFS, we’ll need to create a mount point. We’ll want to know the subnet-id and availability zone of the current instance, to create a mount point there. Luckily, we can do this with a simple curl request, with dynamic data retrieval. So to create the mount point, you run:

aws efs create-mount-target --file-system-id fs-f3efc785 --subnet-id $(curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/$(curl http://169.254.169.254/latest/meta-data/mac)/subnet-id)

Though you’ll have to make sure that aws-cli is configured to the right region. We’ll have to wait a bit until the mount point is ready before we can proceed.