When I set up the personal computer cluster‘s individual nodes, the SSD was set up to have three partitions. The partition mounted at /mnt/brick will be used to set up a GlusterFS volume. This GlusterFS volume will be used to persist data for services we run on the Docker Swarm that was previously set up on the cluster.

The GlusterFS software needs to be installed on all of the nodes in the cluster. Here is here pssh makes it easy to set up all nodes at the same time.

parallel-ssh -i -h ~/cluster/all.txt -l root "apt-get update" parallel-ssh -i -h ~/cluster/all.txt -l root "apt-get install -y software-properties-common" parallel-ssh -i -h ~/cluster/all.txt -l root "add-apt-repository ppa:gluster/glusterfs-6" parallel-ssh -i -h ~/cluster/all.txt -l root "apt-get update" parallel-ssh -i -h ~/cluster/all.txt -l root "apt-get install -y xfsprogs attr glusterfs-server glusterfs-common glusterfs-client" parallel-ssh -i -h ~/cluster/all.txt -l root "systemctl start glusterd" parallel-ssh -i -h ~/cluster/all.txt -l root "systemctl enable glusterd"

Now set up the GlusterFS volume from the master node and set the the master node’s firewall to enable all traffic from within the cluster’s LAN:

sudo ufw allow from 10.1.1.0/24 sudo ufw reload sudo gluster peer probe node1 sudo gluster peer probe node2 sudo gluster peer probe node3 sudo gluster peer probe node4 sudo gluster pool list

With the last command, you should see the list of nodes all connected. node1 will be listed as localhost since you are currently on it. The next step is to create a GlusterFS volume. First a folder on the /mnt/brick partition is created on every node, then the volume is created:

parallel-ssh -i -h ~/cluster/all.txt -l root "mkdir /mnt/brick/1/" sudo gluster volume create gfs01 \ node1:/mnt/brick/1/ \ node2:/mnt/brick/1/ \ node3:/mnt/brick/1/ \ node4:/mnt/brick/1/ sudo gluster volume list

Note that the GlusterFS volume is created to be a distributed volume. You could create it as a replicated volume in order to have data reliability, though this would create higher latency on writes. The last command will show if the volume gfs01 is available. If it was created with no problem, start the volume and grant access to it with:

sudo gluster volume start gfs01 sudo gluster volume set gfs01 auth.allow 10.1.1.1,10.1.1.2,10.1.1.3,10.1.1.4

Finally, mount the GlusterFS volume to the /mnt/gfs mount point and set it up to remount on reboots:

parallel-ssh -i -h ~/cluster/all.txt -l root "mkdir -p /mnt/gfs" parallel-ssh -i -h ~/cluster/all.txt -l root "mount.glusterfs localhost:/gfs01 /mnt/gfs" parallel-ssh -i -h ~/cluster/all.txt -l root "echo 'localhost:/gfs01 /mnt/gfs glusterfs defaults,_netdev,backupvolfile-server=localhost 0 0' >> /etc/fstab" parallel-ssh -i -h ~/cluster/all.txt -l root "chown -R root:docker /mnt/gfs" parallel-ssh -i -h ~/cluster/all.txt -l root "chmod g+w /mnt/gfs"

Check that everything mounted OK:

parallel-ssh -i -h ~/cluster/all.txt -l michael "df -h"