Last Updated: January 12, 2017

You may have heard of or seen the Kubernetes Raspberry Pi cluster. In this post I share everything I learned when I built one myself and demonstrated it live.

Kubernetes Raspberry Pi cluster by Arjen Wassink from Quintor

Hardware

You can find all the parts you need in Arjen Wassink’s Kubernetes Raspberry Pi Cluster Shopping List.

I highly recommend the ModMyPi stack case listed in Arjen’s shopping list, because the dimensions fit perfectly with the network switch and the USB power hub. I ordered the GeauxRobot Raspberry Pi 2 B 4-layer Dog Bone Stack because it had one-day delivery. However, the dimensions are too small and my cluster doesn’t look nearly as good :(

My Kubernetes Raspberry Pi cluster — doesn’t look nearly as good as Quintor’s

If you have access to a 3D printer, you can print a custom stack case too! Pance Cavkovski shared his 3D design on Twitter and Thingiverse:

Lastly, I also recommend a 2ft cable rather than a 1ft cable for easier access.

Software

Arjen posted the Kubernetes Raspberry Pi Installation Instructions — it’s easy to install and comes in a single package.

Recently, I started to use the kubernetes-on-arm project. It comes with pre-built Kubernetes images, but it also allows you to build the latest Kubernetes version.

There are also many other detailed resources, such as detailed instructions using ArchLinux and Kubernetes, how to compile Kubernetes from scratch for Raspberry Pi, and Cluster computing with Raspberry Pi with Kubernetes.

Networking

There are many strategies to connect your cluster to the Internet. I picked the simplest setup that routes all the traffic via my MacBook laptop.

In OS X, you can enable Internet Sharing; choose System Preferences > Sharing and select the Internet Sharing option. You should share the connection from Wi-Fi to computers using an Apple USB Ethernet Adapter.

However, I needed to assign a static IP address to each Raspberry Pi (especially the one that runs the Kubernetes master). I spent hours trying to figure this out. To save you time, this is the trick when using OS X!

Get the Ethernet NIC MAC address of each Raspberry Pi In OS X, create a file named /etc/bootptab For each MAC address, add the following content



# Section 1 — ignored

%%

# Section 2 — used

# Hardware types: 1=Ethernet, 6=Wireless

# See

# hostname hwtype hwaddr ipaddr bootfile

rpi1 1 b8:27:eb:5b:0d:0f 192.168.2.101 boot

rpi2 1 b8:27:eb:a1:5f:5a 192.168.2.102 boot

.

.

.

rpiN 1 xx:xx:xx:xx:xx:xx 192.168.2.10x boot # Bootptab file# Section 1 — ignored%%# Section 2 — used# Hardware types: 1=Ethernet, 6=Wireless# See http://www.ietf.org/rfc/rfc1700.txt # hostname hwtype hwaddr ipaddr bootfilerpi1 1 b8:27:eb:5b:0d:0f 192.168.2.101 bootrpi2 1 b8:27:eb:a1:5f:5a 192.168.2.102 bootrpiN 1 xx:xx:xx:xx:xx:xx 192.168.2.10x boot

If your Raspberry Pi already received a DHCP assigned IP address in the past, you’ll need to force release and refresh for a new IP address. From the Raspberry Pi, run:

$ sudo dhclient -r

Reducing Failure Detection Periods for Demonstration Purposes

In Kubernetes, the controller manager component will check node health periodically, and then evict the pods on unhealthy nodes after a timeout period. There are a couple of command line arguments you should be aware of:

--node-monitor-grace-period=40s: Amount of time which we allow running Node to be unresponsive before marking it unhealty. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status.

--node-monitor-period=5s: The period for syncing NodeStatus in NodeController.

--pod-eviction-timeout=5m0s: The grace period for deleting pods on failed nodes.

To demonstrate how Kubernetes handles node failure in a live demo, I did not want to wait for 40-seconds to elapse before marking a node as unhealthy, nor can I wait for 5 minutes to see the pod being rescheduled. Thus, after installing Kubernetes on the Raspberry Pi, I needed to update the controller manager startup command line arguments.

If you used kubernetes-on-arm, you can update /etc/kubernetes/static-pods/master/master.json on the master node. Find the container configuration for controller-manager, and update the command attribute:

{

“name”: “controller-manager”,

“image”: “kubernetesonarm/hyperkube”,

“command”: [ “/hyperkube”, “controller-manager”,

“--master=127.0.0.1:8080”,

“--v=2”,

“--node-monitor-grace-period=10s”,

“--pod-eviction-timeout=5s”,

“--service-account-private-key-file=...",

“--root-ca-file=...” ],

“volumeMounts”:[...]

}

Base Images

Most of the existing container images are built for the x86 architecture. You need to run ARM binaries/images on the Raspberry Pi. Luckily, Hypriot created several Raspberry Pi compatible Docker base images you can use to run applications written in: Java, Go, Node, Ruby, and more. They also have MySQL and Redis images as well. You can find many other Raspberry Pi compatible images by searching for image names containing “rpi-”.

Visualization

I believe that visualization is extremely important during a live demo of a complicated process. To help visualize Kubernete’s failure detection and pod rescheduling, I used a version of a Kubernetes visualizer that was originally developed by Brendan Burns.

The visualizer code and how to use it are on my GitHub:

Last but not least, pre-pull all of the images from all nodes for a responsive demo.

Great Resources

There are many people working with the Raspberry Pi Kubernetes cluster. Many have shared what they are working on, including:

What’s next?

Scaling stateless workload is easy. But how do you schedule workload with persistent data? In the next part of this blog series, I’ll show you how I share a 250GB SSD as Kubernetes Persistent Volumes with LVM and NFS.

Last but not least, you can run Kubernetes in the cloud too! One of the easiest way to run a Kubernetes cluster is using Google Container Engine. Give it a try!