Installation and Setup

I have used the i5 board to run the master node and J1900 to run the worker nodes

Architecture Installing kubernetes Networking using WeaveNet Installing OpenVPN Initializing Helm and Tiller Installing Prometheus Challenges and Solutions Backing up kubernetes master

Architecture

This is how I have designed the system. I had raspberry pi which I used as an entry point to the kubernetes cluster. All the nodes are still connected to the router and have an IP address provided by the DHCP server of the router, but I connect to the cluster only through the raspberry pi. I run a reverse DNS service on my pi to connect it from the internet. This helps me manage better. You can run cronjobs and set up SSH tunnels to expose your kubernetes services to the internet in the PI.

Installing kubernetes

I do not want to rewrite this again, as there are great articles out there which can help you with this. The one I liked and used is https://blog.alexellis.io/kubernetes-in-10-minutes/

Networking using WeaveNet

After you install kubernetes, some of your pods will not provision until you set up networking. For some reason, my ubuntu 18.04 keeps on crashing when installing Flannel as mentioned in the article above. So instead, I used weavenet, which worked perfectly. You can find details on how to install it in the link below.

Installing OpenVPN

This is completely optional and depends on your use case. For me, this was very important as I wanted to access my services on Kubernetes using the hostnames as they are of type ClusterIP and not NodePort or LoadBalancer. Below is a great document on how you can install it.

Initializing Helm and Tiller

Helm and Tiller is great to manage and install apps

Installing Prometheus

Challenges and Solutions (1.14.2)

I faced a lot of challenges while setting up the infrastructure and I like to share them as it might help people who are working on such projects.

Service Load balancing: Unlike cloud infrastructure in GKE, the kubernetes in bare metal do support load balancing option out of the box. Which means, you cannot expose your services outside of your kubernetes using service type Load Balancer, you will need to use NodePort to expose the service and then access those services using a NodeIP:NodePort. This is another reason why a VPN set up would make more sense. But, fortunately, there is a solution called MetalLB which will help you use the LoadBalancer service type. You can install it using the below link

2. Persistent Volume: Local Storage is another such thing which does not come out of the box with Kubernetes installation. You have to manually set it up and there is no support for dynamic provisioning of volume for local storage as of 1.14.2.

Building this cluster while learning Kuberenetes was difficult for me, as I was not aware of a lot of kubernetes internals like persistent volume, storage classes, persistent volume claims and more. I figured about this challenge while trying to install a Redis instance, which required me to setup Persistent Volume. I read a lot about PV which confused me more on how to implement it correctly, as I was not aware of NodeAffinity and other things. Below is a document which helped me understand this by example.

Official Doc: https://kubernetes.io/blog/2019/04/04/kubernetes-1.14-local-persistent-volumes-ga/

Example: https://github.com/stefanprodan/k8s-podinfo/blob/master/docs/4-statefulsets.md

Backing Up Kubernetes Master

One random day after setting up my cluster completely, my master node crashed and Ubuntu was not able to boot due to some error. I tried recovering the node but unfortunately couldn’t do it. That is when I started searching for Kubernetes backup option. I found this great tool and some useful links to back up your Kubenetes master into Google Storage Buckets or AWS S3

Project: https://velero.io/docs/v1.0.0/install-overview/

Useful Link: https://heptio.github.io/velero/master/install-overview.html

You can also, set it up to back up your master consistently.