Adding a worker node to k8s cluster is not a fancy topic anymore as there are millions of tools exist anymore to manage/maintain the existing k8s clusters. That’s why I will try to explain how we can add a worker node which is not located at the same network with masters where the only open port between these isolated networks is just the k8s api port (secure-port Default: 6443).

If you ask why we may need this kind of setup, one of the answer is that you might need to manage your distributed worker nodes from central cluster. Furthermore ,the master can run on public cloud while the worker nodes run on physical servers at your private datacenter. Perhaps, you will be using managed k8s service of one of the public cloud vendors and integrate your physical k8s workers to this managed clusters. As we all know ,all vendors are in a race condition to enhance their market rate at K8s world. Google GKE, Azure AKS, AWS EKS are the leader of the market recently and they may come with a great offer which can change your mind.

After a long intro, let s deep dive the steps those I have used to add my external node to my k8s cluster running on AWS EC2 that initialized via kubeadm. Please note that this cluster is not AWS EKS and kubeadm is used to initialize the cluster after commisioned the EC2 instances via terraform.

Section 1:Initialize the Kube cluster via Kubeadm.

Here is the basic steps of kubeadm for an ubuntu instance.

1.Please note that as the k8s improvement is going crazy, the versions will be different when you take a look at this story.



2. curl -s

3. cat <<EOF >/etc/apt/sources.list.d/kubernetes.list

deb

EOF

4.apt-get update

5.apt-get install -y kubelet kubeadm kubectl docker.io 1. apt-get update && apt-get install -y apt-transport-https2. curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -3. cat < /etc/apt/sources.list.d/kubernetes.listdeb http://apt.kubernetes.io/ kubernetes-xenial mainEOF4.apt-get update5.apt-get install -y kubelet kubeadm kubectl docker.io

2.In addition to basic k8s utilities,you need to install a cni network to let the master become ready with network perspective of view. In this sample,I ll be using weave network.Please issue the following instruction at master node .

kubectl apply -f " https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '

')"

3.When you run kubectl commands to see the k8s object ,you may get connection refused error, to skip this error please run below commands .

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

4.Prior initialize kubeadm ,create the self signed certs.

openssl req -newkey rsa:2048 -nodes -keyout ca.key -x509 -days 365 -out ca.crt

5.Copy these self signed certs to the corresponding directory. (This step should be re-run in case of any kubeadm reset)

sudo cp ca.crt ca.key /etc/kubernetes/pki/

6.Get the base64 of the crt. Base64 format of the crt will be use at worker node join process.

Copy/paste the crt to the https://www.base64encode.org/ to get the base64 formated crt.

7.Modify the kubelet via your favourite editor. A sample version can be seen below, defining cidr block is important as well at this step.

root@ip-172-31-35-109:/etc/default# cat /etc/default/kubelet

KUBELET_EXTRA_ARGS=--cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=kubenet --non-masquerade-cidr=192.168.0.0/16 --pod-cidr=10.244.0.0/16

8.Init the cluster via below parameters .(CIDR network and token must be identified. Advertise IP is important as the crt will be using this IP while serving.) In addition ,please remind the kubeadm token has also be defined (You can manually define a token and use it directly).

Kube api IP also declared as advertise address (35.180.109.126)

sudo kubeadm init --apiserver-advertise-address=35.180.109.126 --pod-network-cidr=10.244.0.0/16 --token=jb35bt.zhvs4asaanx0fw50 --token-ttl=0

9.Run below commands and ensure master is ready to go.

Section 2: Add the external node to K8S Cluster.

In order to commission your private machine as a k8s worker node,the first step is to install basic packages as we did at master node.

Please note that all components version at both master and worker nodes has to be same for a proper configuration.



2. curl -s

3. cat <<EOF >/etc/apt/sources.list.d/kubernetes.list

deb

EOF

4.apt-get update

5.apt-get install -y kubelet kubeadm docker.io 1. apt-get update && apt-get install -y apt-transport-https2. curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -3. cat < /etc/apt/sources.list.d/kubernetes.listdeb http://apt.kubernetes.io/ kubernetes-xenial mainEOF4.apt-get update5.apt-get install -y kubelet kubeadm docker.io

2. Create a config.yaml file at worker node. At this config file, you will configure kubernetes token and refer the discovery.yaml where we configure base64 formatted cert and api IP.

Please see the sample below.

apiVersion: kubeadm.k8s.io/v1alpha2

clusterName: kubernetes

discoveryFile: discovery.yaml

kind: NodeConfiguration

nodeRegistration:

criSocket: /var/run/dockershim.sock

name: ""

token: jb35bt.zhvs4asaanx0fw50

3. Create a discovery.yaml at worker node.Base64 format crt and the api IP of the cluster has to be configured.

4.Then run below command to join to the cluster .This instruction will automatically joint your node to k8s cluster if the API IP:port (35.180.109.126:6443 for our scenerio) is reachable from node .

sudo kubeadm join --config=config.yaml

5.Ensure your worker node joint to the cluster by running below command at master node and give a try to run a nginx pod by following commands.

kubectl get nodes

kubectl create deployment --image nginx my-nginx

Hope you see the similar screen as below.

Wrapping up, for sure self signed certs should not be in scope of production system,thought insecure configurations are always strong option for development environment.Hope this story helps for your need.

Thanks