TL;DR: Inspired by Kelsey Hightower’s kubernetes-the-hard-way, I am here putting the steps I did to setup K8s in my laptop with VirtualBox VMs, instead of Google Cloud resources(saving some bucks here).

Jump to part 2

Setup overview we will accomplish by the end of the post:

The IP addresses and Network requirements:

We will be dealing basically with 4 networks:

10.240.0.0/24 – VirtualBox Host Network, which will be used for interconnecting between the VMs. You may watch https://www.youtube.com/watch?v=m0ZWtcJT7fc for more info on how to set the host-only network in VirtualBox. 10.200.0.0/16 – POD CIDR that will be used to assign POD IP addresses 10.32.0.0/24 – Cluster Network, which will be used by K8s for all its internal components like service endpoints 192.168.33.0/24 – VirtualBox NAT network, which will be used for connecting to the Internet(eg: to install a package)

Therefore, the addressing in the cluster would be:

VM IP address kube-master-01 10.240.0.100 kube-master-02 10.240.0.101 kube-worker-01 10.240.0.200 kube-worker-02 10.240.0.201 LB-server 10.240.0.50 POD CIDR subnet 10.200.1.0/16 – for kube-worker-01

10.200.2.0/16 – for kube-worker-02 Cluster CIDR & Service cluster IP range 10.32.0.0/24

The diagram now looks something like this:

In case you are new to Kubernetes, I would suggest you read this awesome post by Nicolas Leiva: Kubernetes Networking: Behind the scenes

Extra: my network interface config

This might help you to compare your network setup, here is a copy of /etc/network/interfaces file:

source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # NAT Network auto enp0s3 iface enp0s3 inet static address 192.168.33.100 gateway 192.168.33.1 dns-nameservers 8.8.8.8 # HOST-ONLY network auto enp0s8 iface enp0s8 inet static address 10.240.0.100

Step 1: Installing the Client Tools

In this step, we will be installing tools like cfssl, cfssljson, and kubectl. This can be done from any machine(mostly your workstation or laptop).

1.1 Install cfssl and cfssljson in Mac:

curl -o cfssl https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/darwin/cfssl curl -o cfssljson https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/darwin/cfssljson

chmod +x cfssl cfssljson

sudo mv cfssl cfssljson /usr/local/bin/

1.2 Install cfssl and cfssljson in Linux:

wget -q --show-progress --https-only --timestamping \ https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/linux/cfssl \ https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/linux/cfssljson

chmod +x cfssl cfssljson

sudo mv cfssl cfssljson /usr/local/bin/

1.3 Verification cfssl and cfssljson:

cfssl version cfssljson --version

cfssl version Version: 1.3.4 Revision: dev Runtime: go1.13 --- --- cfssljson --version Version: 1.3.4 Revision: dev Runtime: go1.13

1.4 Install kubectl in Mac:

curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.15.3/bin/darwin/amd64/kubectl

chmod +x kubectl; sudo mv kubectl /usr/local/bin/

1.5 Install kubectl in Linux:

wget https://storage.googleapis.com/kubernetes-release/release/v1.15.3/bin/linux/amd64/kubectl

chmod +x kubectl; sudo mv kubectl /usr/local/bin/

1.6 Verification of kubectl:

kubectl version --client

kubectl version --client Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

Step 2: Generate the CA configuration file, certificate, and private key(Certificate Authority)

Make a directory called ~/k8s-requirements/

mkdir -p ~/k8s-requirements/ cd ~/k8s-requirements/

2.1 Generate the CA configuration file, certificate, and private key(Certificate Authority).

To do copy and paste the below in your terminal/shell:

{ cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "8760h" }, "profiles": { "kubernetes": { "usages": ["signing", "key encipherment", "server auth", "client auth"], "expiry": "8760h" } } } } EOF cat > ca-csr.json <<EOF { "CN": "Kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "Kubernetes", "OU": "CA", "ST": "Oregon" } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca }

Make sure you can see the below files are created:

ca-key.pem ca.pem

2.2 Client and server certificates for each Kubernetes component and a client certificate for the Kubernetes admin user:

{ cat > admin-csr.json <<EOF { "CN": "admin", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "system:masters", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ admin-csr.json | cfssljson -bare admin }

Make sure you can see the below files are created:

admin-key.pem admin.pem

2.3 Generate a certificate and private key for each Kubernetes worker node

for instance in kube-worker-01 kube-worker-02; do cat > ${instance}-csr.json <<EOF { "CN": "system:node:${instance}", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "system:nodes", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF INTERNAL_IP_kube-worker-01=10.240.0.200 INTERNAL_IP_kube-worker-02=10.240.0.201 cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -hostname=${instance},${INTERNAL_IP}_${instance} \ -profile=kubernetes \ ${instance}-csr.json | cfssljson -bare ${instance} done

Make sure you can see the below files are created:

kube-worker-01-key.pem kube-worker-02-key.pem kube-worker-01.pem kube-worker-02.pem

2.4 Generate the kube-controller-manager client certificate and private key:

{ cat > kube-controller-manager-csr.json <<EOF { "CN": "system:kube-controller-manager", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "system:kube-controller-manager", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager }

Make sure you can see the below files are created:

kube-controller-manager.pem kube-controller-manager-key.pem

2.5 Generate the kube-proxy client certificate and private key:

{ cat > kube-proxy-csr.json <<EOF { "CN": "system:kube-proxy", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "system:node-proxier", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ kube-proxy-csr.json | cfssljson -bare kube-proxy }

Make sure you can see the below files are created:

kube-proxy-key.pem kube-proxy.pem

2.6 Generate the kube-scheduler client certificate and private key:

{ cat > kube-scheduler-csr.json <<EOF { "CN": "system:kube-scheduler", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "system:kube-scheduler", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ kube-scheduler-csr.json | cfssljson -bare kube-scheduler }

Make sure you can see the below files are created:

kube-scheduler-key.pem kube-scheduler.pem

2.7 Kubernetes API Server Certificate:

Ensure to include the IP addresses of your nodes so that they can validate the certs by the remote clients. As you may notice, I have added few extra IP addresses just incase we need in future to add more nodes as master or worker.

{ IP_PRIVATE_ADDRESSES=10.32.0.1,10.240.0.50,10.240.0.100,10.240.0.101,10.240.0.102,10.240.0.200,10.240.0.201,10.240.0.202,10.240.0.203,10.240.0.204,10.240.0.205 KUBERNETES_HOSTNAMES=kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local cat > kubernetes-csr.json <<EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "Kubernetes", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -hostname=${IP_PRIVATE_ADDRESSES},127.0.0.1,${KUBERNETES_HOSTNAMES} \ -profile=kubernetes \ kubernetes-csr.json | cfssljson -bare kubernetes }

Make sure you can see the below files are created:

kubernetes.pem kubernetes-key.pem

2.8 Generate the service-account certificate and private key for Kubernetes Controller Manager

{ cat > service-account-csr.json <<EOF { "CN": "service-accounts", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Portland", "O": "Kubernetes", "OU": "Kubernetes The Hard Way", "ST": "Oregon" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ service-account-csr.json | cfssljson -bare service-account }

Make sure you can see the below files are created:

service-account-key.pem service-account.pem

2.9 Transfer the files to respective servers:

Ensure that you change the username in the below commands and that you already create a directory named ‘~/k8s-requirements/’ in home directory of respective nodes, as mentioned in the earlier section.

#worker nodes: scp kube-worker-01.pem kube-worker-01-key.pem nandan@10.240.0.200:~/k8s-requirements/ scp kube-worker-02.pem kube-worker-02-key.pem nandan@10.240.0.201:~/k8s-requirements/ #controller/master nodes: scp service-account-key.pem service-account.pem kubernetes.pem kubernetes-key.pem ca-key.pem ca.pem nandan@10.240.0.100:~/k8s-requirements/ scp service-account-key.pem service-account.pem kubernetes.pem kubernetes-key.pem ca-key.pem ca.pem nandan@10.240.0.101:~/k8s-requirements/

I can understand that it is becoming a lengthy blog post, and getting super confusing. But hold on with me, and try to understand that we are building the K8s cluster from scratch. If you don’t understand any step, just have patience and follow along. I promise everything will be clear if you do hands-on. It took me 3 times during my initial days, to understand all the components.

We will continue to the 2nd part.