In this article, we will be setting up a self-built kubernetes on Alibaba Cloud Elastic Compute Service instances using Linux flavors (Centos7 and Ubuntu 16.

By Anish Nath, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

Alibaba Cloud Container Service for Kubernetes is a fully-managed service compatible with Kubernetes to help users focus on their applications rather than managing container infrastructure. There are two ways to deploy Kubernetes on Alibaba Cloud, one through Container Service (built-in) and the other through an Elastic Compute Service (ECS) instance (self-built). If you are not sure which installation method suits your needs better, then refer to the documentation Alibaba Cloud Kubernetes vs. self-built Kubernetes.

For the most part, choosing Alibaba Cloud Container Service is the preferred choice as it saves time and reduces the complexity to maintain Kubernetes clusters. However, there may be cases where a manual installation is better suited to your needs. In this article, we will be setting up a self-built kubernetes on Alibaba Cloud Elastic Compute Service instances using Linux flavors (Centos7 and Ubuntu 16.04).

Prerequisites

A valid Alibaba Cloud account. If you don't have one already, sign up to the Free Trial to enjoy $300 worth in Alibaba Cloud products. An ECS instance running Ubuntu 16.04 or RHEL7 or Centos7. You can select your preferred region and configurations; this will not affect the outcome of the server setup. A sudo password for your server.

Kubernetes Security Group Setup

These required ports needs to be opened in the pod security group, to do this:



Log on to the ECS console. Select a region. In the left-side navigation pane, select Networks & Security and select Security Group. Click Create Security Group. Set the Inbound rules for your pod network as shown below. For my example, 192.168.0.0/16 is the pod network IP address; you'll need to replace it with your own.

Authorization Object Protocol Type Port Range Authorization Policy 192.168.0.0/16 TCP 6443/6443 Allow 192.168.0.0/16 TCP 2379/2379 Allow 192.168.0.0/16 TCP 2380/2380 Allow 192.168.0.0/16 TCP 10250/10250 Allow 192.168.0.0/16 TCP 10251/10251 Allow 192.168.0.0/16 TCP 10252/10252 Allow 0.0.0.0/0 TCP 22/22 Allow

Note:



Alibaba Cloud Container Service and the pod CIDR block cannot overlap with the VPC CIDR block. The service CIDR block cannot overlap with the VPC CIDR block or pod CIDR block. The security group rules is enable communication between kubernetes master and cluster.

Connect to Your Alibaba Cloud Server

Locate the Internet IP address (Public IP address) associated with your Alibaba Cloud ECS Instance. There are other ways to connect to your ECS instance as well. Visit the official ECS documentation to learn more.

Before proceeding, stop and check whether:



ECS public IP is enabled. ECS is up and running. ECS security group is pointed to pods-security group. ECS security group Authorization Object matches with ECS instance.

Setup Environment

This example utilizes two ECS instances running in the Alibaba Could environment and with these hostnames

Master Server name Minion Cluster kube-master kube-minion-1

Install Kubernetes-master and Kubernetes-minion

These are the minimum requirements to setup kubernetes ECS instance master and minion.

Requirement Kubernetes-master Kubernetes-minion Disable system swap and SELinux Y Y remove any swap entry from /etc/fstab Y Y net.bridge.bridge-nf-call-iptables is set to 1 Y Y Install Docker & enable on restart Y Y Install kubeadm Y Y Install kubelet Y N Install kubectl Y N Configure docker cgroupsfs Y N Creating Network Addons (flannel/Calico) Y N

Disable system swap with swapoff -a . This will immediately disable swap and remove any swap entry from /etc/fstab

Disable SE Linux by setenforce 0

You should ensure net.bridge.bridge-nf-call-iptables is set to 1 in your sysctlconfig

cat <<EOF > /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system

If you are running CentOS or RHEL7, install Docker and enable on restart with these commands.

yum -y update yum install -y docker

If you are running Ubuntu, install Docker and enable on restart with these commands.

apt-get update apt-get install -y docker.io

Start and enable Docker and check if docker service is running.

systemctl start docker systemctl enable docker systemctl status docker

Install Kubernetes

You will need to install these packages on all of your machines:



kubeadm: the command to bootstrap the cluster. kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers. kubectl: the command line util to talk to your cluster.

Kubernetes Installation on CentOS7/RHEL7

Setting up Kubernetes yum repository

cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF

Install Kubernetes in CentOS7/RHEL7

yum install -y kubelet kubeadm kubectl

Kubernetes Installation on Ubuntu

Setting up Kubernetes apt repository for Ubuntu

apt-get install -y apt-transport-https curl apt-get install -y docker.io curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat <<EOF >/etc/apt/sources.list.d/kubernetes.list deb http://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get update

Install Kubernetes in Ubuntu

apt-get install -y kubelet kubeadm kubectl

Running Kubernetes

Enable & start kublet

systemctl enable kubelet systemctl start kubelet

Verify that your Docker cgroup driver matches the kubelet config: (kube-master node)

docker info | grep -i cgroup Cgroup Driver: cgroupfs

Configuration for cgroup drive is right in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

[Service] Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs"

Reload daemon and restart kubelet: (kube-master node)

systemctl daemon-reload systemctl restart kubelet

Flush Reset Kubernetes component (This will wipeout all the cluster config if exist any do not run this after the config create)

kubeadm reset -f

Set up Kubernetes Network add on



For flannel to work correctly, --pod-network-cidr=10.244.0.0/16 has to be passed to kubeadm init. For Calico to work correctly, --pod-network-cidr=192.168.0.0/16 has to be passed to kubeadm init.

Creating flannel Networks

Note apiserver-advertise-address is the IP of the kube-master

kubeadm init --service-cidr 10.96.0.0/12 --kubernetes-version v1.11.0 --pod-network-cidr 10.244.0.0/16 --apiserver-advertise-address 192.168.1.130

You should get information back on initiating commands as a normal user, as well as the network that you need to deploy as well as how to join worker nodes to the cluster.

To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You can now join any number of machines by running the following on each node as root:

Join the Cluster (kube-minnion)

kubeadm join 192.168.1.130:6443 --token 5m8qxr.46rpadiwt8fcka0v --discovery-token-ca-cert-hash sha256:b05a0b8849a57432247c06200864f5ce99d40ffdcae965293c0026204ef33da4

Run kubectl get nodes on the master to see this node join the cluster. It will few seconds to get your cluster in ready state

root@kube-master:kubectl get nodes NAME STATUS ROLES AGE VERSION kube-master Ready master 2m v1.11.0 kube-minion Ready <none> 47s v1.11.0

Apply flannel Addons

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml

You will get output like this

clusterrole.rbac.authorization.k8s.io/flannel created clusterrolebinding.rbac.authorization.k8s.io/flannel created serviceaccount/flannel created configmap/kube-flannel-cfg created daemonset.extensions/kube-flannel-ds created

Enable and restart kubelet engine

systemctl restart kubelet systemctl status kubelet

Verifying the Installation

Verify the Cluster Information

root@kube-master: kubectl cluster-info Kubernetes master is running at https://192.168.1.132:6443 KubeDNS is running at https://192.168.1.132:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

Verify the Services

root@kube-master:kubectl get services -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 2m

Create Tokens

[root@kube-master ~]kubeadm token create I0710 04:08:37.149017 8685 feature_gate.go:230] feature gates: &{map[]} d49l0d.mheeem1dkrw3n43

That's it! You have successfully configured a self-built Kubernetes cluster on Alibaba Cloud. To learn more about Kubernetes on Alibaba Cloud, visit Alibaba Cloud Container Service for Kubernetes.