By Kan Junbao

Introduction

Alibaba Cloud provides local disk configurations for some Elastic Compute Service ECS instances. Local disks have the advantages of low latency, high random IOPS, high throughput, and high cost effectiveness, and has great advantages in some applications with high performance requirements.

In the Kubernetes system, local disks can be used through HostPath, LocalVolume, and other types of PV.

HostPath: The volume itself does not contain scheduling information. If you want to fix each pod on a node, you need to configure scheduling information, such as nodeSelector, for the pod;

LocalVolume: The volume itself contains scheduling information, and the pods using this volume will be fixed on a specific node, which can ensure data continuity.

This document describes how to use the Alibaba Cloud local disk using LocalVolume.

Use LocalVolume Provisioner to automatically create LocalVolume, and the PV comes with the information about the node to which it belongs.

Create a PVC to consume LocalVolume. Pods using this PVC will be scheduled to the node specified by the PV.

How Provisioner Works

1) Introduction to Local Disks:

To learn more about ECS local disks, refer to Local disks

Local disks cannot be mounted or unmounted;

Local disks and ECS must coexist. The number of local disks is determined when ECS is created;

The type of the machine determines the number of local disks, which can be obtained through API;

By default, local disks are mounted from /dev/vdb, which may be different from custom images.

2) Mounting Principle:

Provisioner mounts local disks to the target directory at startup:

The number of local disks in this machine can be queried through API;

Local disks are mounted from the initiating device (/dev/vdb, by default) to the target directory in turn (the initiating device can be configured);

The device detects the file system. If the device does not format the file system, a file system is created;

Provisioner polls and checks the target directory in real time, and creates a PV based on the target directory list:

When a PV is created, StorageClass can support the configuration of reclaimPolicy and volumeBindingMode;

After LocalVolume PV is deleted, it is automatically created again;

PV name: local-pv-{node name}-{device Name}. The PV name will be added to the label;

Deploy Alibaba LocalVolume Provisioner

1) Create ConfigMap:

The configuration parameters of Provisioner are placed in the ConfigMap, provisioner-config:

vendor: optional. If it is alibabacloud, the Alibaba Cloud local disk is enabled;

hostDir: the mount directory of the local disk on the host;

mountDir: the mount directory of the local disk on the container after it is mounted to the host;

volumeMode: the mount mode, supporting file systems and blocks;

fsType: the file system type, which defaults to ext4;

deviceStartWith: optional. It defaults to vdb, the starting name of the local disk;

mkFSOptions: optional. It is null by default, indicating that the parameter after “mkfs. *” is performed;

mountOptions: optional. It is null by default, indicating that the parameter of “mount -o” is performed;

apiVersion: v1

kind: ConfigMap

metadata:

name: local-provisioner-config

namespace: kube-system

data:

storageClassMap: |

local-volume:

vendor: alibabacloud

hostDir: /mnt/disks

mountDir: /mnt/disks

blockCleanerCommand:

- "/scripts/shred.sh"

- "2"

volumeMode: Filesystem

fsType: ext4

deviceStartWith: vdb

mkFSOptions: ""

mountOptions: "nodelalloc"

2) Create StorageClass

LocalVolume StorageClass does not actually go to Provisioner PV, but is used to define configuration details, such as reclaimPolicy and volumeBindingMode.

reclaimPolicy: It supports Retain and Delete;

volumeBindingMode: It supports Immediate and WaitForFirstConsumer.

kind: StorageClass

apiVersion: storage.k8s.io/v1

metadata:

name: local-volume

provisioner: kubernetes.io/no-provisioner

reclaimPolicy: Retain

volumeBindingMode: WaitForFirstConsumer

3) Create a Provisioner Service

For deployment through Daemonset, set hostPID and hostNetwork to true;

The privileged permission is added;

Deployment through AK or STS is supported;

The AK-free STS function is supported;

The node needs to configure the RAM role information, and the role has access to the cloud disk.

When using the STS function, you do not need to configure ACCESS_KEY_ID and ACCESS_KEY_SECRET parameters;

To configure STS, see Create custom authorization policies

apiVersion: extensions/v1beta1

kind: DaemonSet

metadata:

name: local-volume-provisioner

namespace: kube-system

labels:

app: local-volume-provisioner

spec:

selector:

matchLabels:

app: local-volume-provisioner

template:

metadata:

labels:

app: local-volume-provisioner

spec:

hostPID: true

hostNetwork: true

serviceAccountName: admin

containers:

- image: registry.cn-hangzhou.aliyuncs.com/plugins/local-volume-provisioner:v1.12-7802d35-aliyun

imagePullPolicy: "Always"

name: provisioner

securityContext:

privileged: true

env:

- name: MY_NODE_NAME

valueFrom:

fieldRef:

fieldPath: spec.nodeName

- name: ACCESS_KEY_ID

value: ""

- name: ACCESS_KEY_SECRET

value: ""

volumeMounts:

- mountPath: /etc/provisioner/config

name: provisioner-config

readOnly: true

- mountPath: /mnt/disks

name: local

mountPropagation: "HostToContainer"

- mountPath: /etc/kubernetes

name: etc

volumes:

- name: provisioner-config

configMap:

name: local-provisioner-config

- name: local

hostPath:

path: /mnt/disks

- name: etc

hostPath:

path: /etc/kubernetes

Consume LocalVolume

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

name: myclaim

spec:

accessModes:

- ReadWriteOnce

resources:

requests:

storage: 10Gi

storageClassName: local-volume

kind: Pod

apiVersion: v1

metadata:

name: mypod

spec:

containers:

- name: myfrontend

image: nginx

volumeMounts:

- mountPath: "/data"

name: mypd

volumes:

- name: mypd

persistentVolumeClaim:

claimName: myclaim

Original Source

https://www.alibabacloud.com/blog/use-a-local-disk-through-localvolume-provisioner-in-a-kubernetes-cluster_594930?spm=a2c41.13057465.0.0