Image from epicurious.com

This paper serves the purpose of reference for the well-known sidecar pattern of Kubernetes.

Goal

Implement the HTTPS interface for a container application that doesn’t have https implemented.

Solution

Run a Nginx container beside the app in the same pod. The Nginx web server listens on the HTTPS port, and reverse proxy the request to the actual app in the same pod.

Sample App

A toy app to illustrate the idea. The golang app is listed as below. Notice that no https is implemented.

package main import (

"fmt"

"log"

"net/http"

"os"

"time"

) func hello(w http.ResponseWriter, r *http.Request) {

fmt.Fprintf(w, "Hello!")

} func date(w http.ResponseWriter, r *http.Request) {

fmt.Fprintf(w, "time now: %s", time.Now().Format("15:04:05"))

} func main() {

http.HandleFunc("/", hello)

http.HandleFunc("/date", date) port := os.Getenv("LISTENING_PORT") if port == "" {

port = "8080"

}

log.Printf("listening on port:%s", port) err := http.ListenAndServe("localhost:"+port, nil)

if err != nil {

log.Fatalf("Failed to start server:%v", err)

}

}

Build the app,

export CGO_ENABLED=0

go build -o myhandler src/*.go

Prepare the Dockerfile,

FROM alpine

RUN addgroup -S appgroup && adduser -S appuser -G appgroup && mkdir -p /app

ADD myhandler /app

RUN chmod a+rx /app/myhandler



USER appuser

WORKDIR /app

ENV LISTENING_PORT 8080 CMD ["./myhandler"]

Build the image and push it to Docker hub.

sudo docker build -t zhiminwen/hello:v1 .

Followings are the steps to implement the HTTPS interface for this app.

Step 1. Create the certificate

Use cfssl to create the required HTTPS certificate. Download the cfssl binary for your platform.

Create CA

Prepare the following ca config file, say “certs/myca.json”

Run command

cd certs

cfssl.exe gencert -initca myca.json | cfssljson -bare myca

This creates the following files of

myca.pem: the CA cert, public key

myca-key.pem: the CA cert, private key

2. Create SSL cert for Nginx

Create the following “ca-config.json” file with the profiles defined as

{

"signing": {

"default": {

"expiry": "43800h"

},

"profiles": {

"server": {

"expiry": "43800h",

"usages": [

"signing",

"key encipherment",

"server auth",

"client auth"

]

},

"client": {

"expiry": "43800h",

"usages": [

"signing",

"key encipherment",

"client auth"

]

}

}

}

}

Define the cert requirement in json format as below, save as “myrequest.json”.

{

"CN": "hello-server",

"hosts": [

""

],

"key": {

"algo": "rsa",

"size": 2048

}

}

Generate the signed certs with the command below,

cd certs cfssl gencert -ca=myca.pem -ca-key=myca-key.pem -config=ca-config.json -profile=server -hostname="127.0.0.1" myrequest.json | cfssljson -bare hello-server

The following certificates are then created

Certificate: hello-server.pem

Private key: hello-server-key.pem

We will then use these certificates to configure Nginx SSL.

Step 2. Create nginx.conf

The configuration for nginx.conf is minimum. We define a SSL server with TLS v1.2, and redirect the traffic to the actual application.

worker_processes 1; events {

worker_connections 1024;

} http {

include mime.types;

default_type application/octet-stream; sendfile on;

keepalive_timeout 65; server {

listen 443 ssl;

server_name localhost; ssl_certificate /app/cert/hello-server.pem;

ssl_certificate_key /app/cert/hello-server-key.pem; ssl_protocols TLSv1.2;

ssl_ciphers EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:!EECDH+3DES:!RSA+3DES:!MD5;

ssl_prefer_server_ciphers on; location / {

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header Host $http_host;

}

}

} proxy_pass http://127.0.0.1:8080/

The ssl_certifcate and ssl_certificate_key are the public key and private key of the https server.

The location block redirects the traffic to the application container which is located at the same pod with Nginx.

Step 3. Create configMap and secret

Create the K8s configMap for the nginx.conf and secret for the certs.

kubectl create cm hello-sidecar-nginx-conf --from-file=nginx.conf=./nginx.conf kubectl create secret generic hello-sidecar-nginx-certs --from-file=hello-server-cert=./hello-server.pem --from-file=hello-server-key=./hello-server-key.pem

We are using the “ — from-file=key=filename” format, so the configMap and secret have the key fields specified as what we have defined.

Step 4. Create K8s deployment

The full deployment file is listed as below,

---

apiVersion: v1

kind: Service

metadata:

name: hello

labels:

app: hello

spec:

type: NodePort

ports:

- port: 443

targetPort: 443

protocol: TCP

name: https

selector:

app: hello

---

apiVersion: apps/v1

kind: Deployment

metadata:

name: hello

labels:

app: hello

spec:

replicas: 1

selector:

matchLabels:

app: hello

template:

metadata:

labels:

app: hello

spec:

containers:

- name: hello

image: zhiminwen/hello:v1

imagePullPolicy: IfNotPresent

env:

- name: LISTENING_PORT

value: "8080"

- name: tls-sidecar

image: nginx

imagePullPolicy: IfNotPresent

volumeMounts:

- name: secret-volume

mountPath: /app/cert

- name: config-volume

mountPath: /etc/nginx/nginx.conf

subPath: nginx.conf

volumes:

- name: secret-volume

secret:

secretName: hello-sidecar-nginx-certs

items:

- key: hello-server-cert

path: hello-server.pem

- key: hello-server-key

path: hello-server-key.pem

- name: config-volume

configMap:

name: hello-sidecar-nginx-conf

The main app is nothing special. It behaves as its normal.

Define the configMap volume and the secret volume.

For the key of “hello-server-cert”, we specify the path with the file name, so the pod will mount using the file name of “hello-server.pem”. The same goes for the other key.

Name the nginx container as “tls-sidecar”. Mount the config-volume with both mountPath and subPath so that only the file “nignx.conf” will be presented in the target directory, and avoid overwriting the whole default directory of Nginx.

Mount the secret to the directory of “/app/cert” to match the exact file name defined in the nginx.conf.

Lastly, create a service that exposes the Nginx https server port 443 as nodePort.

Apply the yaml file to deploy it.

Testing

Once the pods are running, find out the NodePort. Do a HTTPS connection to the nodePort, the https is working.

Check the certificate,

See the second paper of the sidecar series, where I explored the client certificate authentication and the Prometheus in IBM Cloud Private.