This is post is aimed at newcomers to Kubernetes who want to work through a practical example of how to write a Golang API for managing your todo list and then how to deploy it to Kubernetes.

Every developer loves a good TODO list, right? How would we get anything done otherwise.

Every developer loves a good TODO app, right?

We’ll start by covering the bill of materials, then go on to configure Kubernetes, provision a Postgresql database and then install an application framework that can help us deploy Go APIs easily to Kubernetes without getting into the weeds.

We’ll create two endpoints in the API — one to create a new TODO item and one to select all TODO items.

The full code example for this tutorial is be available on GitHub: https://github.com/alexellis/kubernetes-todo-go-app

Bill of materials:

Docker installed locally

Kubernetes runs code in container images, so you’ll need to install Docker on your computer.

Install Docker now: https://www.docker.com

Register for a Docker Hub account to store your Docker images: https://hub.docker.com/

Kubernetes cluster

You can pick a local cluster, or a remote cluster, but which is best? Lightweight options like k3d work on any computer that can run Docker, so running a local cluster, no longer requires copious amounts of RAM. A remote cluster can also be a very efficient way to work, but bear in mind that all your Docker images need to be uploaded and downloaded for each change.

Install k3d now: https://github.com/rancher/k3d

k3d won’t install kubectl (which is the CLI for Kubernetes), so install that separately from here: https://kubernetes.io/docs/tasks/tools/install-kubectl

Golang

You’ll need to install Golang on your computer along with an IDE. Go is free and you can download it for MacOS, Windows or Linux here:

Install Go now: https://golang.org/dl

An IDE

I would recommend using Visual Studio Code, it has a set of plugins you can add for Go and is free. Some folks prefer Goland from Jetbrains, if you’re a Java programmer, you’ll probably prefer to pay for Goland as it’ll remind you of their other products.

Install VSCode or Goland now: https://code.visualstudio.com or https://www.jetbrains.com/go

Build the cluster

You’ll need to have installed all of the software in the previous section.

Create a new cluster with k3d:

k3d create

Configure kubectl so that is points at the new cluster, bear in mind that multiple cluster can be used from the same machine, so pointing at the right one is crucial:

export KUBECONFIG="$(k3d get-kubeconfig --name='k3s-default')"

Check that the cluster as at least one node, you can see here that I have Kubernetes 1.17, which is a relatively recent version:

kubectl get node NAME STATUS ROLES AGE VERSION

k3d-k3s-default-server Ready master 48s v1.17.0+k3s.1

We’ll be storing our TODO items in a database table, so we now need to install a database. Postgresql is a popular relational database, which we can install into the cluster using its helm chart.

Install arkade

arkade is a Go CLI that is similar to “brew” or “apt-get”, but for Kubernetes applications. It uses Helm, kubectl, or a project’s CLI to install a project or product to your cluster.

curl -sLS https://dl.get-arkade.dev | sudo sh

Now install Postgresql

arkade install postgresql ===================================================================== = PostgreSQL has been installed. =

=====================================================================

You’ll also see connection string information printed out and how to run the Postgresql CLI via a Docker image inside the cluster.

You can get this information at any time with arkade info postgresql .

Design a table schema

CREATE TABLE todo (

id INT GENERATED ALWAYS AS IDENTITY,

description text NOT NULL,

created_date timestamp NOT NULL,

completed_date timestamp NOT NULL

);

Run arkade info postgresql to get the connection info again, it should be similar to:

export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode) kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.6.0-debian-9-r0 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host postgresql -U postgres -d postgres -p 5432

Now you have a prompt as per: postgres=# , you can create the table by pasting it in, then run \dt to show the table:

postgres=# \dt

List of relations

Schema | Name | Type | Owner

--------+------+-------+----------

public | todo | table | postgres

(1 row)

Install the application framework

Just like PHP developers accelerated their workflow by using LAMP (Linux Apache + Mysql + PHP) and Rails developers accelerated their workflow with a pre-built stack, Kubernetes developers can also make use of application frameworks.

The PLONK Stack stands for Prometheus, Linux, OpenFaaS, NATS and Kubernetes.

Prometheus provides metrics, auto-scaling and observability to check on the health of your system and to allow it to respond to spikes in demand and to save costs by providing metrics for decisions about scaling to zero.

Linux, whilst not the only option for running workloads on Kubernetes is the default and the easiest to use.

OpenFaaS originally started to provide portable functions to developers, but works great when deploying APIs and microservices too. Its versatility means that any Docker containers with a HTTP server can be deployed and managed.

NATS is a popular CNCF project used for messaging and for pub/sub. In the PLONK stack it provides the ability to run requests asynchronously and for queuing.

Kubernetes is the reason we’re here. It provides scale-out, self-healing, and declarative infrastructure. And if you don’t need most of that, its API is made simple via OpenFaaS.

Install the PLONK stack via arkade:

arkade install openfaas

Read the info message and run each command to:

Install the faas-cli

Get your password

Port forward the OpenFaaS gateway UI (via port 8080)

And to login via the CLI

Like before, you can get the info message via arkade info openfaas .

Deploy your first Go API

There are multiple ways to create a Go API with PLONK. The first option is to use a Dockerfile and to manually define a TCP port, a health-check, a HTTP server and so on. This can be done via faas-cli new --lang dockerfile API_NAME , however there is an easier, less manual way.

The second way is to take advantage of the pre-built templates offered by the Function Store:

faas-cli template store list | grep go go openfaas Classic Golang template

golang-http openfaas-incubator Golang HTTP template

golang-middleware openfaas-incubator Golang Middleware template

Since we want to create a traditional HTTP-style API, the golang-middleware template will be best suited.

In the beginning of the tutorial you registered for a Docker Hub account to store your Docker images. Each workload in Kubernetes needs to be built into a Docker image before it can be deployed.

Pull in the special template:

faas-cli template store pull golang-middleware

Scaffold an API with golang-middleware and your Docker Hub username:

export PREFIX=alexellis2

export LANG=golang-middleware

export API_NAME=todo faas-cli new --lang $LANG --prefix $PREFIX $API_NAME

You’ll see two files generated:

./todo.yml — provides a way to configure the deployment and to set its template and name

— provides a way to configure the deployment and to set its template and name ./todo/handler.go — this is where you write your code and add any other files or packages you require

Let’s make a quick edit and then deploy the code.

package function import (

"net/http"

"encoding/json"

) type Todo struct {

Description string `json:"description"`

} func Handle(w http.ResponseWriter, r *http.Request) {

todos := []Todo{}

todos = append(todos, Todo{Description: "Run faas-cli up"}) res, _ := json.Marshal(todos) w.WriteHeader(http.StatusOK)

w.Header().Set("Content-Type", "application/json") w.Write([]byte(res))

}

If you’re not using VScode and its plugins to edit and format the code, then run this after each change to ensure the file is formatted correctly.

gofmt -w -s ./todo/handler.go

Now deploy the code by building a new image, pushing it to the Docker Hub and deploying it to the cluster via the OpenFaaS API:

Invoke your endpoint when ready: curl http://127.0.0.1:8080/function/todo {

"description": "Run faas-cli up"

}

Allow users to create new todo items

Now allow users to create new items in their todo list. First you’ll need to add a reference or “dependency” to the Postgresql library for Go.

We can achieve this through vendoring or via Go modules which were introduced in Go 1.11 and set as a default in Go 1.13.

Edit handler.go and add the module we require to access Postgresql:

import (

"database/sql"

_ "github.com/lib/pq"

...

In order for Go modules to detect the dependency, we have to declare something inside the file, which we will use later on, if we don’t, then VSCode will remove these lines upon save.

Add this in the file under the imports

var db *sql.DB

Always run these commands inside the todo folder where handler.go exists, not at the root level with todo.yml .

Initialise a new Go module:

cd todo/

ls

handler.go export GO111MODULE=on

go mod init

Now update the file with the pq library:

go get

go mod tidy cat go.mod module github.com/alexellis/todo1/todo go 1.13 require github.com/lib/pq v1.3.0

Whatever contents you see inside go.mod , copy them to GO_REPLACE.txt

cat go.mod > GO_REPLACE.txt

Now let’s check that the build still works before adding additional code for the insert.

faas-cli build -f todo.yml --build-arg GO111MODULE=on

You’ll note that we now pass a --build-arg to tell the template to use Go modules.

During the build you’ll see that the modules are downloaded on demand from the Internet.

Step 16/29 : RUN go test ./... -cover

---> Running in 9a4017438500

go: downloading github.com/lib/pq v1.3.0

go: extracting github.com/lib/pq v1.3.0

go: finding github.com/lib/pq v1.3.0

? github.com/alexellis/todo1/todo [no test files]

Removing intermediate container 9a4017438500

Configure secrets to access Postgresql

We can establish a connection pool within the init() method that will run only once when the program starts.

// init establishes a persistent connection to the remote database

// the function will panic if it cannot establish a link and the

// container will restart / go into a crash/back-off loop

func init() { if _, err := os.Stat("/var/openfaas/secrets/password"); err == nil {

password, _ := sdk.ReadSecret("password")

user, _ := sdk.ReadSecret("username")

host, _ := sdk.ReadSecret("host")

dbName := os.Getenv("postgres_db")

port := os.Getenv("postgres_port")

sslmode := os.Getenv("postgres_sslmode") connStr := "postgres://" + user + ":" + password + "@" + host + ":" + port + "/" + dbName + "?sslmode=" + sslmode var err error

db, err = sql.Open("postgres", connStr) if err != nil {

panic(err.Error())

} err = db.Ping()

if err != nil {

panic(err.Error())

}

}

}

You’ll notice that some information is fetched from os.Getenv reading from environment. These values are what I would consider non-confidential and the values are set in the todo.yml file.

For the others, such as password and host, these are confidential and are stored in Kubernetes secrets.

You can create these via faas-cli secret create or via kubectl create secret generic -n openfaas-fn .

The line sdk.ReadSecret comes from the OpenFaaS Cloud SDK with the following import: github.com/openfaas/openfaas-cloud/sdk . It reads the secret file from disk and returns the value, or an error.

Get the secret values from arkade info postgresql .

Now for each password run the following:

export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode) export USERNAME="postgres"

export PASSWORD=$POSTGRES_PASSWORD

export HOST="postgresql.default" faas-cli secret create username --from-literal $USERNAME

faas-cli secret create password --from-literal $PASSWORD

faas-cli secret create host --from-literal $HOST

Check the secrets exist as expected:

faas-cli secret ls

NAME

username

password

host # And via kubectl: kubectl get secret -n openfaas-fn

NAME TYPE DATA AGE

username Opaque 1 13s

password Opaque 1 13s

host Opaque 1 12s

Edit your YAML file and add the following:

secrets:

- host

- password

- username

environment:

postgres_db: postgres

postgres_sslmode: "disable"

postgres_port: 5432

Next, update the Go modules and run the build again:

cd todo

go get

go mod tidy cd .. faas-cli build -f todo.yml --build-arg GO111MODULE=on Successfully built d2c609f8f559

Successfully tagged alexellis2/todo:latest

Image: alexellis2/todo:latest built.

[0] < Building todo done in 22.50s.

[0] Worker done. Total build time: 22.50s

The build worked as expected, so run faas-cli up with the same arguments to push and deploy the image. If the credentials and SQL configuration is correct we won’t see an error in the logs, however if they are wrong, we’ll hit the panic code in init().

Check the logs:

faas-cli logs todo 2020-03-26T14:10:03Z Forking - ./handler []

2020-03-26T14:10:03Z 2020/03/26 14:10:03 Started logging stderr from function.

2020-03-26T14:10:03Z 2020/03/26 14:10:03 Started logging stdout from function.

2020-03-26T14:10:03Z 2020/03/26 14:10:03 OperationalMode: http

2020-03-26T14:10:03Z 2020/03/26 14:10:03 Timeouts: read: 10s, write: 10s hard: 10s.

2020-03-26T14:10:03Z 2020/03/26 14:10:03 Listening on port: 8080

2020-03-26T14:10:03Z 2020/03/26 14:10:03 Metrics listening on port: 8081

2020-03-26T14:10:03Z 2020/03/26 14:10:03 Writing lock-file to: /tmp/.lock

Things look good so far, now try to invoke the endpoint:

echo | faas-cli invoke todo -f todo.yml 2020-03-26T14:11:02Z 2020/03/26 14:11:02 POST / - 200 OK - ContentLength: 35

So we now have a successful DB connection established and can perform an insert. How do we know that? Because db.Ping() returns an error, and would have thrown a panic otherwise:

err = db.Ping()

if err != nil {

panic(err.Error())

}

See the full reference for the database/sql package.

Write the insert code

This code inserts a new tow into the todo table, and uses a special syntax where the value is not quoted, but instead is replaced by the db.Query code. In the “old days” of LAMP programming, a common error that ended up making lots of systems insecure was not sanitizing inputs and concatenating user input directly into a SQL statement.

Imagine someone having a description input of ; drop table todo , it would not be fun.

So we run db.Query , then pass in the SQL statement, using $1, $2 and etc for each value and then can retrieve a result and / or an error. We should also close this result, so use defer for that.

func insert(description string) error {

res, err := db.Query(`insert into todo (id, description, created_date) values (DEFAULT, $1, now());`,

description) if err != nil {

return err

} defer res.Close()

return nil

}

Now let’s hook this up into the code.

func Handle(w http.ResponseWriter, r *http.Request) {

if r.Method == http.MethodPost && r.URL.Path == "/create" {

defer r.Body.Close()

body, _ := ioutil.ReadAll(r.Body) if err := insert(string(body)); err != nil {

http.Error(w, fmt.Sprintf("unable to insert todo: %s", err.Error()), http.StatusInternalServerError)

}

}

}

Let’s deploy that and try it out?

Check the API’s logs:

faas-cli logs todo 2020-03-26T14:35:29Z 2020/03/26 14:35:29 POST /create - 200 OK - ContentLength: 0

Check the table contents with pgsql:

export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode) kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.6.0-debian-9-r0 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host postgresql -U postgres -d postgres -p 5432 postgres=# select * from todo;

id | description | created_date | completed_date

----+-----------------+----------------------------+----------------

1 | faas-cli build | 2020-03-26 14:36:03.367789 |

2 | faas-cli push | 2020-03-26 14:36:03.389656 |

3 | faas-cli deploy | 2020-03-26 14:36:03.797881 |

Congratulations, you now have a todo API which can accept incoming requests via curl or any other HTTP client and record them in a database table.

Query the todo items

Let’s create a new function to query the todo items from the table:

func selectTodos() ([]Todo, error) {

var error err

var todos []Todo return todos, err

}

We can’t call this method select because that’s a reserved keyword for dealing with go routines.

Now hook up the method into the main handler:

} else if r.Method == http.MethodGet && r.URL.Path == "/list" {

todos, err := selectTodos() if err != nil { http.Error(w, fmt.Sprintf("unable to get todos: %s", err.Error()), http.StatusInternalServerError) } out, _ := json.Marshal(todos)

w.Header().Set("Content-Type", "application/json")

w.Write(out)

}

Now that we have additional fields in our data schema for dates, update the Todo struct:

type Todo struct {

ID int `json:"id"`

Description string `json:"description"`

CreatedDate *time.Time `json:"created_date"`

CompletedDate *time.Time `json:"completed_date"`

}

Now let’s update our selectTodos() method with the query code:

func selectTodos() ([]Todo, error) {

rows, getErr := db.Query(`select id, description, created_date, completed_date from todo;`) if getErr != nil {

return []Todo{}, errors.Wrap(getErr, "unable to get from todo table")

} todos := []Todo{}

defer rows.Close()

for rows.Next() {

result := Todo{}

scanErr := rows.Scan(&result.ID, &result.Description, &result.CreatedDate, &result.CompletedDate)

if scanErr != nil {

log.Println("scan err:", scanErr)

}

todos = append(todos, result)

}

return todos, nil

}

Just like before, we need to defer the closure of the rows for the query. Each value is inserted into a new struct via the rows.Scan method. At the end of the method we have a slice of Todo items.

Try it out:

faas-cli up -f todo.yml --build-arg GO111MODULE=on curl http://127.0.0.1:8080/function/todo/list

Here’s the result:

[

{

"id": 2,

"description": "faas-cli build",

"created_date": "2020-03-26T14:36:03.367789Z",

"completed_date": null

},

{

"id": 3,

"description": "faas-cli push",

"created_date": "2020-03-26T14:36:03.389656Z",

"completed_date": null

},

{

"id": 4,

"description": "faas-cli deploy",

"created_date": "2020-03-26T14:36:03.797881Z",

"completed_date": null

}

]

To remove the nulls we can update the struct’s annotations to add omitempty :

CompletedDate *time.Time `json:"completed_date,omitempty"`

Wrapping up this session

We’re not quite done, but this is a good point to pause and review what we’ve achieved so far.

Installed Go, Docker, kubectl and VSCode (an IDE)

Deployed Kubernetes on our local machine

Installed Postgresql using arkade and helm3

Installed OpenFaaS and the PLONK stack for Kubernetes application developers

Built an initial static REST API using Go and the OpenFaaS golang-middleware template

template Added “insert” functionality to our TODO API

Added “select” functionality to our TODO API

The full code example for what we’ve built so far is available on my GitHub account:

There’s much more we could do here, such as:

Adding authentication with a static Bearer token

Creating a webpage using a static HTML template, or React to render the todo list and to create new items

Adding multi-user support to the API

And much more. We could also dig into the PLONK stack and deploy a Grafana dashboard to start observing our API and understanding how much resources are being used with the Kubernetes dashboard or metrics-server (installed via arkade install )

Find out more about arkade: https://get-arkade.dev

Try the OpenFaaS workshop to explore the above in depth: https://github.com/openfaas/workshop/

Clap and follow me on Medium.com and Tweet to me at:

You can subscribe to my premium newsletter “Insiders Updates” at