Everything runs in the cloud, and this is a major learning point for me. This article and the previous one are about how I took my JVM service from my cushy local environment to the cloud and beyond.

Climbing the Stairway to Heaven, Copyright 1LifeOnEarth

Delegate pod lifecycle to Controllers

I started by running the service as a single Pod, which is the basic unit of execution in a Kubernetes context. However, this is not the recommended way. Instead, we should delegate the lifecycle of a Pod to a controller, such as a Deployment. Controllers manage the lifecycle of pods, which means that they can restart it upon a crash or spin up multiple instances of the same pod for horizontal scaling purposes. We only need to slightly modify our yaml file:

apiVersion: apps/v1

kind: Deployment

metadata:

name: my-service

spec:

selector:

matchLabels:

app: my-service-pod // <- which pod? (see metadata below)

replicas: 1 // <- only one instance of this pod

template:

metadata:

name: my-service

labels:

app: my-service-pod

spec:

containers:

- image: eu.gcr.io/my-project-id/my-app:latest

name: my-app-image

env:

- name: MY_ENV_VARIABLE

value: the_value_of_my_env_variable

I know — it gets kind of verbose. There are tools out there that add an additional layer of abstraction to bring templating to yaml files to reduce copy/paste and boilerplate. The most popular is Helm. My side project only has two services, so I’m fine with basic yaml files.

Checking the status of our cluster now requires looking at deployments too:

kubectl get pods,deployment

And it will show us that indeed we have a deployment running one pod:

NAME READY STATUS RESTARTS AGE

pod/my-service-pod-85f8-ctcdq 1/1 Running 0 10m NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

deployment/my-service 1 1 1 1 10m

Notice that the name of our pod bears a unique string ( 85f8-ctcdq ), which is added by the controller. We only asked for one instance (in replicas: 1 ), but there might be more and the random strings makes sure that each one would have its unique name.

If the pod dies for any reason (such as an internal failure that you didn’t manage), the controller will detect its premature disappearance and spin up a new instance. Pretty slick.

Protect your sensible data in a Kubernetes Secret

Kubernetes offers a “secret service”. Instead of writing your password or API keys in the yaml file, you can embed them into a Secret object that Kubernetes will store safely for you. You can then access your confidential data either as exposed environment variables or as files in a mounted volume.

In the yaml file above we had one environment variable:

env:

- name: MY_ENV_VARIABLE

value: the_value_of_my_env_variable

The creation of a secret object with it is done via kubectl . I can create a file named my_env_var which contains the_value_of_my_env_variable . The command

kubectl create secret generic my-env-secret --from-file=./my_env_var

creates a key-value secret named my-env-secret where the key my_env_var points to the value the_value_of_my_env_variable . We can now rewire the environment variable inclusion as:

env:

- name: MY_ENV_VARIABLE

valueFrom:

secretKeyRef:

name: my-env-secret

key: my_env_var

Just like before, our service will see an environment variable with name MY_ENV_VARIABLE and value the_value_of_my_env_variable .

Accessing secrets as files on a mounted volume has one essential case: storing the credentials to access other Google Cloud services from inside your pod. The procedure is slightly different and explained for instance here.

Logging

You can access logs from your containers via kubectl logs , but that won’t be persisted anywhere. Much better to use Stackdriver, Google’s cloud logging & monitoring resource. In fact, it is enabled by default in all newly created Kubernetes clusters.

The only one issue that I still have pending is the severity of logs. I am using Airframe Log, a lightweight logger for Scala. For some reason, all logs appear in Stackdriver as errors. This is clearly annoying and I will update this post as soon as I find a solution.

UPDATE: solved! I clearly hadn’t read the docs. I switched log formatting to the Airframe BareFormatter (simply logs the string as is), and added a severity field. A typical log now looks like

{ "msg" : "....", "severity" : "INFO" }

Stackdriver now strips the severity part out of this payload and labels accordingly its messages — which contains another wealth of information.

Continuous Delivery?

In the architectural overview for my Caterina project, I mentioned that I chose BitBucket as a cloud versioning provider as I was curious to try out their pipelining feature. I am actually hosting my repos on BitBucket, and I toyed around a little with pipelines, but ended up choosing to not have any continuous delivery setup.

The decision came after noticing how much work it would be to set it all up. When I want to deploy a new version of a service, I need to:

Build the docker image ( sbt docker:publishLocal )

) Upload it to Google Container Registry ( docker push --[SERVICE]:latest )

) Stop currently running Deployments ( kubectl delete -f all.yaml )

) Tell Kubernetes to restart the Deployments ( kubectl apply -f all.yaml )

The downside of this is that in order to deploy, I need to have all things setup on my machine. But to build and deploy from the pipeline systems, I would need to arrange cloud credentials, install SBT and download dependencies, which means that I should put my common library somewhere available (like JFrog artifactory). Overall, it would be extra work and costs that I couldn’t quite justify.

As I am just a single guy with two services, a shell script seems quicker.