Series content

What this article is about

Will your hard-learned Kubernetes knowledge become obsolete through serverless and did you waste 3 years of your life?

How to use serverless without becoming cloud provider locked-in?

This isn’t a comparison of specific tools, but rather general ideas

TL;DR

Serverless on Kubernetes reduces repetitive configuration in a cloud provider independent way. It’s just the result of continuously automating away manual work.

When we’re talking about serverless on Kubernetes we need to consider two different areas:

deploying applications serverless in the cluster (reduce the number of YAML files needed for every app + auto-building containers) running pods’ containers serverless without managing nodes/VMs

Serverless

In short: you don’t or interact much less with the servers and infrastructure necessary to run your applications. When using the word “serverless” today it can point to two different things: CaaS and FaaS.

CaaS — Container as a Service

You create the (Docker) container, throw it at the CaaS and it runs, serves and scales it automatically.

Managed examples are Azure Container Instances, Google Cloud Run or AWS Fargate.

FaaS — Function as a Service

You write code, throw it at the FaaS and it runs, serves and scales it automatically.

Managed examples are Azure Functions, Google Functions or AWS Lambda.

FaaS implementations

How the FaaS runs your code can happen in different ways. One way could be that the FaaS actually builds a container for every code change and then uses a CaaS like this:

FaaS building a container and sending it to CaaS

Another way could be that the FaaS pulls the function’s sourcecode into a pre-defined environment (container) dynamically during boot. Environments would be available for different languages. When using a language which has to be compiled like Go, then compilation also has to be done during boot.

Events / Scaling

FaaS are most of the time used in conjunction with eventing systems which trigger the instantiation of the functions. Events can originate from API-Gateways, Github, Kafka, RabbitMQ, CronJobs etc.

the FaaS encapsulates communication with event sources

For every event, a new function will be created to handle it. If there are multiple events happening at the same time multiple instances will be created to handle these. This way we have automatic scaling.

The FaaS communicates with the various event sources, so the functions themselves don’t have to. They only have to work with one event format the FaaS uses, like CloudEvents or transport via HTTP.

There is the CloudEvents project which describes the structure and metadata of events as a “standard”. It also includes data and a schema which describes that data. Cloud events are envelopes that wrap around the event data. This could be great if adopted by many vendors to gain interoperability.

Kubernetes Applications

Let’s have a look at steps necessary to develop a traditional non-serverless application running on Kubernetes:

quite a bit of manual “server” interactions necessary

We need to build a container, create various Kubernetes resources (YAML files) and then decide how many worker nodes we need to run our app.

The decision of how many worker nodes we need could be handled more dynamically by configuring a Cluster/Node autoscaler. Though even then we still have to configure it and need to set a min+max amount of nodes.

We interact with “servers” quite a bit with this traditional approach. First creating/building a container, then writing YAML files and also defining the amount and resources of the nodes.

Kubernetes Serverless Applications

Now let’s explore serverless approaches when developing apps for Kubernetes.

CaaS — Container as a Service

we reduced the creation of a lot of YAML files

Here we reduce the amount of Kubernetes resources (YAML files) to be created significantly. The CaaS will create all necessary subresources for us, like autoscaler, Ingress or Istio routing.

All we do is provide a (Docker) container and create one single k8s resource, the CaaS-container resource introduced via a CRD. The CaaS decides when to start instances of our app and how many, maybe based on events or on our definitions.

We have to make sure that the container that we build can receive and handle the events coming from the CaaS which can be through HTTP or CloudEvents for example. This may require certain libraries inside the container.

CaaS Example: Knative (Knative provides flexible building blocks which other solutions can use and depend on).

FaaS — Function as a Service

we now also automate the build process and maybe have a nice FaaS webinterface

The FaaS function.yml will contain one K8s resource from the FaaS system, introduced via a CRD. In that resource, we set things like the function name, location of source code, language runtime and trigger events.

If we upload code via a webinterface then creating the FaaS function.yaml wouldn’t be necessary. But keeping functions as code should be good practice. Webinterfaces are good for prototyping or testing modifications.

With a FaaS we also have everything that a CaaS solution provides. But now we reduce the work even further because we have tools running in our Kubernetes cluster which can execute/build our application source code directly.

The container that is built for us already includes the necessary libraries, like HTTP or CloudEvents, to receive events from the FaaS. We don’t have to worry about that.

The source might be stored in a Git repo, is uploaded via a webinterface or is available someplace else. The FaaS will access the code, listen to changes, build the container and then pass it to the CaaS for serving end eventing.

example webinterface of TriggerMesh to upload code and deploy as a function

FaaS Examples:

Cold and warm starts

A cold start would mean that no pod is already running to handle an event, so it’ll take some time creating it. Usually, these pods will stay alive for a bit after they were used last and can be reused. Invocations during the “already-running” period will be called a warm start. Warm starts are faster but also cost resources.

Fission / Environment based FaaS

Fission architecture (source)

One K8s FaaS example, Fission, doesn’t actually build immutable containers for every function’s code change but uses the idea of mutable environment containers (“Generic pods”) which pull code in dynamically to then convert these into “Specific Function pods”. I think this is also they way AWS Lambda works utilising AWS Firecracker.

Observability

Moving from containerized microservices to functions could result in having to manage even more and smaller services than before. This is caused by the ease to create small functions which just listen to and handle one single event. I noticed this myself when I created the same demo app as Container Microservices in Part 1 and AWS Serverless in Part 2.

To manage the higher amount of services or functions its necessary to keep observability (metrics, logging, tracing). This is why most Kubernetes FaaS and CaaS integrate already with for example Prometheus, Jaeger and service mesh like Istio or Linkerd.

Kubernetes Serverless Nodes

In the previous section we talked about K8s serverless applications and we saw workflows when using CaaS or FaaS. These simplify processes and reduce repetitive work by a lot.

But the developers or operators are still interacting with servers: the VMs used as worker nodes in the cluster. They still need to specify how many nodes to have and their resources (CPU/memory).

Now we go one step further and make the actual underlying Kubernetes nodes serverless using the Virtual Kubelet.

FaaS on top of Kubernetes with Virtual Kubelet as nodes. The ultimate setup?

The Virtual Kubelet simulates a worker node to Kubernetes which can then be used to schedule pods on, like with any other normal node. Though the pods’ containers aren’t running on a VM, but in the Serverless Container offerings of cloud providers like AWS Fargate, Google Cloud Run or Azure Container Instances.

Read more in my detailed article about the Virtual Kubelet.

The combination of K8s serverless applications and K8s serverless nodes for Kubernetes could be a powerful one. But why then still use K8s at all if we serverless everything away?

Why still use Kubernetes?

sorry, I was in the mood for a goofy image O:)

Kubernetes provides great and flexible building blocks rather than being made for easy interaction and end-users. This makes K8s complex and requires a lot of repetitive work when using directly.

Kubernetes became a cloud provider independent standard. Using serverless frameworks on top of it makes sense to keep this independence when going serverless. We can always define our applications in more detail if necessary because it’s still just running K8s under the hood.

By using serverless on K8s we can reduce repetitive work by a very high percentage, which then allows us to spend more time on building the actual applications.

Recap

traditional

serverless

Conclusion

I think modern serverless event-driven architecture has already proven itself and will become more and more prevalent in the coming years. Let’s work on using it on top of open standards like K8s to ensure greater interoperability.

Serverless on Kubernetes is just the result of continuously automating away manual work.

Sources

6. Another great video about Kubernetes, Serverless and OpenWhisk

7. Yet another video presenting OpenFaaS, Knative and Nuclio

Become Kubernetes Certified