Paying for Wait Time

An often-touted feature of Functions as a Service (FaaS) like AWS Lambda, Azure Functions, and a growing list of others is that you don’t pay for idle.

When your function is executing, you pay for some fine-grained interval — often 100 milliseconds. When your function isn’t executing, you don’t pay anything. This is great! The problem of those 10 million idle servers will go away forever!

Unfortunately, it’s not that simple. Let’s explore what happens with a single service that relies on asynchronous IO for efficiency within a serverless architecture. Each function gets its own container with a guaranteed amount of resources. You’re billed for only those resources and the amount of time you’re running. But remember — it doesn’t matter whether you use those resources or not.

While a server-based architecture can make more efficient use resources with asynchronous IO, this isn’t true of Functions as a Service. You don’t pay for idle, but you do pay for wait. As I’ve discussed in prior posts, it’s not that useful to use asynchronous code within a function. They should generally be single-threaded, single-task — otherwise, you’re just using FaaS as a platform to build servers.

Compounding the Wait Time

This problem gets worse as we expand our view from a single service to a set of services interacting over RPC.

When a function makes a call to another service, you pay for the waiting time — even if the code is using async IO. As we discovered earlier, you’re also paying for the execution that’s happening in the called service — so you’re paying double during this time.

The problem is compounded if the called service itself makes calls to other services. At any given time, only one function is actually making use of its resources, but all functions waiting on it are billing for their resources. We’ve essentially brought back the idle server problem.

Let’s take another look at the resource graph for a function using asynchronous IO with callbacks on a server:

The Best of Both Worlds

Instead of referring to functions in our code, what if the diagram referred to functions in our infrastructure — and the FaaS platform managed the invocation of our callbacks? If that was the case, then our server resource diagram becomes our infrastructure diagram:

Since each function is billed independently, we don’t have a fixed provisioning. As a result, we’re getting the benefits of FaaS without the paying-for-wait problem — the best of both worlds!