Serverless development is simply the best. Double click, upload your code and you are done, right? Most people are more than happy to leave it at that. If you are not most people, and up for some Lambda exploration, this article is just for you.

In the previous article we got a shell to the Lambda container, downloaded Lambda runtime environment and discovered its components:

bootstrap.py — the python code wrapping our handler.

awslambda/runtime.so — a python-compatible shared object bootstrap.py uses it for, well, pretty much everything.

liblambda*.so — In his turn, runtime.so uses other shared objects. We will focus on liblambdaruntime.so, in charge of the heavy lifting in managing the Lambda logic.

We also had some fun messing around with bootstrap.py. This time we are going to roll up our sleeves and dive into the binary libraries of the Lambda runtime environment. We will explore Lambda’s billing system and (spoiler alert) have some fun messing with Lambda timeouts.

“Oh, the places you’ll go! There is fun to be done! There are points to be scored. There are games to be won.” — Dr. Seuss. Photo by Joshua Earle

Exploring the Libraries

The libraries (liblambda*.so) are compiled with symbols, so you can learn a lot about the libraries just by going over the symbols names. Also, runtime.so exposes a lot of these functions by importing and wrapping them, so a Python script (bootstrap.py in our case) can use some of them. How convenient!

Partial functions list from liblambdaruntime.so disassembly. Thank god for symbols.

One of the things I initially really wanted to check out was the behind the scenes of the Lambda’s billing system, and just by looking at the function names, I had some experiments I wanted to try. But first — let’s talk a bit about Lambda billing.

Lambda Billing

Lambda has a time-based pricing model, and without going into all the details, the gist of it is the longer it takes your Lambda to run, the more you pay. When invoking a Lambda, you can easily spot its beginning and end in CloudWatch Logs, as well as its duration and billed duration.

CloudWatch logs for a Lambda. You can see both the Lambda’s duration and the billed duration

However, there is a more complicated scenario. Consider the following Lambda:

On a typical run, the duration of this Lambda should be small (the billed duration should almost always be 100 ms). But what happens on the first invocation? Or on cold starts (where the module is re-imported)?