.

.

How to use any programming language on AWS lambda.(01 October 2018) Intro AWS lambda is a service that lets you run code without provisioning or managing servers. At the time of writing this blogpost, AWS lambda natively supports only the following programming languages/runtimes; Node.js, Java, Python, .NET Core and Go Sometimes you may have a usecase for AWS lambda(serverless) but the programming language that you are using is not natively supported by AWS lambda. If that is the case, worry not; you can still use your favorite programming language with AWS lambda even if AWS lambda does not natively support that language. In this article, we explore how to do exactly that. What up now Let's say that our favorite programming language is Nim and we want to run our program written with Nim on AWS lambda. The way to do that(at least the one we will explore in this blog) is by creating a shim using a programming language that is already supported on AWS lambda. The shim will accept lambda requests, it will then pass those requests to our Nim program and finally it will accept/listen for responses from our Nim program and marshal them into a form that AWS lambda can understand. We will use python to create the shim(but you can use any of the other AWS lambda supported languages.) The workflow is;

The python program/shim gets a request from AWS lambda. it serializes that request into json. it writes that json into stdin The Nim program reads from stdin it unmarshals what it has read from stdin and acts on it. it creates a json marshaled response it writes that json response to stdout The python program/shim reads that response from stdout it unmarshals what it read(the response) it sends the response back to AWS lambda.

# lambda.py import os import sys import json import shutil import traceback import subprocess os.environ["PATH"] = ( os.environ["PATH"] + ":" + os.environ.get("LAMBDA_TASK_ROOT", "LAMBDA_TASK_ROOT") ) def handle(event, context): try: proc = subprocess.Popen( ["./main"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True ) # write to binary program write_data = json.dumps({"event": event}) + "

" proc.stdin.write(write_data) proc.stdin.flush() # read from binary program line = proc.stdout.readline() event = json.loads(line) proc.stdin.close() proc.stdout.close() proc.terminate() proc.wait(timeout=1.2) except Exception as e: exc_type, exc_value, exc_traceback = sys.exc_info() traceback.print_exception(exc_type, exc_value, exc_traceback, limit=2, file=sys.stdout) return {"error": repr(e)} return event

receives request from AWS lambda(lambda calls the handle function for every invocation)

function for every invocation) runs the binary Nim program( ./main ) via python's subprocess lib, converts the lamda request to json and it then writes that request to the Nim program's stdin

) via python's subprocess lib, converts the lamda request to json and it then writes that request to the Nim program's stdin it reads from the Nim program's stdout using readline and converts what it has read(the response) from json

it returns that response to AWS lambda

you need to make sure that you terminate whatever you are writing with a newline, +"

"

you should actually flush what you have written, proc.stdin.flush()

you should use readline() to read from Nim program's stdout as opposed to read()

it will just read from stdin

create a response that has the event it got from the shim echoed back, the current time and a message containing Nim's version

it will convert that response to json and write it to stdout.

# main.nim import json import times import system # read from stdin var request: string = readLine(stdin) let jsonReq = parseJson(request) # for this example, we only use event; in real life you may also want to use context as well. let event = jsonReq["event"] var response = %* { "EchoEvent": event, "Message": "hello fom Nim version: " & system.NimVersion, "CurrentTime": format(times.now(), "d MMMM yyyy HH:mm") } # write to stdout echo response

echo '{"event": "myLambdaEventName", "context": "myLambdaContext"}' | nim compile --run main.nim Hint: operation successful (34568 lines compiled; 0.703 sec total; 59.941MiB peakmem; Debug Build) [SuccessX] {"EchoEvent":"myLambdaEventName","Message":"hello fom Nim version: 0.19.0","CurrentTime":"1 October 2018 13:55"}

compile the Nim program into an executable(statically linked, ideally)

create a zip file containing the Nim executable and the python program

upload that zip file to AWS lambda

nim c -d:release main.nim

zip mylambda.zip main lambda.py

So the python program and the Nim program communicate using JSON over stdin/stdout.You do not have to use json over stdin/stdout if you do not want; you could use rpc, unix sockets etc. We have talked about python talking to the Nim program, but how will python talk to the Nim program if we can't run Nim on AWS lambda? The answer lies with the fact that AWS supports running arbitrary executables So, as long as you are using a programming language that can compile binaries(that are either statically linked or built for the matching version of Amazon Linux) then you are good to go.Let's create our shim python program,Thepart is necessary for you to be able to reference(via relative path) any binaries that you upload. So, the python shim;The part where you need to be careful in writing the shim is:These precautions are necessary to avoid deadlocks [1] Our actual Nim program won't do much;NB: the previous caveats about eliminating deadlocks also apply to the Nim program(or whichever programming language you are using.) The nim program,You can execute the Nim program locally on your machine:We have created the python shim and the Nim program, to tie it all together we need to:To create the Nim executable, run;This creates an executable in the current directory, the executable is calledThe command,should be ran from a Linux 64bit computer(since that is the architecture of the AWS lamda machines.) If your programming language supports cross compilation, you can cross-compile to linux 64bit(without having to change machines). Nim supports cross compilation but - this was my first Nim program ever - I was not able to cross-compile so I had to compile using a 64bit machine running ubuntu. Next we create a zipfileThis creates a zip file calledin the current directory.Finally, let's open up the AWS lambda console and: 1. create an AWS lambda function where we chose Python 3.6 as the runtime:2. upload our zip file:make sure the runtime is Python 3.6 and Handler isie our python file is calledand the function inside it is3. click save and you'll be taken to a page that shows the unzipped file contents:4. create a AWS lamda test:5. execute/run that lambda test:As you can see, the lambda function was executed/invoked and it ran calling our Nim binary which returned results accordingly.We have been able to use our language of choice(Nim) on AWS lambda even though lambda doesn't support Nim as a programming language. Shims are a powerful way of connecting two seemingly incompatible systems. If you are interested in running your programs on AWS lambda without having to write your own shim, TJ Holowaychuk has you covered with his up project which I had written about in an earlier blog post This all begs the question; instead of AWS rolling out access to lambda one language at a time, why cant they create a common interface/shim that all languages can target? All the code in this blogpost can be found at: https://github.com/komuw/komu.engineer/tree/master/blogs/lambda-shim