Today I was working on my 2020 plan to turn things into AWS Lambdas.

Better put: hopefully only the things I should turn into Lambdas. You know, the sorts of things that don’t really need a cloud server sitting around being billed by the hour when a microservice could handle the job on-demand.

The particular feature stack I was working on involved relying on a custom Region service that could look up US/Canadian postalcodes and provide a variety of information about them: alternate city names, latitude/longitude — you get the idea.

Our own custom data is glommed onto the standard postalcode data, so traditionally this has been served up by a RESTful API running on its own EC2 instance running a LAMP stack (Amazon Linux 2, Apache, MySQL/Aurora, and PHP).

“Wouldn’t it be nice?” I thought, “If my app could call a Region resource through Amazon’s API Gateway and just let a Lambda handle this?”

And in my world the thought process for Lamba’izing a service has, historically, gone a bit like this:

Can I even do this?

Let me prototype that real quick.

OK, I found a lot of problems.

Should I even be doing this?

Oh, I’m already done. (or, conversely: Never mind, this was a bad idea.)

Laying out Requirements

I won’t bore you with a full readout of every single API resource this Region service provides. Suffice it for my explanation that its #1 job was to look up a postalcode using a GET request and return some custom data about it.

In fact, it would look like this: GET https://api.com/region/postalcode/20008

Not complicated, right? That is exactly the kind of interface that AWS API Gateway makes incredibly simple to make using custom path parameters.

Custom Path Parameter for a GET Method Request (AWS API Gateway)

But the cloud instance also had a job that wasn’t just fetching data and spitting out a useful JSON object for the client app to consume: it was handling periodic updates of the Region data using an external vendor.

In short, the codebase that I was converting from PHP to Python had a hook to handle importing vendor data into the database, and while it wasn’t part of the API directly, it was using the same Region classes and part of the same codebase.

To Import, or Not to Import

Suddenly I was faced with a question: I could build a single container of Python scripts (after converting the logic from PHP) that could handle all of the functions the original PHP service was, including updating the database with the latest postalcodes — I believed this.

Should I?

The truth was I didn’t even know if an AWS Lambda could handle that kind of job. Downloading a file, unzipping it, mass-updating some tables?

A preliminary Google search didn’t seem promising: all the top results used S3 as a temporary storage method rather than the Lambda’s own storage mechanism.

Still, I decided to try anyway.

On Motivations and Methods

Converting the older PHP logic to a set of Python 3 scripts, and abstracting out the data and actions to separate ini and sql files was worth most of the effort in and of itself.

I was able to streamline the codeflow purely through being forced to rearchitect the original, OOP-based PHP to a set of modular Python packages.

The unit tests were easy to write and since the Region package contained local copies of its dependencies I could just call the Region.py package and invoke the lamda_handler function with test data directly: no main required.

All the resources like GET /region/postalcode or GET /region/city mapped 1:1 between the two versions of the codebase.

But the import functionality was decidedly not ever meant to be part of an API, so why did I try converting the import method to something usable by AWS Lambda even if I was going to restrict access to it?

In short: a combination of curiosity and the desire to have a single set of packages I could work with.

Instead of using cron to call the update every week, I could use AWS CloudWatch to fire an event triggering the import process.

Having finished the code conversion and tested the Python scripts on my local dev machine it was time to test it as a Lambda.

First, Zipping Up the Work

The AWS Lambda interface makes it incredibly easy to work with your Lambdas while debugging.

You can zip up a file of everything you want to use for the Lambda, and upload it all at once. The in-page editor is an IDE and lets you navigate files, edit things, save, and run tests.