Some time ago I was asked to look at the Resizing Images with Amazon CloudFront & Lambda@Edge article and see if it was something that could be useful.

I had some thoughts about it as it involved lambda functions that would be executed on every image request & reading and writing files to a s3 bucket

Lambda counts a request each time it starts executing in response to an event notification or invoke call, including test invokes from the console. You are charged for the total number of requests across all your functions

Duration is calculated from the time your code begins executing until it returns or otherwise terminates, rounded up to the nearest 100ms.

The price depends on the amount of memory you allocate to your function

Lambda@Edge functions are metered at a granularity of 50ms

We would also be stuffing the S3 drive with more and more images for each resolution requested as they would be written back to storage by the Lambda function after image processing & serving it

With some spare CPU standing and burning money, wouldn’t it be cool to process our own images with it?

I found Imaginary, It had all the docker stuff already done so putting it in Kubernetes was a breeze and so this PoC was born

Imaginary

Fast HTTP microservice written in Go for high-level image processing backed by bimg and libvips. imaginary can be used as private or public HTTP service for massive image processing with first-class support for Docker & Heroku. It's almost dependency-free and only uses net/http native package without additional abstractions for better performance

Supports multiple image operations exposed as a simple HTTP API, with additional optional features such as API token authorization, URL signature protection, HTTP traffic throttle strategy and CORS support for web clients

imaginary can read images from HTTP POST payloads, server local path or remote HTTP servers, supporting JPEG, PNG, WEBP, and optionally TIFF, PDF, GIF and SVG formats if libvips@8.3+ is compiled with proper library bindings

imaginary is able to output images as JPEG, PNG and WEBP formats, including transparent conversion across them

imaginary also optionally supports image placeholder fallback mechanism in case of image processing error or server error of any nature, therefore an image will be always returned by the server in terms of HTTP response body and content MIME type, even in case of error, matching the expected image size and format transparently

It uses internally libvips , a powerful and efficient library written in C for image processing which requires a low memory footprint and it's typically 4x faster than using the quickest ImageMagick and GraphicsMagick settings or Go native image package, and in some cases it's even 8x faster processing JPEG images

To get started, take a look the installation steps, usage cases and API docs

Deploying it to Kubernetes

My deployment for Imaginary looks like this

There is a lot more options if you consult the Imaginary docs

And we need a service to be able to connect a ingress to it

And top it off with a ingress ( Don’t forget to update the domain )

Signing image url’s

The documentation for Imaginary contains a good example in Go how to sign your URL’s (don’t forget the order of the request fields)

Here is my version with a little twist to give me a better url for copy/pasting

Once applied to your cluster you should be able to test your image service by loading up

http://<your service url>/resize?nocrop=true&type=jpeg&url=https%3A%2F%2Fwww.google.com%2Flogos%2Fdoodles%2F2015%2Fgoogles-new-logo-5078286822539264.3-hp2x.gif&width=200&sign=9Rawdy5gEmqGTRgxUOO7fFMYegivSQd1jNSuJljY2PM

Note that the example URL above assumes you use the signKey in my example. If you used a different key you need to generate a new URL signature

Cache it all!

If you are on AWS one can put a CloudFront distribution in front of the whole setup and leverage image caching at the edge to take a lot of load away from the backend. Image operations would only happen when the images falls out of the cache due to LRU so the more popular image, the less it would have to be requested from the backend

I’d imagine Google has some service similar to CloudFront and CloudFlare would probably be able to front the service as well

Closing words

I casually reference the ingress manifest in this article. In reality it can be a quiet complex setup to make ingress in Kubernetes to work depending on your environment and I suggest to spend the time to properly research which solution will be best in your usage case

I personally recommend Skipper from Zalando. It’s serving me well in our production environment at work. Which you btw can read about here