A common problem with image processing Rails apps has been the excessive memory consumption within the worker processes. When images are large enough, our worker will consume a big portion of our memory and will take forever to finish. If you are deploying on Heroku, you will probably be paying for the cheapest worker and the R14 notification emails will be flooding your inbox.

A few weeks ago, we discovered another AWS’ amazing product: Lambda. From their documentation, “AWS Lambda is a compute service that runs your code in response to events and automatically manages the compute resources for you, making it easy to build applications that respond quickly to new information”. In addition, Amazon has a very nice and easy to follow post on how to use AWS Lambda for image processing.

In this post, we will use as our starting point the post mentioned above to construct a solution that works with our Rails app.

Processing images using S3 + Lambda (AWS post summary)

The basic flow for image processing goes as follows:

A user uploads an image to an S3 source bucket. The bucket emits an event which is listened by our AWS Lambda function. The function will then process the image and store it in a target bucket.

Why is it using two different buckets?

AWS S3 buckets can be configured to trigger events when a file is created. Since we will be creating new images from the original one, we do not want an event loop!

If you need further details on how to create an AWS Lambda function or any other part of the flow, go ahead and read the post. It is amazingly easy to follow!