At OLX we have a media server responsible for delivering millions of images per day to our users. The media server processes images on demand by resizing, rotating, adding watermarks, re-encoding to common formats and compressing.

In this post we explain how we optimized one of our services to be more effective by moving it to Rust, covering:

Decision making on language and libraries;

Our journey until rolling out to production;

Some insights on our results.

Image Processing Pipeline

We estimate that around 80% of our traffic is cached and served by our CDN provider and the remaining 20% still represents millions of requests a day and tens of thousands concurrently per second. On our busiest cluster, we get ~30000 requests/second at peak. That demands a lot of horse-power. We have a simple architecture to power everything up, consisting of the CDN in front of our users, backed up by a public API and an image processor service.

Overview of architecture

Our first solution was using thumbor in front of our services to do the heavy lifting of image processing. It served us well for some years, but it costs us a lot to scale it to a point where the response time is still acceptable. These services run in Kubernetes on AWS and to give you an idea of the load: at peak traffic with~30000 requests/second we need~6000 pods to run this service.

This component was a prime candidate for optimization, so we started designing a more efficient image processor. When you think about processing images and being fast, a language that comes to mind is C/C++, but we didn’t want to write C/C++ (and I believe at this point we don’t need to explain why, right?). So which language could give us the same performance, with no runtime penalty as C/C++?

It happens that Rust, which has been the most loved language by StackOverflow for some years in a row, provides a fast interface with C/C++ libraries through FFI. Fortunately, OLX gives us the freedom to experiment with new technologies, and, although we didn’t have experience with it, we decided to try and see how it plays out. In the worst-case scenario, we could continue running our current solution.

The pain, I mean the struggles, err… The process of learning Rust

After reading the official Rust book, I thought I was ready to start developing, so I was eager to write something with Rust. Writing a drop-in service for thumbor was the perfect opportunity to do so, as it is not a huge project and not even complicated in terms of business logic¹.

We chose the language, but we still had some decisions to make, for instance, libraries to power this service. After some research, we’ve stumbled onto Actix-Web. It looked like the most mature web framework and after seeing some buzz about it being the fastest web framework, we thought it would be the best candidate.

The missing piece was an image processing library. Firstly we thought about the image crate, but although it would be awesome to use a Rust library, it didn’t support all the required features. In the end, we decided to go with OpenCV. A well-established and mature library and with very good FFI bindings implementation for Rust. And thus we set off on our MVP journey.

One of the most common struggles on the internet from people learning Rust is that you have to fight the borrow checker² all the time. I have no idea how many times I saw the compiler telling me some variable didn’t live long enough, was being used after a move or that the compiler could not infer a lifetime for a reference. It turns out the compiler can sometimes guide you for the fixes and the messages are very friendly. With time and practice, we got the hang of it.