Example problem: File manipulation

I’m working on a side-project that allows users to upload photos to be displayed on an LED matrix. In other words, I’m letting the internet decide what my wall art should look like.

Right now, I’m working on the ‘web’ part, which is a Node/React app that allows users to upload a photo. It needs to turn a regular, large image into a matrix of pixels, like so:

I am aware that this is probably a horrible, horrible idea :)

The client-side app is a very simple React app. The real business happens on the server. The steps are as follows:

Read the file into a Buffer.

Crop and resize the image to be 32x16 (the size of the LED matrix).

Save the resized image to disk (for reference).

Extract the RGB colour data for the 512 pixels in the image.

Send it to the client for previewing.

To accomplish this, I need to rely on:

fs , the built-in Node module for working with the file system

the built-in Node module for working with the file system imagemagick-native, an NPM package that wraps ImageMagick. ImageMagick is a command-line utility for manipulating images, and it’s absolutely fantastic (I’ve used it before, for ColourMatch!).

Original Sin: Callbacks from Hell

Here’s what the route looks like, using the traditional “callback” style of asynchronous management:

This solution works, and it’s not even that bad. The biggest issues I see with it are:

No unified error-catching, each step’s failure needs to be handled separately.

A lot of visual clutter. It’s doing something pretty straight-forward, but it’s obfuscated by all the callback noise.

These problems will be compounded i̶f̶ when the requirements grow, and the route becomes more complex.

Let’s see how we can make it better…

A Promise Worth Keeping

Our first order of business is to convert each of these methods to support promises. Async/await is built upon functions that return promises, so we need to build some!

I’m gonna assume some familiarity with what promises are and how they work; if this is unfamiliar or rusty to you, MDN has a great article.

Let’s start with our filesystem methods, readFile and writeFile.

When we invoke one of these functions, it returns a promise that has been built to mimic the original, callback-based fs methods.

The original methods follow the errback Node style of returning two arguments: first the error, and then the data itself. We can take advantage of this signature, and resolve/reject based on the “truthiness” of the err argument. If there’s an error, reject with it; otherwise, resolve with the buffer.

You may have noticed that this solution is not very DRY. The two functions are incredibly similar. Let’s generalize it:

We’ve created a new wrapWithPromise helper function, which we can use to create new promises based on any errback functions we have!

This function is pretty dense, so let’s examine it in more detail.

We’ve set it up to be curried; We want to be able to create substitute functions (so, a readFilePromise instead of readFile), and to do that, we need to supply the function — readFile — before we know what the arguments will be (the path to the file).

So we create a function that returns the substitute function, and that function can be invoked with our “real” arguments.

Confused?

Currying (and first-class functions in general) take a while to feel comfortable with. Don’t worry if you’re having a hard time following :)

We’re using the ES6 rest operator to collect the arguments into an array. This is an important step, because the two functions we’re wrapping don’t have the same arity.

Next, we return a promise. The promise only has one job, and that’s to invoke the wrapped function with the supplied arguments. We’re using the ES6 spread operator to “unpack” them.

Because we’re working on the assumption that any wrapped function accepts an errback callback, the final statement doesn’t change; we reject when an error is provided, otherwise we resolve with the returned result.

External Packages often work the same way

We’ve tackled the fs module’s methods, but what about imagemagick-native, our NPM package to do image processing?

Happily, Because it follows the same errback signature, this is a piece of cake.

The world’s shortest Gist.