I found this design pattern as part of my previous series and thought I’d expand it to its own mini-post.

First some background: sometimes we have a series of asynchronous operations that need to occur in a particular order. For example, in a human resources app:

A job applicant uploads their CV, then the server saves that raw data and processes it for particulars (e.g.: university GPA, if it’s for a graduate role, or perhaps keywords instead), then the server process then serialises that data and saves it as a database entry, then the server responds with a “We’ve successfully saved your application and look forward to speaking with you soon”, or some other such canned message.

We can take these fundamentally asynchronous operations – file I/O, parsing, and database calls – and process them as Promises:

app . post ( '/apply' , ( req , res , next ) => { saveUploadedFile ( res . body ) . then ( filePointer => processApplicationDocument ( filePointer )) . then ( serialisedData => saveToDatabase ( serialisedData )) . then ( dbResponse => res . send ( 'Thank you.' )) . catch ( error => { handleError ( error ); res . send ( 'Whoops!' ); }); });

Or we can use async / await syntax:

app . post ( '/apply' , async ( req , res , next ) => { try { const filePointer = await saveUploadedFile ( res . body ); const serialisedData = await processApplicationDocument ( filePointer ); const dbResponse = await saveToDatabase ( serialisedData ); res . send ( 'Thank you.' ); } catch ( error ) { handleError ( error ); res . send ( 'Whoops!' ); } });

And both of these work. Unfortunately, they’re brittle – the structure of the code depends on the business logic in which it’s operating. What if we’re performing the same process, of performing asynchronous operations in a queue, over and over again?

Well, in the series I just finished writing I stumbled on a useful pattern to handle this. I’m sure there are others, but it boils down to this:

Each step is its own function, which returns a Promise:

// Gross oversimplification of what really happens: function saveUploadedFile ( fileData ) { return new Promise (( resolve , reject ) => { const filePointer = './abcde.pdf' ; fs . writeFile ( filePointer , fileData , ( err ) => { if ( err ) { return reject ( err ); } else { return resolve ( filePointer ); } }) }); }

These functions are then collected in an Array:

const queue = [ saveUploadedFile , processApplicationDocument , saveToDatabase , ];

That Array is then reduced with an asynchronous processing function:

const result = queue . reduce ( async ( prev , next ) => { try { const result = await prev ; } catch ( err ) { handleError ( err ); // and optionally... throw err ; } return next ( result ); }, true );

What’s great is that we can then refactor this abstraction as its own function, and pass arbitrary numbers of asynchronous operations to it.

I’ve got a hunch that this already exists in a library somewhere, but haven’t taken the time to research that. If you do know, let me know by email or hit me up on Reddit – I’d love to get in touch.

There’s a companion repo on GitHub here if you want to immediately clone and get going.