[Today’s random Sourcerer profile: https://sourcerer.io/sandangel]

Since 2015 the JavaScript language has radically changed. A pair of features stand out as having the potential to revolutionize JavaScript programming: async/await functions, and the ES6 module format. While most of the new JavaScript features are simple syntactic sugar, these two are huge.

Async functions offer us a way out of the pits of callback hell, by eliminating the need for callbacks through an ingenious application of Promises. It is async functions that delivers the promise of Promises, if you will.

The ES6 module format gives us a standard modularization mechanism that’s portable across browsers and Node.js.

Node.js 10 has native support for both features making it an excellent choice for learning about the future of JavaScript. Async functions became available during the Node.js 8.x time-frame, and are already affecting how Node.js programmers approach coding. Native support for ES6 modules became available in the Node.js 9.x release train, as an experimental feature.

Let’s write a simple Express application using both async/await functions and ES6 modules. It’s an implementation of the old-school fortune program, showing a random entry from a fortune file. The origins of this program are from the mid-80’s, and there are several variations of fortune files available. The file included with the repository associated with this project was copied out of the MacPorts instance of the fortune program.

The instructions assume a Unix-like system (macOS, Linux, etc). If you’re on Windows, consider installing the Windows Subsystem for Linux, or interpolating the commands to run in the CMD.EXE shell.

To get started install either Node.js 9.11.1 (the most recent version as of this writing), or Node.js 10.x if it’s available by the time you read this article. If you’re using 9.x, implementing the __dirname variable requires a small workaround we’ll cover.

The code in this article can be found in our repository at: https://github.com/sourcerer-io/sourcerer-blog/tree/master/nodejs-async-await-esmodules

Set up the project:

This generates for us a simple Express application. We can start it right now by running:

At this stage the app doesn’t do anything special. To get going with async functions and ES6 modules we need to perform some major surgery.

Let’s start by renaming some files. In Node.js, ES6 modules are required to have a .mjs file extension. This choice by the Node.js leadership is controversial to a few, but they were unable to implement ES6 modules using the standard .js file extension. You will find some references to Michael Jackson Script for these files. Whether you like it, or not, it is what it is, so let’s do the following:

The last line is because we do not need the users module.

Reading fortunes in an ES6 module

Before we modify those modules, let’s write a simple ES6 module, routes/read-fortunes.mjs. It will contain the code to read a fortunes file. There isn’t a standardized format for fortunes files. The chosen file has each fortune separated by a line consisting solely of a “%” character. So it’s simply a matter of splitting the file on such lines.

The fs-extra module supports the API of the standard fs module, along with a few helpful additions. The most important feature of fs-extra is that its functions return Promises. Inside an async function, Promises are an easy-to-handle alternative to callbacks.

The readFortunes function is a demonstration of the power of async functions and Promises.

The normal fs.readFile, like most of the Node.js baked-in functions, takes a callback function which we’ve totally avoided. With callback functions it is easy to get into a situation requiring several stages of asynchronous processing. Using callbacks requires several nesting levels and careful coding to ensure of handling all possible failure modes. Each processing stage means another level of callback, and more complication to ensure results and errors arrive at a useful location.

For contrast let’s review how we got to async functions, first with a callback approach

It’s the same functionality but the intent of the programmer is obscured. The callback function, and callback-oriented error handling, introduces boilerplate code that impedes our understanding of what the programmer intended.

While the callback-oriented paradigm served JavaScript and Node.js programmers well, the phrase “callback hell” demonstrates the pain we’re enduring. It forces us to handle errors and results unnaturally, which is the root cause of the of the problem-set solved by async functions.

Bottom line: Our colleagues programming in other languages can perform asynchronous operations without a messy pyramid of asynchronous callbacks. Why can’t we?

Let’s look at what Promise’s brought to the table:

This is cleaner because it has less boilerplate code. Some of us were excited by Promises because it flattened nested callbacks into a chain of Promise handler functions. As promising as Promises were, they weren’t quite good enough if only because results and errors still land in inconvenient unnatural places.

There’s an architectural step to skip over, Generator functions, because the explanation is tricky, and in any case we end up in an almost identical place.

The async version of this function is the clearest. The programmers intent shines clearly, errors and results land in a natural convenient location, and the only boilerplate is the async and await keywords.

An async function is introduced by the async keyword. The await keyword is only usable inside an async function. For any Promise object inside an async function, the await keyword suspends execution of the function until the Promise is either resolved successfully, or resolves to a rejection. For a rejection, an Exception is thrown containing data from the rejection. Otherwise the successful result is returned.

In this case the fs-extra version of fs.readFile returns a Promise that, when resolved, will result in the data read from the file. Unless an error occurs, in which case the Exception is thrown. Both behaviors are the natural way to handle errors and results in any programming language.

The purpose of readFortunes is reading the fortunes file (specified in the FORTUNE_FILE environment variable), and splitting it into an array determined by lines with a % character. Because the fortunes file we chose has an initial entry containing a file version number, we need to throw it away using the slice function. The resulting array is stored in a variable global to the module.

Now that we’ve got that function out of the way, let’s turn to the default export of this module:

ES6 modules have a very different paradigm to the CommonJS modules traditionally used in Node.js. In CommonJS modules we assign anything to be exported from the module into the module.exports object. This simple approach was good enough to build the whole Node.js ecosystem with the hundreds of thousands of modules in the npm registry. But ES6 modules offer a chance to have the same module paradigm on both browser and server.

With ES6 modules exporting a thing from a module is handled with this new keyword, export. Multiple things can be exported from a module, using export each time. The default export (the export default phrase) is the primary thing one can export from a module, and there can be only one default export.

In this case the default export is an async function letting us use async. It only calls readFortunes if the fortunes have not already been read. Once the fortunes array is in memory, it selects a random fortune. Very straightforward.

To test this module, create a file named tread.mjs containing the following:

We saw async regular functions earlier. An async arrow function looks like:

The idiom shown here is a way to run async code in a simple Node.js script, by creating the async function and immediately calling it.

The import keyword is the opposite end of the export keyword shown earlier. An ES6 module exports things with the export keyword, and another ES6 module imports those things using the import keyword.

The default export from an ES6 module is used as shown here. In this case because it is an async function, it automatically returns a Promise, and we must use await to wait for the Promise to resolve. To see any errors we’ve attached a catch block.

The default Express Router, as an ES6 module

When we generated the blank application, a Router module was created as routes/index.js. We then renamed it to routes/index.mjs, but did not convert it to an ES6 module. Let’s now do so:

Instead of defining a default export, the Router object is exported as router.

We define one route which in turn uses an async function. Because Express router functions do not understand async functions we need to approach this carefully.

An Express middleware function would call next() to cause Express to proceed to the next routing function. We don’t have a middleware function, and in the typical case we’re required to not call next(). Instead we are to call res.render, or res.send, or similar functions to send the response to the browser. If, on the other hand, we’ve detected an error we are to call next(err).

Using a try/catch block as shown in this case consumes the errors in the natural fashion. It is natural is for errors to throw an exception, and therefore it is natural method to detect errors by catching exceptions.

Express main application as ES6 module

By default express-generator sets up the application code in two files, app.js and bin/www. What we’ll do is combine the two into one file, app.mjs. Since we already renamed app.js to app.mjs we just need to get on with the surgery.

This sets up the imported modules and the debug utility.

The import of routes/index.mjs uses a new form of the import statement we’ve not seen. This is similar to the object destructuring assignment, another new ES2015 feature. An imported module is an object representing the entirety of the module. Using this method we can import a selected portion of the module, and at the same time give imported functions a different name.

Suppose we had several routing modules, each of which exported a Router object under the name router. Code in app.mjs must distinguish one router object from another. This is how that’s done. The as clause is similar to a database query in that the imported function becomes known by the supplied name.

One unfortunate feature of adopting ES6 modules is that the global variables Node.js injects into CommonJS modules are not available. In a few lines of code we want to use the __dirname variable to reference files in the file-system, but that variable is not available in the ES6 module context where this code lives.

The unworkable solution is to construct a pathname like ./public/assets/stylesheets/style.css. Such a pathname relies on the current working directory being the home directory of the application, but that will not always be the case. The __dirname variable was convenient because Node.js guaranteed it to be equal to the pathname of the directory containing the currently executing file. That meant always being able to reference files sitting next to the module.

The preferred solution is to calculate the directory pathname using the import.meta.url variable as shown here. In Node.js 9.x that variable was not available, but it is available beginning with 10.0. The solution for Node.js 9.x is to construct a CommonJS module named dirname.js containing this:

Such a module can be used as shown here to get the __dirname variable.

This is the remainder of the application wiring. It sets Handlebars as the preferred views engine, enables the public directory as the location of static files, brings in the routes/index.mjs router, and a few other straightforward tasks. Some of this code is copied from bin/www.

Preparing templates

The final task is a small changes in the templates. The provided views/layout.hbs is acceptable for our purpose and does not require change.

We need to change views/index.hbs to this:

Earlier in the routes/index router, we supplied a fortune message in the variable named fortune. This is where we receive the fortune message.

Running the Fortunes app

Running the application is very simple. The package.json we got from express-generator to make it simple:

Then open a web browser connecting to http://localhost:3000 and you’ll see something like this. Reload the window a few times, and you’ll see new messages each time.

Wrapping up

This introduction to async functions and ES6 modules is your start to understanding how they can improve your code. No longer do we have to struggle with nested callbacks that obscure our intention and increase debugging difficulty.

Ryan Dahl’s key observation inspiring him to create Node.js was that blocking operations, like database queries, caused the architectural complexity of threads in other languages. JavaScript gave us a simple mechanism to avoid blocking operations by an ingenious use of a single-thread event-loop architecture. But by using anonymous callback functions the design led us into the pits of callback hell.

Async functions are our way out of that problem.

The code in this article can be found in our repository at: https://github.com/sourcerer-io/sourcerer-blog/tree/master/nodejs-async-await-esmodules