What is an abstract syntax tree (AST)?

In order to understand the whole process of metaprogramming and write our own codemod, first we have to understand how the JavaScript program is read by the machine.

The JavaScript engine does not read our code the same way we do. Instead it transforms the code to an abstract syntax tree. The AST is a tree-based data structure representing all the variables, calls and flow-control structures used within our code. When the program runs, the JavaScript engine traverses the tree and executes the commands represented by each node. If you want to learn more about the ASTs and discover the different parsers available, head to this brilliant article by Juan Picado.

AST Examples

Let’s start with a simple example:

Simple as 2+2

and see an AST representation of it:

We can see that our code has been transformed into simple tree with a BinaryExpression (which represents addition) as a node and two arguments (left-hand and right-hand) as the child nodes.

Now let’s consider some more complex code and its AST representation:

One declaration, one conditional statement and two console.logs

AST of the short code example. It gets complicated quite fast.

In this example the tree grew quite significantly. As you can see our conditional expression has been transformed into the node containing 3 children nodes:

test : the condition under which the “consequent” expression will be executed

: the condition under which the “consequent” expression will be executed consequent : code that gets executed if the test subtree evaluates to true

: code that gets executed if the test subtree evaluates to true alternate: code that gets executed otherwise

We can explore the tree further and inspect each of these paths separately. Our x > 3 statement has been converted to BinaryExpression subtree with two arguments — similar to our 2+2 example.

Our console.log(x) expression got expanded into MemberExpression which is an AST representation of taking a property from the object ( console.log ) and then combining it with the arguments ( x ) under the CallExpression node.

Our example code (on the left) and the AST (on the right)

If you want to inspect your code’s AST, you can use AST Explorer to generate an AST representation. You can explore the tree in the JSON format on the right. When you hover over a node the corresponding code will get highlighted.

This form of code is really easy for machines to analyze and modify. In fact it’s exactly what jscodeshift uses to modify!

Real life examples

Before we start writing our own code modifications, we should take a look at what codemods can do and where they are useful. Lots of them are documented really well.

Migrating to Jest

The Jest migration guide recommends using jscodeshift to automate the migration process from popular test frameworks like AVA, Chai, Expect.js, Jasmine or Mocha to Jest. The codemods themselves can be found on the jest-codemods GitHub. There are many great stories describing how this codemods helped developers to migrate their projects faster, such as when Kent C. Dodds migrated a codebase from AVA to Jest at PayPal.

React breaking changes

React includes codemods as part of the migration guides between versions that introduce breaking changes. These guides (in theory) allow you to migrate between versions with just a single command. In practice, some manual fixing is usually needed but having a codemod speeds up the process significantly.

Gatsby migration v1 -> v2, aka codemods are hard

On the other hand, codemods are not always that helpful. Recently I tried to migrate an old Gatsby v1 project to the latest version using the official codemods. Unfortunately, even though our codebase was fairly simple, some codemods didn’t work and others generated invalid code. Writing a good codemod is not straightforward given that it needs to run on all possible codebases. In our case migrating Link components broke the app because we had an abstraction on top of Gatsby’s class. After the migration all of the imports were duplicated and we had to fix them manually.

Migrating lodash / underscore to native functions

Another interesting codemod I have found migrates a codebase from lodash and underscore to ES6 equivalents. In the past using lodash and underscore was justified by the fact that most of the methods they provided weren’t part of JavaScript’s functionality. Nowadays most of these methods are included in the language itself. Unfortunately migrating by hand would be time consuming and error-prone. Thanks to codemods we can do it with a single command: