Background to our journey

Let me share a story. One that perhaps is not so different from what you are facing currently. I am currently working with developing a service that has a React component as one of its main consumables, to be integrated into our company’s main application or standalone. This component has a series of internal dependencies and itself is a dependency of this main application, so far nothing new under the JS world.

Thing is, despite our best efforts, our relatively small component was super slow, especially on 3G. Sounds familiar?

We can always try to guess where we have performance problems, why is it not loading fast enough and such, but more often than not this leads us down the wrong path.

So, how can you keep an eye on how your application is performing in the wild? You can load the app in the browser and fire up the dev tools in it and render. Although that is fine for more granular insight, it does not easily give you something to compare against. Nor does it gives you an overall progression/history of your app’s performance.

In this first chapter we are going to learn about measuring our performance and keeping a constant watch on it.

Enter Lighthouse

So if you've never heard of lighthouse, here’s a bit of a background story. On the Chrome Dev Summit of 2016, Paul Irish introduced us to a new tool called Lighthouse. It has since evolved and has become a go-to tool for gathering insight on how your app fares on the wild. Nowadays Lighthouse is available in different forms and flavors, from CI-ready modules to node CLI and built-in on Chrome dev tools. As anyone with enough experience knows, performance optimizations and governance starts with measurements. And that’s where Lighthouse comes in.

I wanted to keep track how performance of a single PR as well as our entire codebase progressed over time and use those numbers for further investigation.

As a side note, testing performance on slow connections is, well, slow. To enable faster testing cycles, Lighthouse version 3 simulates slow connections. This is great for CI but since the numbers are somewhat artificial, you may need to use Chrome’s integrated Lighthouse audit to get true hardware-throttled numbers.

How to integrate lighthouse

You have two options on how to use Lighthouse in CI. You can choose to install it with npm install -g lighthouse for the 'raw' node CLI, or Lighthouse CI for the easier plug-and-play.

The node CLI provides a great deal of insight and metrics that you can use, including: performance, SEO, usability and offline capabilities. I ended up choosing the node CLI for my project’s needs.

Lighthouse CLI is highly customizable and really well documented. You can customize what audits to run and configure how it will run. Mobile device emulation and also CPU throttling, it is all well covered on the type definitions.

Gathering metrics and detecting changes

Now that we have decided how we want to integrate lighthouse into our workflow, it is time to gather the metrics and detect regressions and improvements per pull-request.

In our workflow Lighthouse is used to run and store metrics every time a branch gets merged into master. We use these metrics as a benchmark for each pull request to master, generating a digest file containing the regressions or improvements that we care about.

A little setup tip: Lighthouse currently only properly supports using chrome-launcher, meaning that you’ll have to have Chrome installed in your CI. For that we use puppeteer and use the default puppeteer.executablePath() to easily provide a chrome binary for chrome-launcher using the chromePath option.

So, in summary, the lighthouse task will run against a statically hosted fresh build of the pull-request’s branch and gather metrics. Then it will compare those metrics against a report file from master and identify possible regressions and improvements. A digest file will be saved for each regression type. These digest files will then be used to report back on the pull-request as a well formatted comment.

example of a lighthouse report on a pull-request

There are many ways you can configure your lighthouse flow. On the setup that we have, we introduced the code of the pull-request branch into an isolated create-react-app build to reduce noise and make it more likely that we’re comparing more grounded numbers instead of less meaningful metrics caused by third party interferences.

Your Lighthouse task will point to the static build, which can be hosted on a simple Github page or something similar. For deployment of a create-react-app read the documentation here.

The master report is updated every time master is updated.

Using lighthouse for performance governance

Having Lighthouse integrate in your pull-request workflow proves itself when you are evaluating performance improvements. Here we have two images of an update on the same pull-request where I was trying some code splitting and chunk prefetching techniques. This is because with performance, the best approach is not always the one you think it is.

Initial commits on PR, some regressions identified

Final lighthouse update on pull-request, now we even have improvements

On the first image I ended up introducing regressions. My prefetching was too aggressive so on mobile devices it ended up hogging the main thread and introducing large parsing times. This in turn blocked the render for quite a bit, slowing things down everywhere.

On the second preview, I took a smarter approach. I created high, medium and low priority chunks. High priority chunks were shipped in parallel with the main chunk. The rendering only had to wait for the main chunk to start. The medium and low priority chunks were scheduled to be evaluated later.

This allowed for the main thread to be freed much earlier and, by shipping a more meaningful experience upfront, we avoided unnecessary loading times.

Hopefully you’ve found this approach useful. For code samples you can take a look at this example repository for some inspiration for your own pipeline. Have you already integrated Lighthouse in your pull-request workflow in a different way? Let me know in the comments!

Want to know more about prefetching and other performance optimization techniques? This is the first chapter of a series where we will go in depth on performance techniques and governance. Till next time!