We decided to rebuild some parts of the app by using React.

Our developers are already familiar with building React applications (e.g. embedded widgets).

We already have a few React component libraries which can be shared across multiple projects.

The new pages have some interactive UI elements.

There is a huge React ecosystem with lots of tools.

With JavaScript in the browser, it is possible to build a Progressive Web App with lots of nice features.

Pre-rendering and server-side rendering

The issue with client-side rendered applications built, for example, with React Router DOM is still the same as with Ember.js. JavaScript is expensive, and it takes a while to see the First Contentful Paint in the browser.

Once we decided on using React, we started experimenting with other potential rendering options to allow browsers to render the content faster.

Conventional rendering options with React

Gatsby.js allows pre-rendering pages with React and GraphQL. Gatsby.js is a great tool which supports many performance optimizations out of the box. However, using pre-rendering doesn’t work for us since we have a potentially unlimited number of pages with user-generated content.

Next.js is a popular Node.js framework which allows server-side rendering with React. However, Next.js is very opinionated, requires to use its router, CSS solution, and so on. And our existing component libraries were built for browsers and are not compatible with Node.js.

That is why we decided to experiment with some hybrid approaches, which try taking the best from each rendering option.

Runtime pre-rendering

Puppeteer is a Node.js library allows working with a headless Chrome. We wanted to give Puppeteer a try for pre-rendering in runtime. That enables using an interesting hybrid approach: server-side rendering with Puppeteer and client-side rendering with the hydration. Here are some useful tips by Google on how to use a headless browser for server-side rendering.

Puppeteer for runtime pre-rendering a React application

Using this approach has some pros:

Allows SSR, which is good for SEO. Crawlers don’t need to execute JavaScript to be able to see the content.

Allows building a simple browser React application once, and using it both on the server-side and in browsers. Making the browser app faster automatically makes SSR faster, win-win.

Rendering pages with Puppeteer on a server is usually faster than on end-users’ mobile devices (better connection, better hardware).

Hydration allows building rich SPAs with access to the JavaScript browser features.

We don’t need to know about all possible pages in advance in order to pre-render them.

However, we faced a few challenges with this approach:

Throughput is the main issue. Having each request executed in a separate headless browser process uses up a lot of resources. It is possible to use a single headless browser process and run multiple requests in separate tabs. However, using multiple tabs decreases the performance of the whole process.

The architecture of server-side rendering with Puppeteer

Stability. It is challenging to scale up or scale down many headless browsers, keep the processes “warm” and balance the workload. We tried different hosting approaches: from being self-hosted in a Kubernetes cluster to serverless with AWS Lambda and Google Cloud Functions. We noticed that the latter had some performance issues with Puppeteer:

Puppeteer response time on AWS Lambdas and GCP Functions

As we’ve become more familiar with Puppeteer, we’ve iterated our initial approach (read below). We also have some interesting ongoing experiments with rendering PDFs through a headless browser. It is also possible to use Puppeteer for automated end-to-end testing, even without writing any code. It now supports Firefox in addition to Chrome.

Hybrid rendering approach

Using Puppeteer in runtime is quite challenging. That’s why we decided to use it in buildtime with the help of a tool which could return an actual user-generated content in runtime from the server-side. Something which is more stable and has a better throughput than Puppeteer.

We decided to try the Elixir programming language. Elixir looks like Ruby but runs on top of BEAM (Erlang VM), which was created to allow building fault-tolerant and stable systems.

Elixir uses the Actor concurrency model. Each “Actor” (Elixir process) has a tiny memory footprint of about around 1–2KB. That allows for running many thousands of isolated processes concurrently. Phoenix is an Elixir web framework which enables high throughput and allows handling each HTTP request in a separate Elixir process.

We combined these approaches by taking the best from each world, which satisfies our needs:

Puppeteer for pre-rendering and Phoenix for server-side rendering

Puppeteer pre-renders React pages the way we want during buildtime and saves them in HTML files (app shell from the PRPL pattern).

We can keep building a simple browser React application and have a fast initial page load without waiting for JavaScript on end-users’ devices.

Our Phoenix application serves these pre-rendered pages and dynamically injects the actual content to the HTML.

That makes the content SEO friendly, allows processing a huge number of various pages on demand and scaling more easily.

Clients receive and start showing the HTML immediately, then hydrate the React DOM state to continue as a regular SPA.

That way, we can build highly interactive applications and have access to the JavaScript browser features.