Around a year ago, we started discussing ways to improve performance on Daniel Wellington's e-commerce website. Emerging markets like India require highly optimized web experience that we as web developers need to deliver on. We chose Gatsby as a tool to help us reach optimal performance.

In this post I will take you through our journey with Gatsby at Daniel Wellington, the challenges that we faced when scaling it up and how we overcame them.

A bit of context

The Gatsby platform we are building is planned to support many markets and locales (currently around 40 and counting). Each market has around 1,000 pages, so we're in the ballpark of 10k to 50k pages.

Our migration to Gatsby is also not a work from scratch: we rely on the existing backend structure to provide us with features such as inventory and cart management. The whole payment flow is also not part of this migration.

Optimizing build time

As we started to migrate a few markets to Gatsby, we quickly realized that the build time would easily skyrocket into a range that wouldn't work for our team. Each market was taking around 5 minutes to build, so at 40 markets we would reach hours of build time.

We dug into the issue, and realized that our GraphQL queries were several orders of magnitude slower than the benchmarks (even the complex ones) provided by Gatsby.

One reason for this issue being our need for customization: every page (frontpage, category page or campaign page) can be fully customized, which results in our GraphQL queries being complex – and slow.

We started optimizing them by running them only when needed. For instance, our footer is the same on every single page (for a market), so it's better to run the query for that component in gatsby-node 's createPages and inject the result via the context of the page, rather than having the query for the footer in the page's GraphQL query.

It was a good improvement, but even the resulting 30-minute build time was too slow, because it was bottlenecking our content editors.

Improving the preview experience

On a highly customizable website, previewing the changes you make is crucial. The fastest way to change something on a Gatsby platform is by using what we developers use daily: the development server. This server has a feature that allows you to live reload the data on the development server and everything on the website will be updated automatically. Neat!

We started using this feature at the beginning of the migration, and content managers loved the feature. They just had to click a button to see their changes, all it took was a few seconds.

Unfortunately, that does not scale. The development server is made to be used for development, and we quickly ran into issues. The server would start crashing randomly after a few hundred reloads, and maintaining it was a big pain. It became worse when we reached around 20K pages, at this point the server was just running out of memory at startup time, and could not be accessed anymore.

We decided to solve these 2 major problems by slicing our website into smaller (and faster) websites. Since we have so many markets and locales, and each of these markets are pretty silo-ed (they don't communicate or share data with each other), it made a lot of sense to separate them from each other