I was just wondering, given that most of the big JavaScript frameworks offer a Command-line interface (CLI) tool nowadays — to automate the creation of new projects and the building of production assets — how do they actually compare to each other? I mean surely they must all be hitting the same ‘ballpark’ when it comes to bundle size/render perf right? Maybe it’s not as close as you might think.

I decided to test this out by installing 6 popular CLI tools — Create React App, Angular CLI, Ember CLI (for both Ember & Glimmer), Vue CLI, Create Inferno App and Create *Preact App globally onto my laptop, and then follow the official documentation for each.

I was only interested in the ‘out of the box’ project generation — so there was literally zero application code added by me. I just ran the relevant command to scaffold a project, then I immediately ran the production build…

I think this is an interesting test, because although each framework can give you more or less features by default, the point here is that the authors of the framework must deem this default scaffold+build process to be ‘the best you’re going to get’ out of the tool, and that’s what I found fascinating in the results.

So let’s start in this first post by looking at just two easy metrics, JavaScript bundle size & JS first render time.

Note: see https://github.com/shakyShane/arewereadyformobileyet for the automated results.

JavaScript Bundle Size

For each CLI, I ran the ‘as-documented’ command for producing a production-ready build. T̶h̶e̶n̶ ̶I̶ ̶m̶a̶n̶u̶a̶l̶l̶y̶ ̶g̶z̶i̶p̶p̶e̶d̶ ̶t̶h̶e̶ ̶o̶u̶t̶p̶u̶t̶ ̶(̶u̶s̶i̶n̶g̶ ̶d̶e̶f̶a̶u̶l̶t̶ ̶m̶a̶c̶O̶S̶ ̶g̶z̶i̶p̶ ̶c̶o̶m̶p̶r̶e̶s̶s̶i̶o̶n̶ ̶s̶e̶t̶t̶i̶n̶g̶s̶)̶ ̶t̶o̶ ̶g̶e̶t̶ ̶t̶h̶e̶s̶e̶ ̶r̶e̶s̶u̶l̶t̶s̶.̶

Then, I spin up a lightweight server pointing at the resulting ‘build’ or ‘dist’ directory with Gzip enabled. Finally I pump the URL from this server into Lighthouse & measure the size of all the Javascript combined.

Total amountJavascript in KB (as served by the browser)

Notes

Preact (and other micro libs, didn’t want to clutter the chart) obviously come out on top given they are simply a thin layer on top of the DOM. You’ll naturally get more features out of something like Angular & Ember, given they are full frameworks — but I’ve left a micro lib in here because with PWAs becoming so popular/necessary, it’s often the case that Apps are built by composing many of these micro libs together and given the starting point is 8kb, you’d need to add a LOT of them before your code is anywhere near the size of what Ember/Angular are shipping by default.

frameworks — but I’ve left a micro lib in here because with PWAs becoming so popular/necessary, it’s often the case that Apps are built by composing many of these micro libs together and given the starting point is 8kb, you’d need to add a LOT of them before your code is anywhere near the size of what Ember/Angular are shipping by default. Interesting how close React & Vue are nowadays — although I believe Vue can be trimmed further depending on which options you select in the CLI scaffold stage (I just went with whatever was presented as a ‘default’)

Angular seems to have shed about 50kb since I last tried a few months ago, so they are certainly making progress;

Ember is off the chart (size-wise) and cannot be considered suitable for mobile at this point (to be fair, I don’t think they claim to be).

Glimmer (by the Ember team) is the newest, and coming in at 34kb is impressive to say the least. Great work.

First JavaScript Render

Next, I throttled the connection speed in the network panel Chrome Dev Tools (down to regular 3G) and applied the 5x CPU slowdown in the timeline section. Then I hit ‘reload’ with screenshots enabled and then scrubbed through until I spotted something appear on screen that was the result of the framework running.

Totally un-sciency way of measuring how quickly frameworks ‘boot’

There are of course many other ways of measuring this more accurately — depending on what type of answer you want. Not to mention the difference that may be made via SSR etc, but still, it’s pretty obvious that more JS === more time. Even if a page were rendered by the server and picked up on the client side by the JS framework — the render times in that chart above would simply become the ‘time to interactive’ as the JS still needs to download/be-parsed/be-executed, whether there’s static HTML there or not. A user ‘seeing something’ on the screen is simply not enough if the core product includes JS interactivity.

Interactive in 2 seconds…

I’ve often heard the mythical goal is ‘interactive in 2s’ (with the 3G throttle and CPU slowdown) and recently at my day-job I set a target on a very large project of having the initial bundle (enough JS for rendering, interactivity and Ajax) to be less than 50kb . Safe to say that hitting this target required me to chose a micro view lib like Preact, which, because of its tiny size, allowed me ‘space’ to bring in a few essential libraries whilst still keeping things below 50kb.

How I came up with the results

Create React App

Commands:

Install: yarn global add create-react-app

Scaffold: create-react-app cra-test

Production Build: yarn run build

JS Bundle Size: 46kb

First render: 770ms

Angular 2 CLI

Note: Why is aot not automatically part of the target=production configuration? It’s extremely easy to miss and I only knew about it because of a Podcast addiction :p

Commands:

Install: yarn global add @angular/cli

Scaffold: ng new ng-test

Production Build: ng build --aot --target=production

JS Bundle Size: 92kb (4 files)

First render: 1500ms

Ember CLI (Ember app)

Note: The Ember scaffold process does NOT render any JS by default, which leads to a false ‘first render’ score — so I followed the instructions and added an outlet as directed. This is the only CLI tool I had to do this for.

Commands:

Install:

- yarn global add ember-cli

- yarn global add bower

Scaffold: ember new ember-test

Production Build: ember build --target=production

JS Bundle Size: 198.4kb (4 files)

First render: 4200ms

Ember CLI (Glimmer app)

Note: There also appears to a --suppress-sizes flag, but when I used that, the JS bundle it produces appeared to be the same size + caused an error in the browser

Commands:

Install: yarn global add ember-cli/ember-cli

Scaffold: ember new glimmer-app -b @glimmer/blueprint

Production Build: ember build --target=production

JS Bundle Size: 34kb

First render: 1000ms

Vue CLI

Note: Vue was by far the most confusing CLI experience — the need to specify a template + lots of questions about tooling — it feels very off-putting to newcomers, where’s the ‘default’?

Commands:

Install: yarn global add vue-cli

Scaffold: vue init webpack vue-cli (followed by lots of questions)

(followed by of questions) Production Build: npm run build

JS Bundle Size: 43.48kb (4 files)

First render: 840ms

Create *Preact App

Note: I’m not entirely sure this is well supported, but it’s the first/only CLI tool I could find for a ‘micro’ lib.

Commands:

Install: yarn global add create-preact-app

Scaffold: create-preact-app preact-test

Production Build: npm run build

JS Bundle Size: 8.8kb

First render: 412ms

Create Inferno App

Commands:

Install: yarn global add create-inferno-app

Scaffold: create-inferno-app inferno-test

Production Build: yarn run build

JS Bundle Size: 70kb

First render: 737ms