So for sure the ergonomics of React are excellent. If you have a lot of subtle DOM changes, it’s extremely helpful that it will catch them without you having to worry. In this case (and I wonder how many others) there’s not much to capture really, so it’s a hammer to crack a nut.

Results, please!

I ran the tests on both my MacBook Pro and my Nexus 5, using Chrome stable (currently 43). Both devices are at the upper end of the capability range, so it’s likely that the trend and not the actual values that apply more broadly.

I used window.performance.now() either side of the JavaScript of note (ignoring the network requests, for example) to capture the amount of time spent processing the DOM changes, and I also set a requestAnimationFrame so I could measure to see how long it took for the next frame to land. It’s only rough, but by marking the start of the next frame, we get a sense of how long it took for the page to render (styles, layout, paint, etc) once React or my Vanilla version finished.

Desktop: React

Here I’m using React to add 100 Flickr photos at a time (~5 DOM elements per photo factoring in the name, link and so on), starting at 100 and heading up to 1,200.

What surprised me was that the time taken for 100 pictures (around 500 elements) was 60ms on my MacBook Pro.

There’s a fairly flat graph here. I know from running this test a bunch that there is a slow trend upwards, but you have to get to some seriously large DOM sizes for it to matter. That said, I do have a pretty hefty MacBook Pro, so the CPU is probably churning through at a monster rate.

What did surprise me, though, was that the time taken to compare the DOM with 100 pictures (around 500 elements) was 60ms on my MacBook Pro. Even running the test a bunch of times, the best I could get was around 45ms. That’s locking up the main thread, so if I have any work to do at all when React is comparing trees, I’m all out of luck.

Other thing of note: the JavaScript time is going up, but the time to actually manipulate the DOM (Total time minus JavaScript time) remains steady at ~10ms, which implies that we’re not doing much to trouble Blink. That’s as it should be, since React is supposed to keep my DOM changes to a minimum.

Mobile: React

Now things get interesting. Let’s repeat the React test, but this time on a mobile device: my Nexus 5.

The user ends up with a completely non-responsive UI.

This chart worries me a great deal.

The DOM manipulation time is again pretty steady (since the red line tracks the blue line pretty closely), but the base value is ~500ms for 100 pictures, and it goes up and up and up. By the time you’re diffing a tree with 1,200 Flickr photos (a good amount of DOM at 6,000 elements) the main thread has locked up to the tune of around 1.5 seconds. The net result: the user ends up with a completely non-responsive UI. Sure, I got good ergonomics building the thing, but the cost to my users would be way too high.

Desktop: Vanilla

Now let’s have a look at the Vanilla approach. Here I’m using my own JavaScript to create a fragment for the new images, the DOM for the new images, and to skip images that already exist.

The workload here remains pretty steady. Depending on how you look at it, it’s somewhere between 1/2 to 1/3 the cost of the same thing in React. It seems the majority of the JavaScript time is going into creating the elements and parsing any HTML generated. Since each batch of photos is the same size, the CPU load for each is roughly the same. There are no trees to compare, like we see in React.

As above, the DOM manipulation time remains relatively steady throughout, at around ~2ms. It’s probably so light because my MacBook Pro has enough CPU to throw at the problem.

I have no idea what’s happened at the end. Either V8 optimized the JS, or there was some kind of reporting error.

Mobile: Vanilla

Since it’s getting faster with each run I can only conclude that V8 is being a smartypants and optimizing things on the go.

This chart is a bit of a bamboozler, to be honest. Since it’s getting faster with each run I can only conclude that V8 is being a smartypants and optimizing things on the go, ruining a perfectly good test. It’s interesting (and good!) that it seems able to do that with my plain old Vanilla JS, but it doesn’t seem to get as far with React. That makes sense since my code is super simple, and React, very necessarily, is not.

However, we can still see that the workload is in the 80-335ms range, which is way higher than I would personally like for work of this kind. I reckon you could work around it somewhat by attaching fewer elements on each pass, and keeping the user updated on your progress. With the Vanilla approach that’s perfectly plausible.

Bonus round: Vanilla vs React

Finally, I want to put up the numbers (just the JavaScript ones) for React alongside the Vanilla code on the Nexus 5:

For mobile there’s a remarkable cost to using React over not doing so, and the cost is so high as to be reasonably prohibitive.

This, at least as far as this test goes, is pretty much the summary. For mobile there’s a remarkable cost to using React over not doing so, and the cost is so high as to be reasonably prohibitive.

Conclusions

Was this a fair test? I imagine some folks will feel that it isn’t. I can see that: I’ve only tested how well React copes with increasing tree sizes. At the lower end of the test, however, there were only ~500 DOM elements in the component, which shouldn’t be considered preposterously large. One of the claims of React is that it’s the DOM that is slow, not JavaScript, so perhaps this test is fair after all. This kind of component is very common, and I doubt most development teams would say “we’re going to use React for everything apart from these components here”, but maybe I’m wrong.

Here are my final thoughts:

React has lovely ergonomics. Writing JSX and React was a lot of fun, and it’s always lovely when you don’t have to think too deeply about the edge cases of your code. In that respect, React is brilliant: it’s going to catch minor changes all over your tree that you could well have missed.

Writing JSX and React was a lot of fun, and it’s always lovely when you don’t have to think too deeply about the edge cases of your code. In that respect, React is brilliant: it’s going to catch minor changes all over your tree that you could well have missed. React has significant costs, especially on mobile. React has a lot of computational work required to do all of its checks. On mobile the cost is far higher than I think is reasonable.

React has a lot of computational work required to do all of its checks. On mobile the cost is far higher than I think is reasonable. Always, always, always test. For your app React may well be an epic win. I know there are many who love it, and I can see why. Just be sure to profile (if you don’t know how, I have a free Udacity course you could take) to check the impact on your project! It could be that you have the pathologically bad case which is ill-suited the the tool you’re using.

It seems to me that developer ergonomics should be less important than our users’ needs.

What’s really at stake here, to my mind at least, are the performance benefits (read: user experience) and the developer ergonomics. React is very pleasant for developers to use, but at what cost to the user? It seems to me that developer ergonomics should be less important than our users’ needs, as painful as that can be for us developers. Despite the claims, React does seem to have significant performance implications, at least under certain circumstances.

I really enjoyed using React, but I wouldn’t personally use it on an app I’m building; I just don’t think it would be fast enough.

Other bits and pieces

Thanks to Pascal Hartig for reviewing my React code and pointing out where I was being an utter noob, to Jake Archibald for providing Flickr API code, and to Addy Osmani for proof-reading.

Some questions you may or may not have: