A little over a year ago I started the noVNC project, an HTML5 VNC client. noVNC does a lot of processing of binary byte array data and so array performance is a large predictor of overall noVNC performance. I had high hopes that one of the new binary byte array data types accessible to Javascript (in modern browsers) would give noVNC a large performance boost. In this post I describe some of my results from testing these binary byte array types.

After reading the title, you may have thought: "Wait ... Javascript doesn't have binary byte arrays." Actually, not only does Javascript have access to binary byte arrays but there are two unique variants available (technically neither are part of ECMAScript yet).

Jump list:



The Options



Typed Arrays:

Those who follow browser development and HTML standardization may already be aware of one of these array types. ArrayBuffers (technically: Typed Arrays) are a required part of the proposed WebGL and File API standards. To use an ArrayBuffer as a byte array you create a Uint8Array view of the ArrayBuffer.

The following Javascript creates an ArrayBuffer view that contains 1000 unsigned byte elements that are initialized to 0:

var arr = Uint8Array(new ArrayBuffer(1000));



ImageData arrays:

But there is an older and more widely supported form of binary byte arrays available to Javascript programs: ImageData. ImageData is a data type that is defined as part of the 2D context of the Canvas element. ImageData is created whenever the getImageData or createImageData method is invoked on a Canvas 2D context. The "data" attribute of an ImageData object is a byte array that is 4 times larger than the width * height requested (4 bytes of R,G,B,A for each pixel).

The following Javascript creates a ImageData byte array with 1000 unsigned byte elements that are initialized to 0:

var ctx = getElementById('canvas').getContext('2d'), arr = ctx.createImageData(25, 10).data;



Traditional Solutions:

There are two traditional ways of representing binary byte data in Javascript. The first is with a normal Javascript array where every element of array is a number in the range 0 through 255. The second method is using a string in which the values 0 through 255 are stored as Unicode characters in the string and read using the charCodeAt method. For this post I'm going to ignore the string method since Javascript strings are immutable and updating a single character in a Javascript string implies reconstructing the whole string which is both unpleasant and slow.

The following creates a normal Javascript array with 1000 numbers that are initialized to 0:

var arr = [], i; for (i = 0; i < 1000; arr[i++] = 0) {}



The Bad News

We are left with three methods for representing binary byte data: normal Javascript arrays, ImageData arrays, and ArrayBuffer arrays. One might expect that since ImageData and ArrayBuffer arrays are fixed sized, have elements with a fixed type, and are used for performance sensitive operations (2D canvas and WebGL) that the performance of these native byte arrays would be better than normal Javascript arrays for most operations. Unfortunately, as of today, most operations are slower when using these byte array types.



Testing

Originally I planned to show the performance numbers comparing browsers on Linux and Windows. However, I discovered that there is very little difference (for these array tests) between the same version of a browser running on Windows vs Linux. Since all the Linux browsers also run on Windows (but not vice versa) I have limited the performance results to Windows.

For this post I have hacked together four quick tests to compare normal Javascript arrays with ImageData and ArrayBuffer arrays. All the tests use arrays that contain 10240 (10 * 1024) elements and repeat the operation being tested many times in order to push the test times into a more easily measured and comparable range. Each test is also run 10 times (iterations) and the mean and standard deviation across all 10 iterations is calculated.

You can run tests yourself by cloning the noVNC repository and loading the tests/arrays.html page. These test results are based on revision bbee8098 of noVNC. Running the test in a browser will output JSON data in the results textarea. This JSON data can then be combined with JSON data from other browser results and run through the utils/json2graph.py python script which uses the matplotlib module to generate the graphs.

The test machine has the following specifications:

Acer Aspire 5253-BZ893

AMD Dual-Core C50 at 1GHz

3GB DDR3 Memory

AMD Radeon HD 6250

Windows 7

Here are the main browsers that were tested:

In addition, older browser versions were also tested to see if the browsers are making progress:

Chrome 9.0.597.98

Chrome 10.0.648.204

Chrome 11.0.673.0 (build 75038)

IE 9.0 Platform Preview 7

Firefox 3.6.13

Firefox 3.6.16

Firefox 4.0 beta 11

Please note that I am not a professional performance tester so I probably haven't made use of optimal testing techniques and there is certainly a possibility that I have made mistakes that invalidate some or all of the numbers. I welcome constructive criticism and dialog so that I can expand and improve on these results in the future.



The Four Tests:

create - For each test iteration, an array is created and then initialized to zero and this is repeated 2000 times. randomRead - For each test iteration, 5 million reads are issued to pseudo-random locations in an array. sequentialRead - For each test iteration, 5 million reads are issued sequentially to an array. The reads loop around to the beginning of the array when they reach the end of the array. sequentialRead - For each test iteration, 5 million updates are made sequentially to an array. The writes loop around to the beginning of the array when they reach the end of the array.



Test Results:

First let's take a look at how the different array types perform at the different tests.

This is the only test where ImageData and ArrayBuffer arrays have a significant performance advantage because they are automatically initialized to 0 when created. IE 9 and Opera do not currently support ArrayBuffer arrays.

Chrome and Opera have the best overall performance although Opera does not support ArrayBuffer arrays yet. Firefox has particularly bad random read performance across the board. The results show that there is little advantage to using ImageData or ArrayBuffer arrays for random reads and their performance in Chrome and Opera is significantly slower.

For the sequential read test the situation is quite different. Firefox has consistent and leading performance across all array types. Chrome has an order of magnitude worse performance for ImageData and ArrayBuffer arrays. Opera 11 show a 3X fall in performance for ImageData arrays compared to normal Javascript arrays. Normal arrays are still the best choice overall.

The sequential write test relative results are very similar to sequential reads with a slow down across the board. Firefox again shows comparable performance across all three array types. Opera 11 continues to show a 3X fall in performance with ImageData arrays. Chrome continues to show an order of magnitude speed different between normal arrays and the binary arrays.

Now let's slice the data differently to see how the different browsers compare across the different array types.

Chrome is the best overall performer here with Opera pulling a close second. However, the most notable result in this view is the terrible performance of Firefox random reads. Given the huge amount of jitter in the Firefox result compared to the others, my guess is this is a degenerate case and that Mozilla has some low hanging fruit here.

Opera is now the overall performance winner with Chrome pulling a close second. The Firefox problem with random reads continues to show up with ImageData arrays (although this time without the jitter). Excluding the random read result, Firefox would be the clear winner. IE 9 has a good showing here coming in a close third overall.

The pack thins out significantly since only Chrome and Firefox support ArrayBuffers. Once again Firefox shows pessimal random read performance. With that result excluded (or fixed), Firefox would be the clear winner against Chrome.



Test Result Summary:

Chrome has the best overall performance for normal arrays.

Opera has the best overall performance for ImageData arrays with Chrome a close second.

Firefox has good performance except for random reads where performance drops off a cliff on all array types.



Browser Improvements/Regressions

Now we will compare some older browser versions to see if the browser vendors are making progress over time in improving the performance of the binary byte array types.



Firefox



Firefox mostly shows steady improvement for normal arrays, but once again the terrible random read perform rears its head in the shift from 4.0 beta 11 to the 4.0 release version.

Again, Firefox mostly shows steady improvement for ImageData arrays. This time the terrible random read performance was introduced somewhere between the Firefox 3 and Firefox 4 code base.

Only Firefox 4 supports ArrayBuffer arrays. The awful random read performance still exists.



Chrome



No strong trends appear in the Chrome data for normal arrays. The array create speed shows a significant dropoff in Chrome 12. For sequential writes there was a 2X regression for Chrome 10 and 11.

There appears to be a significant regression in Chrome 12 related to ImageData performance. The amount of the dropoff (3X to 6X) and the significant jitter indicate to me that their is a obvious propblem that should be fixed.

There are no strong trends in Chrome ArrayBuffer array performance although there seems to be a weak trend towards worse performance.



IE 9



The final release of IE9 shows a huge performance decrease compared to the Platform Preview 7. If Microsoft is able to recover this performance in a subsequent release then this would significantly change the standing of IE 9 in relation to the other modern browsers.



Final Thoughts

ImageData and ArrayBuffer arrays have different performance characteristics within the same browsers. I'm not sure why this should be the case. In fact, I would recommend that the WHATWG/W3C and browser vendors standardize on ArrayBuffers for both purposes. This could be done by adding an additional attribute to the ImageData object perhaps named 'buffer'. The new 'buffer' attribute would be a generic ArrayBuffer containing the the image data memory. The existing 'data' attribute would become a Uint8Array view of the ArrayBuffer (this would maintain backwards compatibility). In addition to code consolidation within the browsers (and one place to focus optimization effort) this change would allow developers to create a Uint32Array view of the buffer which would allow whole pixel updates (3 colors + alpha) with one operation.

Using ImageData and ArrayBuffer (typed array view) arrays will generally not give better performance for binary byte data than just using normal Javascript arrays. This is unfortunate since these binary array types exist specifically to serve performance sensitive functionality (2D and 3D graphics). It is also surprising since they have a fixed size and a fixed element type which in theory should allow faster read and write access to the elements. I suspect (and hope) that this performance problem is due to the fact that not enough optimization effort has been applied by any of the browser vendors to these binary arrays.



Requests:

Mozilla, Google (and Apple), Microsoft and Opera: please spend some effort to optimize your Javascript binary array types!

Microsoft and Opera: it would be nice if you would implement WebGL. But if not, please at least implement typed array (ArrayBuffer) support since it stands on its own and will likely be used in the near future in other places where it make sense such as FileReader objects and in the WebSocket API to support binary data.



Followup Posts:

The browser wars are back and new browsers versions are being released every few weeks. My plan is to continue updating these tests to include the most recent browser versions. I would also like to expand the tests to include a random write test and also to test the random and sequential read performance of binary data stored in Javascript strings. Stay tuned.