In a previous article, I talked about I/O performance. Remember when Nexus.js used to convert 400MB of text from UTF-8 to UTF-16LE in 1.8 seconds and I bragged about it?

That was quite an achievement for me, but then Christian Landgren pointed out that Node.js easily beat that using cluster.fork(), and it finished in 1.178s.

I was quite distraught, until I split the FileSourceDevice implementation into two types of devices:

First, you have the FilePullDevice, which is the old — but renamed — type.

And then I added the all new FilePushDevice.

FilePushDevice relies on events, it inherits EventEmitter and dispatches the ‘data’ event whenever data is available, so let’s try it out:

Madness!

It just converted 400MB of data — outputting 800MB! — in 0.484s! That’s almost a quarter of the time it took before!

This performance is the result of piping the streams in parallel, switching from the iconv library to ICU, and good data flow. I also took the time to tune the buffer sizes, which will be programatically adjustable in a later commit. (Don’t forget to remind me, brain!)