Microsoft — Visual Studio Code

Last week, a new version (1.17) of Visual Studio Code was released. While there are many fascinating improvements and features introduced, the one that caught my eyes is “Integrated Terminal performance” section. Let’s check out what they’ve done!

(written on 2017–10–19, based on Xterm.js v3, VS Code 1.17)

Old Performance Issue

The integrated terminal in VS Code is powered by another popular open-source project “Xterm.js”, which is currently maintained by SourceLair and community contributors. There are many real-world use cases of Xterm.js, such as RStudio and JupyterLab (Ah ha! Data scientist!), not to mention SourceLair itself. One can say that Xterm.js dominates web-based terminal world as React does in front-end development.

As same as other web-based HTML terminal emulators, old versions of Xterm.js are powered by old fashion. Rendering from HTMLElement s, selecting text via document.getSelection , receiving DOM MouseEvent s. All these works were done by a sequence of DOM manipulations.

Development in modern web technologies lowers the barriers for other devs to contribute to. Every programmer can easily pick up a JavaScript cookbook to study, then boasts about new frameworks on Github trending as he is a JavaScript expert. However, web techs also brings another significant drawback: performance.

As we mention to web performance, two key concepts should be kept in mind: reflow and repaint.

Image from MDN. We focus on layout and paint.

Reflow

When a web page is initialized, the render engine calculates dimensions and position of all elements to layout them in proper place. This is known as layout. Although layout is a normal stage of browser rendering process, any updates of position, dimensions or other properties on a specific element after initialization will lead to an additional layout on that element, and all its child nodes need to do a extra layout, too (except nodes with absolute position). These synchronously layout calculations are called reflows, which might result in performance bottlenecks.

Here is a list of what forces layout / reflow. Read it to avoid reflows.

Repaint

A repaint occurs when changes are made to an element’s visibility, background color, or other styles not relevant to layout. Repaints are less expensive than reflows, but also have some impact on browser responsive time.

DOM manipulations

Poor performance is not web techs or JavaScript’s faults. It always counts on implementation details. Hyper and Upterm are two delightful terminal emulators based on Electron. They add many convenient features to improve our boring daily terminal lives. The pain points are their performance issues due to DOM rendering. Both rendering implementations depend on heavy DOM manipulations.

Imagine you are running a yes -like command that writes output at full 10.2GiB/s speed. You will find that your web-based terminal emulator stuck and cannot response to any mouse event anymore. The high speed yes triggers a tremendous amount of synchronous reflows and repaints. Soon, the browser becomes irresponsive.

Optimization in Dark DOM Era

Back to the original version before refactoring to canvas renderer (which started at c6d4c73c). VS Code used naïve way to render terminal output. We can simplify the old rendering process into following steps:

Setup initial row elements corresponding to each row in terminal. That is to say, if size of the terminal is 80 * 24. It would creates 24 <div> representing each row. Loops lines from current terminal buffer. Each line is an array of character data. A character data stores its own width and characters information. Loop character data in each line. Wrap each characters as innerHTML in a <span> element. Remember that we need to escape special character such like < , > and & . Append <span> s to each row element. Then append each row element to terminal element. If anything changes, repeat previous steps.

What a nightmare!

If you are an experienced web developer, you may feel uncomfortable with all those implementation. Modifying DOM between each updates is highly expensive and leads to janky behavior. Luckily, talented maintainers of Xterm.js found their own solution to improve performance. They

Despite that these methods made performance much better (at least better than other popular project I’ve tried), the implementation still had some space for enhancements. For instance,

Modification of <span> s in <div> rows still trigger some unnecessary reflows.

s in rows still trigger some unnecessary reflows. The old rendering process will always remove entire line from DOM, and then append new element to DOM, even when only a single character changed.

Though the skip-frame mechanism can free CPU from endless requestAnimationFrame events, the skip-frame itself means dropping frame from 60 FPS animation.

events, the skip-frame itself means dropping frame from 60 FPS animation. Most characters used in terminal can represent in ASCII code, but a browser always use UTF-16 DOMString to store the information, which may be seen as a waste of memory usage.

Simple Intro of Canvas

As the recent blog post said, the new canvas-based rendering engine renders 5 to 45 times faster, and reduces input latency, power usage and many more. Sounds perfect! However, not all front-end developers are familiar with canvas API. Here are some basic concepts of canvas API you need to know.

The <canvas> is an HTML element providing API for rendering graphics via scripting (JavaScript) on the fly. Modern browsers would also enable hardware-acceleration of canvas rendering by default. You can think of canvas API as an optimized version of browser's repaint powered by GPU.

API Provider

To render a 2D canvas, one developer just need to follow simple steps shown below:

Create a rendering context (by creating a <canvas> element). Access contexts (use canvas.getContext('2d') for 2D rendering context). Start drawing via various commands. All drawing results are rendered onto canvas element as an composited image.

As you can see, learning 2D canvas rendering needs only a small amount (around 70 API for 2D context) of human memory. Most of these drawing API are style-related. As a web developer, you must have seen more CSS styles than these tiny group of drawing commands. Memorizing 70 additional methods is just a piece of cake, huh?

Resource Saver

Another selling points is that 2D Canvas rendering provides a plenty of pixel-awared drawing commands. Instead of updating the whole canvas element, these drawing methods let you decide your region of interest to be re-render in per-pixel level. Almost all drawing API provide optional coordinates and size parameters for devs to tweak what they really desire. The process of updating only changed elements is called invalidation, and that preserves much precious CPU and GPU time.

As opposed to DOM elements, styles in 2D canvas are shared between each path belongs to the canvas. No need to store inline-styling information for every elements. If you want to temporarily store your style state, you can use the standard context.save() and context.restore() methods to push/pop your styles from a stack-like context state container. That's why I call canvas a memory saver!

Performance Booster

As well as saving resources, canvas rendering also gives us ways to jack up performance. One is using detached canvas as an off-screen canvas or using OffScreenCanvas directly. The latter can even draw canvas in worker threads!

Another way is creating a texture atlas. A texture atlas a.k.a. sprite sheet, is an image packed other small pieces of sub-images into itself. When drawing a sub-image, one can picks up the sub-image by its own texture coordinates. The reason to store images in single texture is that GPU is often more performant when accessing a large texture one time than multiple tiny images many times.

Nyancat runs on VS Code without janky!

Canvas to the Rescue

After understand big concepts in 2D canvas. Let’s dive into the pull request that made canvas renderer. First, recap what they’ve done:

Texture atlas (use ImageBitmap ) for ASCII codes and ANSI 256 colors. Unicode characters and true-colored text would be drawn on the fly. Only render changes. To determine state changes, use custom GridCache to store previous state for comparisons with incoming changes. Use four different render layers to separate concerns and reduce the whole canvas re-rendering. Remove skip-frame mechanism because new rendering performance is extremely fast.

Texture Atlas and Color Managements

First, we look into the texture atlas.

Xterm.js constructs a global atlas generator shared between terminals with the same configuration (defined in ICharAtlasConfig and ICharAtlasCacheEntry ). This can reduces some duplicated construction if an app gets a multiple terminal instances such as VS Code. The actual time that a atlas generated is terminal being opened by calling acquireCharAtlas . Internally, acquireCharAtlas would compare between configurations (font size, char width, color, etc.) to avoid overhead works.