Wow! Big improvement with a hardware timer. We get a much more uniform, and therefore smoother, timestamp to key animations off of. So that addresses problem (a), noisy timers in software versus hardware.

With part (b), software timers can be scheduled at bad times relative to vsync. Regardless of what the software does, the hardware display will refresh on its own clock. If our rendering pipeline finishes producing a frame before the next vsync, the display is updated with new content. If we fail to finish producing a frame before the next vsync, the previous frame will be displayed, causing jankiness. Some rendering functions can occur close to vsync and overflow until the next interval. Thus, we actually introduce more potential latency since the frame won’t be displayed on the screen anyway until the next vsync. Let’s look at this in graphic form:

At time 0, we start producing frames. For example, let’s say all frames take a constant time of 10 ms. Our frame budget is 16.6 ms because we only have to finish producing a frame before the next hardware vsync occurs. Since frame 1 is finished 6 ms before the next vsync (time t=16 ms), everything is successful and life is good. The frame is produced in time and the hardware display will be refreshed with the updated content.

Now let’s look at Frame 2. Since software timers are noisy, we start producing a frame 9 ms from the next vsync (time t=32). Since our frame takes 10 ms to produce, we actually finish producing this frame at 1 ms AFTER the next vsync. That means at vsync number 2 (t=32), there is no new frame to display, so the display still shows the previous frame. In addition, the frame just produced won’t be shown until vsync 3 (t=48), because that’s when the hardware updates itself. This creates jank since now the display will have skipped one frame and will try to catch up in the upcoming frames. This also produces one extra frame of latency, which is terrible for games.

Vsync addresses both of these problems since we get a much more uniform timer and the maximum amount of frame budget time to produce a new frame. Now that we know what vsync is, we can finally go on to what Project Silk is and how it helps create smooth experiences in Firefox.

The Rendering Pipeline

In super simplified terms, Gecko’s rendering pipeline does three things:

Paint / draw the new frame on the main thread. Send the updated content to the Compositor via a LayerTransaction. Composite the new content.

In an ideal world, we’d be able to do all three steps within 16.6 ms, but that’s not the case most of the time. Both steps (1) and (3) occur on independent software timers. Thus, there is no real synchronizing clock between the three steps, they are all ad hoc. They also have no relation to vsync, so the timing of the pipeline isn’t related to when the display actually updates the screen with content. With Silk, we replace both independent software timers with the hardware vsync timer. For our purposes, (2) doesn’t really affect the outcome, but is presented here for completeness.

Align Painting with Hardware Vsync

Aligning the timer used to tick the refresh driver with vsync creates smoothness in a couple of ways. First, many animations are still done on the main thread, which means any animation using timestamps to set the position of an animation should be smoother. This includes requestAnimationFrame animations! The other nice thing is that we now have a very strict ordering of when rendering is kicked off. Instead of (1) and (3), which occur at separate synched offsets, we start rendering at a specific time.

Resample Touch Input Events Based on Vsync

With Silk, we can enable touch resampling, which improves smoothness while tracking your finger. Since I’ve already blogged about touch resampling quite a bit, I’ll keep this short. With Silk, we can finally enable it!

Align Composites with Hardware Vsync

Finally, the last part of Silk is about aligning composites with hardware vsync. Compositing takes all the painted content and merges it together to create the single image you see on the display. With Silk, all composites start right after a hardware vsync occurs. This has actually produced a rather nice side benefit — the reduced composite times seen here: