Performance section update - Nov 19, 2015

(TL;DR)

We can finally play HLS using only javascript. It works faster than Flash, unlocking the potential to stream live in 4k and 60fps.

What is HLS?

HLS, which stands for HTTP Live Streaming, is a video format invented by Apple for live streaming. It’s file based, meaning you can serve it over an HTTP connection on the web.

In its simplest form, you have a manifest file, a.k.a playlist, that is the ‘main’ file for a stream. This file contains metadata about the stream and a list of URIs for the video segments of the stream. When a player requests this playlist, a server responds with an updated manifest file that specifies segments that have recently been added to the stream.

This file format has many advantages over alternatives like RTP or RTMP in terms of device support, ease of deployment, and scalability. As a result, it has become a mainstream technology for live streaming, and is widely used for VoD as well.

HLS On All Platforms

If you’re familiar with HLS you might be thinking, “isn’t HLS already supported by HTML?” The answer: sort of! When Apple invented HLS, they also implemented a ‘player’ mechanism inside of their browser. This meant that in Safari you could add a line of HTML to the page, for example, <video src=’path/to/playlist.m3u8’></video> , and voila, you’d have a working HLS player!

But, for various reasons, other browsers didn’t implement this, meaning that systems needed to be designed to stream differently depending on which device and browser a user was operating on. A stream might use HLS on Safari or iOS and RTMP or HDS on desktop. Eventually Chrome and Firefox implemented an HLS player for their mobile versions, but Chrome, Firefox, and IE still don’t use this technology on their desktop versions. A technology for desktop browsers was needed to stream HLS on all devices. Eventually, a few great libraries were released to solve this problem. Flashls, which is open source, and osmfhls, which is proprietary, are two Flash plugin libraries for players that enable HLS on desktop.

Playing HLS Without Flash

HLS experienced mainstream adoption, but Flash was still a requirement. Flash is heavy, slow, and full of security flaws. Experts have been predicting Flash’s demise for a while now, and an alternative is needed. To add some context, we’ll have to take a quick detour to another video format, or rather a standard, MPEG-DASH. Dash basically takes all of the good stuff from HLS and gets rid of the bad pieces. Like HLS, it’s also file based, however, in order to support this new format in the browser W3C decided to create new HTML5 API’s called Media Source Extensions (MSE) and Encrypted Media Extensions (EME). MSE give developers an API to control the video tag’s video buffer and inject data into it. EME is an API that provides DRM support. As a result, it's become possible to inject data into the browser’s video player and use it to implement HLS.

Technically, this requires HLS playback logic that requests HLS files from a server, transmuxes them into the correct mp4 format in javascript, and injects them into the video tag using MSE. There are a few FOSS libraries that do just that, including: hlsjs, videojs’s videojs-contrib-hls, and hasplayer. Now we have HLS in the desktop browser, with only javascript, no Flash necessary. It’s supported in Chrome, Firefox 42+, IE11+, Edge, and Opera.

Performance

It’s time for comparisons, performance, and analysis. In this section, I will show the results for tests run on videojs5 (with videojs-contrib-hls v1.0), its flash predecessor videojs4, hls.js, and its flash predecessor flashls. I’ll focus on CPU performance testing.

For each player I used Chrome's Timeline pane to track the browser's fps, Profile pane to record how much CPU time was utilized, and task manager to monitor the overall CPU usage.

Updated:

1. Some of the pages tested before had UI events for debugging, new measurements have been conducted, screen shots, numbers have all been updated.

2. Added links to the new (now public) experiment pages.

3. Added measurement of the number of frames dropped after playing the entire video.

Hlsjs

The main thread held 60fps, except for the timestamps around 1,000ms, 11,000ms, 21,000ms (each 10 seconds) where it dropped to 30fps. This can probably be traced to the handling of the segment.

I also ran CPU profiling for 29,958 ms. The total CPU time while profiling was 349ms (1.16% of the experiment length), 83.87% of which was Chrome’s program thread.

Examining the Chrome task manager, CPU usage was mostly 0-2%, with 3% peaks. GPU usage was mostly 2%, with 3% peaks.

The number of video frames dropped after viewing the entire video was 638.

Videojs5

The main thread was a constant 60fps, except for non periodic drops, where the biggest one dropped to 29fps.

A CPU profiling test of 31,231ms showed that the total CPU run time was 817ms (2.6%), of which 76.82% was Chrome’s program thread.

Examining the Chrome task manager, CPU usage was mostly 0-2%, with 3% peaks. GPU usage was mostly 2%, with 3% peaks.

The number of video frames dropped after viewing the entire video was 421.

Videojs4

Here we saw a big impact on fps, which decreased to 6 frames per second. This stalled the main thread for up to 160ms at a time.

CPU profiling for 20,084ms showed that the total CPU run time was 5,385.9ms (26.8%) and only 27.8% was Chrome’s program thread.

Examining the Chrome task manager, CPU usage was 16% on average, with peaks of 24% of CPU (divided between flash - 20% and page - 4%)

Chromeless Flashls

The main thread held 60 fps, with minor drops to 54fps.

CPU profiling of 30,105ms showed that CPU was active for 1,520ms (5%), 97.41% of which was Chrome’s program thread.

Examining the Chrome task manager, the page's CPU usage was mostly 1-2%, with peaks of 3%. The GPU was mostly 2% with peaks of 3%. Additionally the flash plugin consumed 2% at most times and 3% at peaks.

The number of video frames dropped after viewing the entire video was 1668.

Conclusion

Both videojs5 and hlsjs performed very well. They easily played 30fps HD videos without setting back the main thread. Both libraries dropped on average less than 1 frame per second in the 60fps HD video, with videojs5 lost a bit less than hlsjs.

Both javascript libraries performed better than flash in overall cpu+gpu consumption and in number of frames dropped. Flashls performed slightly better in keeping Chrome's main thread clean - which is important only when users are still interactive with your site while viewing the video.

Ranking the libraries cpu performance, I would suggest:

videojs5 hlsjs flashls videojs4

The comparison isn’t shown here, but both javascript players started faster than their Flash counterparts did due to Flash’s asynchronous load requirements for the .swf file and the crossdomain file.

It's important to mention that the libraries used are still very young and videojs still hasn’t released javascript HLS support. We will likely see improvements in all of the libraries. These innovations, as well as others, will likely drive higher quality video and create better viewing experiences on the web.

Peer5 is a peer-to-peer CDN that improves content delivery for live and on-demand video streams. Want to try Flashless HLS? Sign up for your free trial here.

After registering, select JW7 and check the "Play HLS in Javascript" checkbox.

Want to learn how to set up your HLS streams to play without Flash? Click here for Peer5's guide to flashless playback.

Notes