The choice for Theora

As we’ve begun to publish our videos of the Fronteers 2009 conference (all of which will be linked from the sessions page), our choice to use the <video> element and the Theora codec has not been without discussion...

At the moment, roughly 65% of the visitors to the Fronteers website use a browser which supports the <video> element in combination with the Theora codec. Specifically, this is Firefox 3.5/3.6 (and other Gecko 1.9.1+ browsers like SeaMonkey 2), Chrome 3/4, and (pre-)alpha builds of Opera 10.50. The (still) 20% IE users won't see video in their browser, but can download the videos, and play them with their media player of choice. (In the unlikely case that this player doesn't support Theora, VLC is a good and open source alternative.) And then there's roughly 10% of Safari users, who will see the poster frame of the video, but won't be able to actually play the video, due to the lack of support in Safari for the Theora codec. As with the IE users, these visitors will have to download the videos in order to be able to watch them. There's been heavy discussion in the Fronteers IRC channel if this was the right choice, or if we should additionally make available a version of the videos using the H.264 codec, which is supported by Safari. Remarkably enough, it took two days before anyone even suggested to use Flash to make the videos play in IE.

The core question is: What's more important? Giving the best experience to the largest number of visitors, or joining the fight for a world in which every user of the internet can watch and create videos without any barriers. Fronteers is increasingly being looked at to provide best practices, and one of our goals as an organization (as decided by the members) is to "use our influence to make standards organizations and implementing parties improve the lot of front-end developers through better standards, techniques, documentation and implementations." On the other hand, can we still be taken seriously as front-end developers if we don't choose the solution which right now works best for the largest number of visitors?

For those who aren't aware of the situation around video codecs, a short summary. HTML5 specifies the <video> element. Originally, this also included specifying the codec to use in combination with this element, namely Theora. Apple, one of the five big browser vendors, was strongly opposed to this choice, due to fears for submarine patents. The risk of such patents always exists, of course, but they'd already taken this risk with other codecs (for example via iTunes), and doing so again with this new codec was apparently out of the question. H.264 as an alternative was unacceptable for open source browser vendors (specifically Mozilla, but also relevant for webkit-based browsers like Chromium, Konqueror, etc.), since there are many software patents which apply to H.264. Although an organization like Mozilla could theoretically bear the costs of a license ($5,000,000 for the current year, roughly 6% of their yearly turnover, with further price increases all but certain), downstream users of the Mozilla source code wouldn't be able to use H.264 themselves. From the open source perspective, this is unacceptable. (Using codecs installed in the OS is no solution either for Mozilla; doing so would incur large security risks, and goes against Mozilla's mission for an open web.) Opera is also opposed to the H.264 codec, mostly (?) due to the high licensing costs. Due to this stalemate, and to better reflect reality, the specification of a codec for the <video> element has been removed from HTML5, at least for the duration of the search for a solution which is acceptable to everyone.

Lacking a mandate from higher-up, the fight for a universally used codec for the web has been moved to implementations and users. H.264 is supported by the big media companies and parties aligned with them. H.264 has a first mover advantage, and hardware support. Opposed to it we find Theora, supported by the open source software world, and all small parties who want to be able to put video on the web by themselves. H.264 is currently free to use for creating video at a small scale, but due to the licensing structure which changes yearly, there's no certainty that this situation will last very long. History teaches us, and all the communication by the MPEG-LA seems to confirm, that this will rapidly change if H.264 becomes the most used video codec on the web.

The situation right now is in flux; everything is possible, and small actions can have large consequences. Theora has, due to support in Firefox and Chrome, by far the highest percentage of direct support in a browser. Mozilla is additionally using its resources to help improve the quality of this codec even further. (There are many myths about the technical inferiority of the Theora codec, mostly due to old tests from back when the bitstream codec had just been frozen, but not much effort had yet gone into the encoder and decoder; more recent tests show that differences in quality are minimal.) On the other hand, YouTube and Vimeo are using H.264 in initial tests with the <video> element. (Mostly because their videos were already encoded in this format?) There are signs that Apple might reconsider Theora, and by buying On2, Google has acquired the progenitor of the Theora codec, which might help with creating more certainty around submarine patents.

The coming year is likely to be crucial. Will we end up in a situation where H.264 is the video standard for the web, and will everyone doing anything with video have to pay money to the MPEG-LA for this? Or will Theora gain enough momentum to entice Apple and the big video sites into supporting it, which will lead us into a world where video on the web becomes free and open, enabling optimal use by everyone?

I know which situation has my preference, and I know which situation is better for us web developers. We web developers additionally play a part in this fight ourselves, and our choices can matter. The more high quality video will be published online in Theora (and not in H.264), the higher the chance that Apple decides it's worth the risk to support Theora in Safari. Although Fronteers is no Wikipedia (which site will host all their videos in Theora), we are beginning to become a sizable organization, with a decent reputation, which we can use to good effect here, helping the fight for an open web.

What is your opinion? Is this a good idea, or should we just be practical and publish H.264 video through a Flash interface? Is it more important to serve 100% of our visitors right now, or is it okay to show some idealism and to say, "We know what's best in the long run!"?

update: As referred to above already, Google bought On2, and thus got their hands on the advanced VP8 codec, a successor to the VP3 codec from which Theora spawned. In May 2010, they made this codec open source as part of the WebM Project, which did not only gain instant support of Mozilla and Opera (with alpha versions of Firefox, Opera and Chrome supporting WebM, and stable versions to be released over the next few months), but also of a host of other software and hardware vendors. Additionally, Google has begun converting every single video on YouTube to WebM, giving a powerful impetus to the growth of the WebM ecosystem.

As of now, Microsoft continues to back H.264 as the primary codec in their browser (with WebM support requiring the codec to be installed separately), while Apple hasn't made an official statement yet. The position of open video on an open web has gotten a lot stronger, but nothing has been decided yet, and the situation remains in flux. As Fronteers, we continue to believe it's in the best interest of the web to only support open standards which can be freely implemented. In addition to making our videos available using the Theora codec, we'll also support WebM, and will continue to recommend not using H.264 for video on the web.

(This is a slightly shortened translation of a blog post originally written in Dutch by me, Sander van Lambalgen. Feel free to leave comments on the original blog post.)