H.264 support in Firefox

Background on H.264

H.264 is a video codec used for compressing video data for optimized storage and transmission. H.264 is analogous to other video codecs such as VP8, MPEG-2, or Theora.

H.264 video streams can be contained in various containers such as MPEG-4 part 14 (.mp4), MOV, Matroska (.mkv), 3GP (.3gp), or Flash video (.flv). You can find a full list of video format support on Wikipedia. The container will often contain separate streams for audio and video.  Example codecs used for audio stream include MPEG-1 Layer 3 (MP3), Advanced Audio Coding (AAC), Vorbis, Opus, Windows Media Audio (WMA), and Free Lossless Audio Codec (FLAC).

For the Web, we're looking to support H.264 video streams paired with AAC or MP3 audio streams in the MPEG-4 part 14 (.mp4) container.

Through this article we will use an Open Source clip of Big Buck Bunny as an example, licensed under the Creative Commons License Attribution 3.0 and pre-encoded in H.264.

Running the Big Buck Bunny video through ffmpeg, an open source transcoding utility, we can see more info about the codecs contained in the video:

we can see that Stream #0:0 is audio encoded with AAC, and Stream #0:1 is video encoded with H.264.

Once ffmpeg is installed, transcoding the H.264 video to vp8 is trivial:

ffmpeg -i bunny.mp4 bunny.webm

Playing the video inside the HTML5 <video> element

The HTML <video> element can be used to embed video directly into an HTML page. There are numerous attributes, but for quick testing a src to a relative or absolute URI, and controls for pausing playback are enough. Multiple <source> tags may be nested within the <video> element to support multiple encodings of the same video across various browsers. Fallback content may also be nested inside the element, providing alternatives for the user when their browser does not support <video>. This can range from a simple text node to a Flash Player fallback.

For testing, we added this simple code fragment to a basic HTML document:

<video controls src="bunny.mp4"></video>

<source> tags can be nested within <video> tags to provide multiple encodings of the same file; in such a case the browser will select the first supported codec, based on MIME type. For example:

<video controls>
    <source src="bunny.mp4" type="video/mp4"/>
    <source src="bunny.webm" type="video/webm"/>
</video>

Playing Video in the Firefox OS Video App (MozActivities)

MozActivities is an addition to Firefox OS that facilitates cross application communication. MozActivities can be initiated by apps of any privilege or even web pages in the browser application. An example MozActivity is is the view activity, which can be used to open URIs in the Browser app, new emails in the Email app, PDF's in pdf.js, or video in the Video app. MozActivities are created programmatically via JavaScript. The view activity for videos must be passed a string that references an absolute path to the video's source. When the video completes playback, the onsuccess callback should fire, barring any errors.

An example script follows:

var url = location.protocol + "//" + location.host + "/bunny.mp4";
var activity = new MozActivity({
    name: "view",
    data: {
                type: ["video/mp4"],
                url: url,
    }
});
activity.onsuccess = function () {
    alert("success");
};
activity.onerror = function () {
    alert("error");
};

When dealing with multiple video formats, you can include multiple video format types in the type list to permit those types to play as well:

data: {
    type: [
      "video/webm",
      "video/mp4",
      "video/3gpp",
      "video/youtube"
    ],
    url: url
}

Because of the need for an absolute path for the view activity, however, packaged apps can't make use of this: they are not going to be able to pass their location information since they use the app:// protocol. Apps are not allowed to load resources from other apps due to security concerns. This is only a problem for packaged apps that try to play video packaged internally with the app. This method works for all packaging models of all devices on all versions of Firefox OS otherwise (but not Firefox Desktop, where MozActivity is not exposed). Though referencing an app:// protocol based URI is prohibited, you can still pass a string referencing an asset hosted somewhere (i.e., external to a packaged app).

Another possibility is using the open web activity instead of the view activity. The process would be to load the asset via XHR as a Blob object and try to play it. Such code might look like this:

function open () {
    var xhr = new XMLHttpRequest();
    xhr.open("GET", "bunny.mp4");
    xhr.responseType = "blob";
    xhr.onload = function () {
        var activity = new MozActivity({
            name: "open",
            data: {
                type: "video/mp4",
                blob: this.response,
            },
        });
        activity.onsuccess = function () { alert("success"); };
        activity.onerror = function () { alert(this.error.name); };
    };
    xhr.send();
};

this.response (xhr.response) should be an instance of Blob with the type attribute set to video/mp4. This also works, but does not provide immediate feedback when the XHR is triggered. It's up to you as the developer to provide UI/X feedback when the XHR is initiated, since the entire asset is downloaded before the MozActivity can be constructed (via the XHR's onload function).

Detecting Playback (HTMLMediaElement.prototype.canPlayType)

One of the main benefits of the web is portability. When new cutting edge features start to become implemented in one browser before the rest, we should always try to advocate feature detection over user agent sniffing because user agent sniffing is brittle and frequently leaves out new user agents. Feature detection of codec support should be done with HTMLMediaElement.prototype.canPlayType. This function is callable directly on any instance of an HTMLVideoElement. An example function that should be used to detect whether the current browser supports H.264 is:

function canPlayH264 () {
    var v = document.createElement('video');
    return !!(v.canPlayType && v.canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"').replace(/no/, ''));
};

This function is taken from Dive into HTML5, and returns a Boolean that can (and should) be used for feature detection.

Firefox support summary

The following table indicates support for H.264 in Firefox and Firefox OS.

Firefox OS H.264 support

In Firefox OS, support for H.264 is an interesting story. At the time 1.0 was released, we were unsure whether we'd want to support H.264. Numerous controversies exist around the codec. H.264 is not supported in web pages in FxOS v1.0, but this decision was reversed for 1.1. Unfortunately, feature detection was broken in 1.0, as canPlayH264 function returned true, giving a false positive! Luckily, the view web activity works for H.264 in 1.0+, as explained above.

Recommended approach

Our first recommendation would be to provide multiple encodings of your data using <source> tags nested in a <video> tag. Transcoding is trivial (once you have the tools set up), and provides the portability to play such media on multiple platforms. It can also be beneficial to have different resolutions that are optimized for mobile. Of course, there are diminishing returns for supporting multiple encodings, so it's up to you to draw the line somewhere.

If you truly have exhausted the ability to support multiple encodings, then in order to still support Firefox OS 1.0, we would recommend the following shim. It will detect if H.264 is playable, then if the platform is Firefox OS v1.0, we can't play the content inline so we'll use a view web activity. The shim needs customization, so read the comments before dropping it into your existing code base.

;(function () {
    // Can I play H.264?
    var v = document.createElement('video');
    var canPlayH264 = !!(v.canPlayType && v.canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"').replace(/no/, ''));
    if (canPlayH264) {
        // Am I lying? (it makes my skin crawl to recommend this, let alone write it)
        // In general, never use UA sniffing (unless feature detection is otherwise impossible)
        if (navigator.userAgent === 'Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0') {
            // Guilty
            Array.prototype.forEach.call(document.getElementsByTagName('video'), function (video) {
                video.addEventListener('click', function (event) {
                    // Don't try to play video
                    event.preventDefault();
                    var activity = new MozActivity({
                        name: "view",
                        data: {
                            // change if video is not mp4
                            type: ["video/mp4"],
                            // replace with absolute path for packaged apps
                            url: video.src,
                        }
                    });
                    // Remove these two functions if not needed
                    activity.onsuccess = function () {
                        alert("success");
                    };
                    activity.onerror = function () {
                        alert("error");
                    };
                });
            });
        } // else I'm not lying, I really can play H.264!
    } // else we can't play H.264 anyways
})();

Key points

• HTMLMediaElement.prototype.canPlayType for H.264 reports true incorrectly in Firefox OS 1.0.
• view web activities require absolute urls (which is a problem for packaged apps).
• open web activities require the entire asset be downloaded, then passed as a blob (has UI/X concerns).
• open web activities are not handled in the Gaia Video App in Firefox OS 1.0.
• Use <source> tags or if (canPlayH264 && isFxOS1dot0) returns true use view web activity.

See also

• MDN <video> reference
• Using HTML5 audio and video