HTML5 Video Element

Exporting Video Frames

It is impossible to imagine Web 2.0 without support for multimedia. Playing audio and video files natively is one of the most important features of modern browsers. Combining video playback with other client-side techniques paves the way for non-trivial use of ordinary markup elements. One of possible scenarios is the export of video frames and their rendering in the background of a canvas or any other HTML element. A video frame is a single segment of video data: it is a snapshot of a movie taken at a point of time. The movie itself is a sequence of frames played at a certain rate.

Examples in our article will be based on a video file displaying the launching of the famous Apollo 11:

The first example deals with the export of video frames to the canvas:

<video autoplay controls loop src="apollo11.webm" onplay="exportFrames(this)"></video>
<canvas id="video-canvas" width=321 height=240></canvas>

The video element loads a WebM file and begins playback automatically. The boolean controls attribute makes the browser display UI controls for media management (play/pause button, timeline bar, volume control, etc). The track is played infinitely: the loop attribute returns the video element to the start of the media resource each time the end is reached. When playback begins, the play event is fired. Its handler is a named function taking the video element as its only argument.

There's a convenience method in the canvas 2D context API which allows a Web author to use a <video> element for drawing images on the canvas plane. The basic signature of the drawImage() presumes the existence of three arguments - an HTMLVideoElement instance and the coordinates of the upper left point of the image.

var context=document.getElementById("video-canvas").getContext("2d");
function exportFrames(video) {
 setInterval(
  function() { // timer-based callback
   context.drawImage(video, 0, 0);
  },
  42 // a frame will be drawn every 42 milliseconds
 );
}

The setInterval() method schedules a function to run at a certain interval. Images in our example will be captured from the video every 42 milliseconds. Computing intervals for different tracks is based on the video frames rate. The frames can be blended and composited with the canvas backdrop to create fancy effects for the video.

Another way of sending video frames to other HTML elements is the use of the element() function. It does not require scripting: all work is done through CSS. The function is still considered experimental, but can be employed with a vendor prefix:

<style type="text/css">
 div {
  width: 321px; height: 240px;
  background-image: -moz-element(#apollo11);
  background-repeat: no-repeat;
 }
</style>
. . .
<video id="apollo11" autoplay loop src="apollo11.webm"></video>
<div></div>

The element() function enables the use of an HTML element in the document as an image. The function accepts an id of the referenced element. This time we do not set controls attribute of the <video>: otherwise UI controls would be reflected in the background of the <div>.

WebKit-based browsers have their own distinctive function pertaining to canvas. This is the -webkit-canvas(). A developer using the function can save himself the trouble of declaring a canvas in the markup: CSS and JavaScript are enough to apply canvas graphics to any other HTML element. Our third example will export video frames to the dynamic canvas which, in its turn, is a background image of the <div> element:

<style type="text/css">
 div {
  width: 321px; height: 240px;
  background-image: -webkit-canvas(apollo11);
  background-repeat: no-repeat;
 }
</style>
. . .
<video autoplay controls loop src="apollo11.webm" onplay="showVideo(this)"></video>
<div></div>

In the background-image: -webkit-canvas(apollo11) CSS rule the apollo11 is an arbitrarily chosen name of the canvas. The name will be used in the script to create the canvas programmatically. To gain this purpose, the getCSSCanvasContext() method is called. It takes four arguments: the context (2d), the canvas name specified in CSS (apollo11) and the canvas plane dimensions. When playback starts, a frame is sent to the canvas every 42 milliseconds. As the canvas is a background image of the <div> element, the <div> background changes simultaneously:

var context=document.getCSSCanvasContext("2d", "apollo11", 321, 240);
function showVideo(video) {
 setInterval (
  function() { // anonymous callback to draw frames on the canvas plane
   context.drawImage(video, 0, 0);
  },
  42 // interval
 );
}