Movie Formats

The Basics
A typical movie file does not only contain video, but contains multiple different streams:


 * 1 Video stream
 * 1 or more Audio streams (e.g. English, German and director commentary)
 * 1 or more Subtitles (e.g. English and Dutch)

A movie file may also contain metadata (title, episode, director, date of recording) and chapters. Not to mention advanced features such as stereo video for stereo television (often improperly called 3D television).

The file that combines (muxes) the different streams and the metadata together is called the movie container.

Your Choices
To create a movie files, you must decide on:


 * The container format
 * The video codec
 * The audio codec
 * The subtitle format

Unfortunately, there is no single solution that works everywhere. Your solution may depend on what is available on your platform (Mac, Windows and Linux support different containers and codes out-of-the-box), the features you require (WebM does not yet support subtitles or multiple audio tracks), if you are willing to pay royalty fees (formats such a H.264 are encumbered by patents), if you need hardware support (H.264 on the iPhone has hardware support and uses much less battery power than any other video format), etc.

Further Information
For more information on the background, you should read the following sites:
 * Video on the Web, by Mark Pilgrim. Specifically aimed at using the HTML5  tag.
 * A gentle introduction to video encoding, also by Mark Pilgrim. Contains a nice list of recent formats, although it is missing the WebM format that surfaced in 2010.
 * Comparison of container formats on good old wikipedia. Not always up-to-date, but a great starting point anyway.

Overview of Common Formats
The following tables are taken from Mark Pilgrims' A gentle introduction to video encoding, with WebM added. (Original sources: DigitalContentProducer.com, doom9.org, HydrogenAudio.org, Wikipedia)

Video Codecs

 * (*) As far as we know. In 2013, Google admitted VP8 infringes on h.264 patents, and has subsequently entered a license agreement with MPEG LA.

Audio Codecs

 * (*) As far as we know

Container Formats

 * (*) Subset of the Matroska container format

The Debate
One of the goals of the HTML5 working group was to define a video format that was supported in all webbrowsers. In the end, this proved unsuccessful, and Ian Hickson had to admit defeat. Basically Apple and Microsoft only support H.264 natively (and not WebM), while Mozilla and later Google only support WebM natively (and not H.264).

Stand-alone Movies
For movies and series use either:
 * The DVD VIDEO_TS format (which retains the original menu structure), but is very large in size.
 * Matroska (MKV) files with VP8 (or optionally H.264) for video codec, Vorbis for audio codec, and VobSubs for subtitles.

See for example Rip DVDs in MPEG-4 AVC (x264), multi audio, subtitles, Matroska.

Video on the Web
For videos on the web use both:
 * MP4 container with H.264 video codec and AAC codec, for use on Safari, IE 9, iPhone and older Androids.
 * WebM (A Mitroska container with VP8 video codec and Vorbis audio codec), for use on Firefox (4+), Chrome, Opera and Android.

If you care about support for older browers, you may add:
 * Ogg container with Theora video codec and Vorbis audio codec, for Firefox 3.5
 * Flash video (Flash Video (FLV) container, with FLV1 video codec and MP3 audio codec), for older webbrowsers, including IE 7 and IE 8.
 * Download button so people can download it for later viewing.

See for example Video on the Web and Video for Everybody for the details.

Personal Video
Most photo and video cameras have output in .AVI or .MOV files. AVI files at least do not support metadata (such as time of shooting or location).

My recommendation is to convert both formats to something that does support metadata, although I have not found something my media browsers has good support for storing metadata in the movie file (I'm using iView MediaPro, which has subsequently bought by Microsoft and later Phase One and rebranded as Expression Media).