Movie Formats

From Exterior Memory
(Redirected from Video Formats)
Jump to: navigation, search

The Basics

A typical movie file does not only contain video, but contains multiple different streams:

  • 1 Video stream
  • 1 or more Audio streams (e.g. English, German and director commentary)
  • 1 or more Subtitles (e.g. English and Dutch)

A movie file may also contain metadata (title, episode, director, date of recording) and chapters. Not to mention advanced features such as stereo video for stereo television (often improperly called 3D television).

The file that combines (muxes) the different streams and the metadata together is called the movie container.

Your Choices

To create a movie files, you must decide on:

  • The container format
  • The video codec
  • The audio codec
  • The subtitle format

Unfortunately, there is no single solution that works everywhere. Your solution may depend on what is available on your platform (Mac, Windows and Linux support different containers and codes out-of-the-box), the features you require (WebM does not yet support subtitles or multiple audio tracks), if you are willing to pay royalty fees (formats such a H.264 are encumbered by patents), if you need hardware support (H.264 on the iPhone has hardware support and uses much less battery power than any other video format), etc.

Further Information

For more information on the background, you should read the following sites:

Overview of Common Formats

The following tables are taken from Mark Pilgrims' A gentle introduction to video encoding, with WebM added. (Original sources: DigitalContentProducer.com, doom9.org, HydrogenAudio.org, Wikipedia)

Video Codecs

Codec Name Inventor Popular usage Patent-free? OSS tools Non-OSS tools
MPEG-1 MPEG WG VCD Expired ~2012 ffmpeg, MP1E QuickTime
MPEG-2 MPEG WG DVD No ffmpeg, mpeg2enc Compressor, Sonic, TMPGEnc
MPEG-4 ASP MPEG WG Movie pirates No ffmpeg, Xvid DivX, 3ivX
H.263 MPEG WG YouTube No ffmpeg Adobe Premiere
H.264 MPEG WG YouTube HD, iTunes, Blu-Ray No x264 Adobe Media Encoder, Apple Compressor, On2 Flix Pro, Sorenson Squeeze, Telestream
WMV Microsoft Microsoft world domination No none Windows Media Encoder, Expression Encoder
VC-1 Microsoft Blu-Ray No none Windows Media Encoder, Compressor, Squeeze
Theora On2, Xiph.org Free software hippies Yes* oggenc, Thusnelda none
Dirac BBC HDTV broadcast Yes* dirac-research, Schroedinger none
VP8 On2, Google YouTube, Free software hippies Yes* libvpx, ffmpeg, libav none
H.265 (HVEC) VCEG/MPEG WG Many vendors (in progress) No OpenHEVC, GPAC HM, VSS
VP9 Google YouTube, Web browsers, Free software hippies Yes* libvpx, ffmpeg (in progress) none

Audio Codecs

Codec Name Inventor Popular usage Patent-free? OSS tools Non-OSS tools
AMR 3GPP Ringtones, VoIP Expired? ffmpeg, RetroCode 3GPP reference encoder
MP2 MPEG WG VCD, DVD, Digital TV Expired TwoLAME Audio Transcoder, Easy CD-DA Extractor
MP3 MPEG WG Music Expired LAME iTunes
AAC MPEG WG iPod, iTunes Store videos No FAAC iTunes, Nero
WMA Microsoft Microsoft world domination No ffmepg Windows Media Encoder
Vorbis Xiph.org YouTube (in progress), Free software hippies Yes* ffmpeg, aoTuV none
AC3 Dolby DVD No ffmpeg, Aften Dolby
DTS DTS Inc. Blu-Ray No none DTS Inc.
Opus IETF VoIP, WebM, WebRTC Yes* libopus, VLC, ffmeg Teamspeak
  • (*) As far as we know

Subtitles (Captions)

Caption format Inventor/Specs Encoding Popular usage
Line 21 Electronic Industries Alliance pseudo-ASCII Analog TV (US + Canada)
Teletext BBC custom Analog TV (Europe)
SubRip A doom9.org forum post from 2004 windows-1252 AVI files, "fansubbing"
Advanced SubStation Alpha Some Word document floating around the net UTF-8 AVI files, "fansubbing"
SAMI MSDN page (multiple) Windows Media Player
SMIL W3C Recommendation (multiple) QuickTime (SMIL 1.0 only)
DVD DVD-Forum Stores as images DVD
MPEG-4 Timed Text ISO 14966-17 UTF-8 MP4 files, iTunes Store

Container Formats

Container Name Extension Inventor/Owner OSS tools
Audio/Video Interchange .AVI Microsoft ffmpeg, AVIMux-GUI
Advanced Systems Format .ASF, .WVM Microsoft ffmpeg (in progress)
Flash Video .FLV Adobe ffmpeg
Matroska .MKV Some guys in Russia ffmpeg, mkvtoolnix
Ogg .OGG Xiph.org ffmpeg, oggmux
MPEG-4 .MPV, .M4V MPEG-LA ffmpeg, MP4Box, mp4creator
QuickTime .MOV Apple ffmpeg
WebM * .WEBM Google, Adobe, Mozilla, Opera ffmpeg, mkvtoolnix
  • (*) Subset of the Matroska container format

Common combinations

Platform/Audience Container Video Audio Subtitles
DVD VOB MPEG-2 AC-3 Line 21, DVD subtitles
DVD pirates AVI MPEG-4 ASP MP3 SRT, ASS
Blu-Ray EVOB H.264, VC-1 AC-3, DTS Blu-Ray Subtitles (SUP)
Blu-Ray pirates MKV H.264 AC-3, DTS SRT, ASS
Flash 8 FLV H.263 MP3 Custom, if any
Flash 9+ MP4 H.264 AAC Flash Timed Text
Windows ASF WMV, VC-1 WMA SAMI
Linux Ogg Theora Vorbis Kate
Mac/QuickTime MOV H.264 AAC QTTextTrack
iPod/iPhone MP4 H.264 AAC MPEG-4 Timed Text
AppleTV MP4 H.264 AAC+AC3 MPEG-4 Timed Text
HTML5 (Firefox, Opera, Chrome, Android) WebM VP8 Vorbis none (will follow future WHATWG / W3C guidelines)
HTML5 (Firefox, Opera, Chrome, Android) WebM VP9 Opus none (will follow future WHATWG / W3C guidelines)
HTML5 (Safari, IE, iPhone) MP4 H.264 AAC none (will follow future WHATWG / W3C guidelines)

The Debate

One of the goals of the HTML5 working group was to define a video format that was supported in all webbrowsers. In the end, this proved unsuccessful, and Ian Hickson had to admit defeat. Basically Apple and Microsoft only support H.264 natively (and not WebM), while Mozilla and later Google only support WebM natively (and not H.264).

Recommended Formats

Stand-alone Movies

For movies and series use either:

  • The DVD VIDEO_TS format (which retains the original menu structure), but is very large in size.
  • Matroska (MKV) files with VP8 (or optionally H.264) for video codec, Vorbis for audio codec, and VobSubs for subtitles.

See for example Rip DVDs in MPEG-4 AVC (x264), multi audio, subtitles, Matroska.

Video on the Web

For videos on the web use both:

  • MP4 container with H.264 video codec and AAC codec, for use on Safari, IE 9, iPhone and older Androids.
  • WebM (A Mitroska container with VP8 video codec and Vorbis audio codec), for use on Firefox (4+), Chrome, Opera and Android.

If you care about support for older browers, you may add:

  • Ogg container with Theora video codec and Vorbis audio codec, for Firefox 3.5
  • Flash video (Flash Video (FLV) container, with FLV1 video codec and MP3 audio codec), for older webbrowsers, including IE 7 and IE 8.
  • Download button so people can download it for later viewing.

See for example Video on the Web and Video for Everybody for the details.

Personal Video

Most photo and video cameras have output in .AVI or .MOV files. AVI files at least do not support metadata (such as time of shooting or location).

My recommendation is to convert both formats to something that does support metadata, although I have not found something my media browsers has good support for storing metadata in the movie file (I'm using iView MediaPro, which has subsequently bought by Microsoft and later Phase One and rebranded as Expression Media).