The "codecs" parameter in common media types - Web media technologies 编辑

At a fundamental level, you can specify the type of a media file using a simple MIME type, such as video/mp4 or audio/mpeg. However, many media types—especially those that support video tracks—can benefit from the ability to more precisely describe the format of the data within them. For instance, just describing a video in an MPEG-4 file with the MIME type video/mp4 doesn't say anything about what format the actual media within takes.

For that reason, the codecs parameter can be added to the MIME type describing media content. With it, container-specific information can be provided. This information may include things like the profile of the video codec, the type used for the audio tracks, and so forth.

This guide briefly examines the syntax of the media type codecs parameter and how it's used with the MIME type string to provide details about the contents of audio or video media beyond indicating the container type.

General syntax

A basic MIME media type is expressed by stating the type of media (audio, video, etc), then a slash character (/), then the container format used to contain the media:

audio/mpeg

An audio file using the MPEG file type, such as an MP3.

video/ogg

A video file using the Ogg file type.

video/mp4

A video file using the MPEG-4 file type.

video/quicktime

A video file in Apple's QuickTime format. As noted elsewhere, this format was once commonly used on the web but no longer is, since it required a plugin to use.

However, each of these MIME types is vague. All of these file types support a variety of codecs, and those codecs may have any number of profiles, levels, and other configuration factors. For this reason, you can add the codecs parameter to the media type.

To do so, append a semicolon (;) followed by codecs= and then the string describing the format of the contents of the file. Some media types only let you specify the names of the codecs to use, while others allow you to specify various constraints on those codecs as well. You can specify multiple codecs by separating them with commas.

audio/ogg; codecs=vorbis

An Ogg file containing a Vorbis audio track.

video/webm; codecs="vp8, vorbis"

A WebM file containing VP8 video and/or Vorbis audio.

video/mp4; codecs="avc1.4d002a"

An MPEG-4 file containing AVC (H.264) video, Main Profile, Level 4.2.

As is the case with any MIME type parameter, codecs must be changed to codecs* (note the asterisk character, *) if any of the properties of the codec use special characters which must be percent-encoded per RFC 2231, section 4: MIME Parameter Value and Encoded Word Extensions. You can use the JavaScript encodeURI() function to encode the parameter list; similarly, you can use decodeURI() to decode a previously encoded parameter list.

When the codecs parameter is used, the specified list of codecs must include every codec used for the contents of the file. The list may also contain codecs not present in the file.

Codec options by container

The containers below support extended codec options in their codecs parameters:

• 3GP
• AV1
• ISO BMFF
• MPEG-4
• QuickTime
• WebM

Several of the links above go to the same section; that's because those media types are all based on ISO Base Media File Format (ISO BMFF), so they share the same syntax.

AV1

The syntax of the codecs parameter for AV1 is defined the AV1 Codec ISO Media File Format Binding specification, section 5: Codecs Parameter String.

av01.P.LLT.DD[.M[.CCC[.cp[.tc[.mc[.F]]]]]]

This codec parameter string's components are described in more detail in the table below. Each component is a fixed number of characters long; if the value is less than that length, it must be padded with leading zeros.

All fields from M (monochrome flag) onward are optional; you may stop including fields at any point (but can't arbitrarily leave out fields). The default values are included in the table above. Some example AV1 codec strings:

av01.2.15M.10.0.100.09.16.09.0

AV1 Professional Profile, level 5.3, Main tier, 10 bits per color component, 4:2:2 chroma subsampling using ITU-R BT.2100 color primaries, transfer characteristics, and YCbCr color matrix. The studio swing representation is indicated.

av01.0.15M.10

AV1 Main Profile, level 5.3, Main tier, 10 bits per color component. The remaining properties are taken from the defaults: 4:2:0 chroma subsampling, BT.709 color primaries, transfer characteristics, and matrix coefficients. Studio swing representation.

ISO Base Media File Format: MP4, QuickTime, and 3GP

All media types based upon the ISO Base Media File Format (ISO BMFF) share the same syntax for the codecs parameter. These media types include MPEG-4 (and, in fact, the QuickTime file format upon which MPEG-4 is based) as well as 3GP. Both video and audio tracks can be described using the codecs parameter with the following MIME types:

Each codec described by the codecs parameter can be specified either as the container's name (3gp, mp4, quicktime, etc.) or as the container name plus additional parameters to specify the codec and its configuration. Each entry in the codec list may contain some number of components, separated by periods (.).

The syntax for the value of codecs varies by codec; however, it always starts with the codec's four-character identifier, a period separator (.), followed by the Object Type Indication (OTI) value for the specific data format. For most codecs, the OTI is a two-digit hexadecimal number; however, it's six hexadecimal digits for AVC (H.264).

Thus, the syntaxes for each of the supported codecs look like this:

cccc[.pp]* (Generic ISO BMFF)

Where cccc is the four-character ID for the codec and pp is where zero or more two-character encoded property values go.

mp4a.oo[.A] (MPEG-4 audio)

Where oo is the Object Type Indication value describing the contents of the media more precisely and A is the one-digit audio OTI. The possible values for the OTI can be found on the MP4 Registration Authority web site's Object Types page. For example, Opus audio in an MP4 file is mp4a.ad. For further details, see MPEG-4 audio.

mp4v.oo[.V] (MPEG-4 video)

Here, oo is again the OTI describing the contents more precisely, while V is the one-digit video OTI.

avc1.oo[.PPCCLL] (AVC video)

oo is the OTI describing the contents, while PPCCLL is six hexadecimal digits specifying the profile number (PP), constraint set flags (CC), and level (LL). See AVC profiles for the possible values of PP.

The constraint set flags byte is comprised of one-bit Boolean flags, with the most significant bit being referred to as flag 0 (or constraint_set0_flag, in some resources), and each successive bit being numbered one higher. Currently, only flags 0 through 2 are used; the other five bits must be zero. The meanings of the flags vary depending on the profile being used.

The level is a fixed-point number, so a value of 14 (decimal 20) means level 2.0 while a value of 3D (decimal 61) means level 6.1. Generally speaking, the higher the level number, the more bandwidth the stream will use and the higher the maximum video dimensions are supported.

AVC profiles

The following are the AVC profiles and their profile numbers for use in the codecs parameter, as well as the value to specify for the constraints component, CC.

MPEG-4 audio

When the value of an entry in the codecs list begins with mp4a, the syntax of the value should be:

mp4a.oo[.A]

Here, oo is the two-digit hexadecimal Object Type Indication which specifies the codec class being used for the media. The OTIs are assigned by the MP4 Registration Authority, which maintains a list of the possible OTI values. A special value is 40; this indicates that the media is MPEG-4 audio (ISO/IEC 14496 Part 3). In order to be more specific still, a third component—the Audio Object Type—is added for OTI 40 to narrow the type down to a specific subtype of MPEG-4.

The Audio Object Type is specified as a one or two digit decimal value (unlike most other values in the codecs parameter, which use hexadecimal). For example, MPEG-4's AAC-LC has an audio object type number of 2, so the full codecs value representing AAC-LC is mp4a.40.2.

Thus, ER AAC LC, whose Audio Object Type is 17, can be represented using the full codecs value mp4a.40.17. Single digit values can be given either as one digit (which is the best choice, since it will be the most broadly compatible) or with a leading zero padding it to two digits, such as mp4a.40.02.

Note: The specification originally mandated that the Audio Object Type number in the third component be only one decimal digit. However, amendments to the specification over time extended the range of these values well beyond one decimal digit, so now the third parameter may be either one or two digits. Padding values below 10 with a leading 0 is optional. Older implementations of MPEG-4 codecs may not support two-digit values, however, so using a single digit when possible will maximize compatibility.

The Audio Object Types are defined in ISO/IEC 14496-3 subpart 1, section 1.5.1. The table below provides a basic list of the Audio Object Types and in the case of the more common object ypes provides a list of the profiles supporting it, but you should refer to the specification for details if you need to know more about the inner workings of any given MPEG-4 audio type.

[1] SAOC and LD MPEG Surround are defined in ISO/IEC 14496-3:2009/Amd.2:2010(E).

WebM

The basic form for a WebM codecs parameter is to list one or more of the four WebM codecs by name, separated by commas. The table below shows some examples:

The strings vp8.0 and vp9.0 also work, but are not recommended.

ISO Base Media File Format syntax

As part of a move toward a standardized and powerful format for the codecs parameter, WebM is moving toward describing video content using a syntax based on that defined by the ISO Base Media File Format. This syntax is defined in VP Codec ISO Media File Format Binding, in the section Codecs Parameter String. The audio codec continues to be indicated as either vorbis or opus.

In this format, the codecs parameter's value begins with a four-character code identifying the codec being used in the container, which is then followed by a series of period (.) separated two-digit values.

cccc.PP.LL.DD.CC[.cp[.tc[.mc[.FF]]]]

The first five components are required; everything from cp (color primaries) onward is optional; you can stop including components at any point from then onward. Each of these components is described in the following table. Following the table are some examples.

WebM media type examples

video/webm;codecs="vp08.00.41.08,vorbis"

VP8 video, profile 0 level 4.1, using 8-bit YUV with 4:2:0 chroma subsampling, using BT.709 color primaries, transfer function, and matrix coefficients, with the luminance and chroma values encoded within the legal ("studio") range. The video is Vorbis.

video/webm;codecs="vp09.02.10.10.01.09.16.09.01,opus"

VP9 video, profile 2 level 1.0, with 10-bit YUV content using 4:2:0 chroma subsampling, BT.2020 primaries, ST 2084 EOTF (HDR SMPTE), BT.2020 non-constant luminance color matrix, and full-range chroma and luma encoding. The audio is in Opus format.

Using the codecs parameter

You can use the codecs parameter in a few situations. Firstly, you can use it with the <source> element when creating an <audio> or <video> element, in order to establish a group of options for the browser to choose from when selecting the format of the media to present to the user in the element.

You can also use the codecs parameter when specifying a MIME media type to the MediaSource.isTypeSupported() method; this method returns a Boolean which indicates whether or not the media is likely to work on the current device.

See also

• Web media technologies
• Guide to media types and formats on the web
• Guide to audio codecs used on the web
• Guide to video codecs used on the web
• Codecs used by WebRTC