Video Codecs Explained: A Beginner's Guide to How Video Is Compressed

If you have ever exported a video and stared at a menu offering H.264, HEVC, ProRes, VP9 and AV1, this guide is for you. Here are video codecs explained from the ground up, in plain language: what a codec is, why we need one, how it differs from a file format, and which ones matter in practice. No maths required.

Why video needs compressing at all

Start with a number. One second of uncompressed 1080p video at 30 frames per second is about 187 megabytes. A two-hour film would need more than a terabyte. Nobody could stream that, and few people could store it.

A codec (short for coder-decoder) is the method used to shrink that data to something manageable, and to rebuild a watchable picture from it later. Modern codecs routinely reduce video by a factor of 100 to 500 while still looking good. That is what makes streaming possible.

How compression works, without the jargon

Codecs use a few big ideas.

Remove what you would not notice. Our eyes are much more sensitive to brightness than to colour detail, so codecs store colour at lower resolution. They also simplify fine textures that the eye is unlikely to miss. This is why codecs are called lossy: some information is thrown away for good.

Don’t repeat yourself within a frame. Large areas of a picture, such as a blue sky, are similar. The codec describes one block and says “the neighbours look like this, with small changes”.

Don’t repeat yourself between frames. In most video, very little changes from one frame to the next. A presenter’s mouth moves; the wall behind them does not. So the codec stores a full picture only occasionally, as a keyframe, and for the frames in between it stores just the movement and differences. Our guide to keyframes goes deeper into this.

Newer codecs are better at all three tricks. They use larger and more flexible blocks, predict movement more accurately and have more tools for describing textures. The price is more computing power, especially when encoding.

Codec vs container: the most common confusion

People often call MP4 a codec. It is not. MP4 is a container, a file format that holds several streams together: video, audio, subtitles and some information about timing.

Think of the container as a box and the codec as the language the contents are written in. An MP4 box might hold H.264 video and AAC audio. Another MP4 might hold HEVC video and AC-3 audio. Both are “MP4 files”, but a device that cannot read HEVC will fail on the second one.

Common containers:

• MP4 (.mp4): the default for the web, phones and most platforms.
• MOV (.mov): Apple’s container, common in editing and camera workflows.
• MKV (.mkv): flexible, popular for archiving and media servers, less supported in browsers.
• WebM (.webm): used on the web with VP9 or AV1 and Opus audio.
• MPEG-TS (.ts): used in broadcast and older HLS streaming.

This distinction explains a lot of “why won’t it play?” moments. The file extension tells you the box, not what is inside. Tools such as MediaInfo, VLC’s codec information window, or our codec support checker tell you what a file contains or what a device can decode.

The video codecs you will meet

H.264 / AVC

Released in 2003 and still everywhere. Almost every phone, browser, TV and editing program can play it, usually with hardware help. When in doubt, H.264 is the safe choice.

H.265 / HEVC

The successor to H.264, from 2013. It produces similar quality at roughly half to two-thirds of the size, especially at 4K. iPhones record in it by default. Support is broad on phones, TVs and newer computers, but patchier in desktop browsers.

VP9

Google’s royalty-free codec, used heavily by YouTube. Similar efficiency to HEVC. Well supported in Chrome, Firefox, Edge and Android.

AV1

The newest widely used codec, released in 2018 by a group including Google, Apple, Netflix, Amazon and Microsoft. Royalty-free and very efficient, but slower to encode, and older devices cannot decode it in hardware.

VVC / H.266

The successor to HEVC, finalised in 2020. More efficient still, but device support is limited so far.

Editing codecs: ProRes, DNxHD, CineForm

These are intermediate or mezzanine codecs, designed for editing rather than delivery. They compress lightly, so files are huge, but they are fast to decode and survive repeated editing without visible loss. You edit in them and export to H.264 or HEVC for viewers.

For a head-to-head comparison of the three main delivery codecs, see H.264 vs H.265 vs AV1.

Audio codecs matter too

Video gets the attention, but every file also carries audio:

• AAC: the standard for MP4 and streaming. 128 kbps stereo is a good default.
• Opus: excellent quality at low bitrates, common in WebM and video calls.
• AC-3 and E-AC-3 (Dolby Digital and Dolby Digital Plus): surround sound for TVs and streaming services.
• MP3: still around, but rarely the best choice for video today.
• PCM / WAV: uncompressed, used in editing, far too large for delivery.

A mismatch here causes real problems. An MP4 with AC-3 audio may play video with no sound in some browsers.

Bitrate, resolution and quality

The codec is one of four things that decide how good a video looks:

1. Codec: how cleverly the data is compressed.
2. Bitrate: how much data you allow per second.
3. Resolution: how many pixels each frame has.
4. Content: how complex the picture is.

A modern codec at a sensible bitrate beats an old codec at a high one. But no codec can rescue a starved bitrate at a high resolution. Our guide to video bitrate explains how to balance them, and the bitrate calculator turns those choices into file sizes.

Hardware vs software decoding

When a device has a dedicated chip for a codec, playback is smooth and uses little power. When it does not, the main processor has to decode the video in software. That works on a desktop, but on a phone or laptop it can drain the battery, heat the device and drop frames at high resolutions.

This is why a codec being “supported” is not the whole story. A laptop might play AV1 in software but struggle at 4K. The codec support checker shows both whether a codec is supported and whether it is decoded efficiently.

How to choose, in practice

Sharing a video with friends or clients: H.264 in an MP4. It will play everywhere.

Uploading to YouTube, Vimeo or social media: H.264 at a generous bitrate. The platform re-encodes it anyway.

Streaming on your own site: H.264 renditions for everyone, optionally HEVC or AV1 for devices that support them. See adaptive bitrate streaming for how players choose.

Archiving your own footage: HEVC or AV1 at high quality, or keep the camera originals.

Editing: ProRes or DNxHD, then export to a delivery codec.

A quick glossary

• Encode: compress video with a codec.
• Decode: turn compressed video back into pictures for display.
• Transcode: decode and re-encode, usually to a different codec or bitrate.
• Remux: move streams into a different container without re-encoding. Fast and lossless.
• Profile and level: subsets of a codec’s features and limits, used for compatibility. H.264 “High” profile, level 4.1, is a common target.
• Bit depth: 8-bit or 10-bit colour. 10-bit is needed for HDR.

Why so many codecs exist

It is fair to ask why the industry did not settle on one codec and stop. There are three reasons.

Efficiency keeps improving. Each new generation squeezes the same quality into fewer bits. For a streaming service delivering billions of hours, a 30 percent saving is worth years of engineering.

Licensing. H.264 and HEVC are covered by patents licensed through pools, with fees for some uses. That pushed Google and later a wide group of companies to create royalty-free alternatives: VP9 and then AV1.

Hardware takes time. A new codec only becomes practical once phones, TVs and laptops ship with chips that decode it efficiently. That usually takes five years or more after the standard is finished, which is why old codecs stay in use for so long.

The result is overlap: most services encode two or three codecs and let each device choose. For a small site, that complexity is rarely worth it. One well-tuned H.264 ladder covers nearly everyone.

A simple test you can run

Take a one-minute clip of your own footage and export it three times at the same quality setting: once as H.264, once as HEVC and once as AV1 if your software offers it. Compare the file sizes, then watch all three on the screen you care about most. You will see the efficiency difference directly, and you will learn which codecs your own devices play smoothly. It takes ten minutes and teaches more than any chart.

Summary

A codec compresses video by throwing away detail you would not notice and avoiding repetition within and between frames. The container, such as MP4 or MKV, is only the box the compressed streams live in. H.264 remains the universal default, HEVC and VP9 save space, AV1 saves more and is royalty-free, and editing codecs like ProRes trade size for speed and quality during production. Pick the codec your viewers’ devices can decode in hardware, give it enough bitrate, and most “why won’t it play” problems disappear.