Deep Dives

The Secret Life of JPEG: How a 1992 Format Took Over the Internet

koboshiCo-founder
·11 min read
The Secret Life of JPEG: How a 1992 Format Took Over the Internet
Summary

JPEG is not magic. It is a pipeline of discrete cosine transforms, quantization tables, and chroma subsampling that throws away data your eyes were unlikely to notice. Thirty-three years later, it still dominates for one reason: everything opens it.

Open any .jpg file in a hex editor. The first two bytes:

FF D8

That is the Start of Image marker. Every JPEG file begins with it. Every JPEG decoder on Earth looks for it. The next bytes tell the decoder which flavor of JPEG this is, what color space to expect, and where the pixel data starts. The format is older than the web browser, yet it still carries roughly 57% of all images served on the modern web according to the 2025 Web Almanac.

The Storage Crisis That Created JPEG

In 1986, a raw 640 × 480 grayscale image ate 307 KB of disk space. A color image at the same resolution needed 921 KB. At a time when a 20 MB hard drive cost hundreds of dollars and 1.44 MB floppy disks were the standard exchange medium, a single uncompressed photo could fill two-thirds of a disk.

The need was obvious: a standard compression format for continuous-tone images — photographs, not line art. Multiple groups were working on the problem. The Joint Photographic Experts Group, formed in 1986 by ISO/IEC and ITU-T, merged the best ideas into a single draft. After six years of refinement, the standard was published in 1992 as ISO/IEC 10918-1.

JPEG was never intended to be the only image format. It was designed for one job: making photographs small enough to store and transmit. It does that job by discarding information in a very specific order.

Why JPEG Compresses So Well

JPEG compression is a pipeline, not a single algorithm. Each stage removes data that the human visual system is least likely to miss.

1. Color Space Conversion (RGB → YCbCr)

Your screen displays RGB. JPEG stores YCbCr. The Y channel carries luminance (brightness). Cb and Cr carry chrominance (blue-difference and red-difference). This matters because the human eye has about 2.5 million cone cells tuned to brightness and only 100,000 tuned to color. We see luminance detail far better than color detail.

2. Chroma Subsampling

Most JPEGs use 4:2:0 subsampling. For every 4 luminance samples, there is 1 Cb sample and 1 Cr sample. That means the chroma channels are stored at one-quarter the resolution of the luma channel. For a 4000 × 3000 image, the Y plane is full resolution. The Cb and Cr planes are 2000 × 1500 each. You just cut the raw data by roughly 50% before any real compression even starts, and most viewers never notice.

3. The Discrete Cosine Transform (DCT)

The image is split into 8 × 8 pixel blocks. Each block is run through a DCT, which converts spatial data (pixel values) into frequency data (how fast the values change across the block). The result is an 8 × 8 matrix of coefficients. The top-left value is the DC coefficient — the average brightness of the block. The other 63 are AC coefficients representing increasingly high-frequency detail.

High-frequency coefficients encode fine texture: hair, grass, noise. Low-frequency coefficients encode broad shapes: skies, walls, skin tones.

4. Quantization

Here is where the loss happens. JPEG applies a quantization table to each DCT block. The table is a second 8 × 8 matrix of divisors. Each DCT coefficient is divided by its matching quantizer and rounded to the nearest integer.

The standard quantization table hammers high-frequency coefficients hardest:

16  11  10  16  24  40  51  61
12  12  14  19  26  58  60  55
14  13  16  24  40  57  69  56
14  17  22  29  51  87  80  62
18  22  37  56  68 109 103  77
24  35  55  64  81 104 113  92
49  64  78  87 103 121 120 101
72  92  95  98 112 100 103  99

A high-frequency coefficient of, say, 7 gets divided by 121 and rounded to 0. It is gone. Irreversible. The decoder never sees it. That is lossy compression: data is destroyed, not just re-encoded.

At quality 90, the quantizers are divided by a smaller scaling factor. At quality 50, the scaling factor is higher. More coefficients zero out. The file gets smaller. The image gets softer.

5. Entropy Coding

After quantization, the remaining coefficients are zig-zag scanned, run-length encoded, and compressed with Huffman coding. This stage is lossless. It only packs the already-destroyed data more efficiently.

The result: a 12 MP uncompressed RGB image is 36 MB. Save it as JPEG quality 90 with 4:2:0 subsampling and it drops to ~3.5 MB. That is a 10:1 reduction with visible quality loss only under magnification.

How Lossy Is It, Really?

The damage is not evenly distributed.

QualityTypical size (12 MP)Visual impact
95+~8 MBNearly invisible; preferred for archiving
90~3.5 MBMinor softening; standard for cameras
75~1.8 MBVisible blur in fine detail; web default
50~1.0 MBBlocking artifacts obvious at 100% zoom
30~600 KBColor banding, mosquito noise, unusable for print

Three distinct artifacts appear as quality drops:

  • Blocking: Visible 8 × 8 grid edges, especially in smooth gradients like skies.
  • Ringing: Oscillating halos around high-contrast edges (text next to background).
  • Color bleeding: Chroma subsampling smears color across sharp boundaries.

The real killer is generation loss. Open a JPEG, edit it, save as JPEG again. Each save re-runs the entire pipeline: RGB → YCbCr → subsample → DCT → quantize. The rounding errors compound. After 10 generations, an image can look like it was painted in watercolor. After 50, it is unrecognizable.

The Green Shift and Other Re-compression Quirks

Re-save a JPEG enough times and you may notice the color temperature drifting. Some images take on a subtle green cast. Others shift magenta. The reason is buried in the chroma channels.

JPEG stores Cb and Cr at reduced resolution and quantizes them aggressively. Each save introduces rounding error in both channels. The conversion back to RGB uses this matrix:

R = Y + 1.402 × (Cr - 128)
G = Y - 0.344136 × (Cb - 128) - 0.714136 × (Cr - 128)
B = Y + 1.772 × (Cb - 128)

Notice that green is computed from both Cb and Cr. When repeated quantization nudges Cb upward and Cr downward by even a single quantization step, the G channel drifts. A positive bias in Cb pushes green down. A negative bias in Cr pushes green up. The interaction is not symmetric because the coefficients -0.344136 and -0.714136 have different magnitudes. The result: a slow accumulation of green in some image regions, especially where the original chroma values were already near quantization boundaries.

This is not a guaranteed effect. It depends on the encoder's quantization tables, the subsampling mode, and the image content. But it is real, reproducible, and one reason professional workflows avoid re-saving JPEGs.

If JPEG Is So Flawed, Why Does Everything Use It?

JPEG is not popular because it is perfect. It is popular because it is sufficient and universal.

  • No patent lock-in: The baseline JPEG patent (held by Forgent Networks) expired in 2006. The format is royalty-free.
  • Hardware decode: Every camera, phone, printer, and browser has a JPEG decoder in silicon or highly optimized C. It costs almost nothing to render.
  • Good enough: For social media, news sites, and email attachments, quality 75 JPEG is indistinguishable from the source on a phone screen.
  • Ecosystem inertia: Content Management Systems, CDNs, image libraries, and legacy archives all speak JPEG. Replacing them requires more than a better format. It requires a better format plus a reason to migrate petabytes of existing assets.

The Formats That Should Have Won

Several formats have tried to dethrone JPEG. None have succeeded completely.

FormatLossyLosslessTransparencyAnimationMax bit depthKey advantage
JPEGYesNoNoNo8-bitUniversal support
PNGNoYesYesNo16-bitPerfect lossless, alpha
WebPYesYesYesYes8-bit25–35% smaller than JPEG, browser-native
HEICYesYesYesYes16-bit~50% smaller than JPEG, Apple's default
AVIFYesYesYesYes12-bitBest compression today, royalty-free
JPEG XLYesYesYesYes32-bitLossless JPEG recompression, progressive decode

PNG solved the lossless problem but produces files 5–10× larger than JPEG for photos. It owns screenshots, UI assets, and graphics.

WebP (Google, 2010) beats JPEG on size and adds transparency and animation. It now ships in every major browser. The 2025 Web Almanac puts WebP at 11% of LCP images, up from 7% in 2024. It is the safe upgrade path today.

HEIC (Apple, 2017) uses HEVC compression inside an ISOBMFF container. It is ~40–50% smaller than JPEG and holds multiple images per file. It dominates the Apple ecosystem and stalls everywhere else because of HEVC patent pools.

AVIF (AOM, 2019) derives from AV1 video. It achieves the best compression ratios of any widely supported format — roughly 30% smaller than WebP at equivalent quality. The downside is decode speed. AVIF images can take 2–3× longer to render than JPEG on mobile devices, eating battery and delaying Largest Contentful Paint.

JPEG XL (ISO/IEC 18181, 2021) is technically superior to all of them. It compresses 50–60% smaller than JPEG. It decodes fast. It supports progressive decoding: a usable image appears after downloading only ~1% of the file. Most importantly, it can losslessly recompress existing JPEGs for ~20% size savings with bit-for-bit recovery of the original. No other format can do this.

Where We Are Now

JPEG XL had a rough childhood. Google added experimental support to Chrome in 2021, then removed it on October 31, 2022 — the "Halloween decision." The stated reason: insufficient incremental benefit over existing formats. The backlash was immediate. The Chromium issue became the second most-starred in the project's history. Google was accused of protecting AVIF, a format tied to the Alliance for Open Media that Google co-founded.

In late 2025, Chromium reversed course. A new Rust decoder (jxl-rs) landed in Chrome Canary. Chrome 145, released February 2026, shipped JPEG XL support behind a flag. Safari has supported it since 2023. Firefox Nightly is integrating the same Rust decoder. JPEG XL is not enabled by default yet, but it is back in the codebase.

AVIF, meanwhile, is the pragmatic choice for 2026. Browser support is broad. Encoders are improving. Cloudinary and Cloudflare both serve AVIF automatically via Accept header negotiation. The median page serving AVIF or WebP shows 81% good LCP rates versus 64% for JPEG-only pages, per CoreDash data.

The Bottom Line

JPEG is a 33-year-old compromise. It throws away color resolution, high-frequency texture, and numerical precision in exchange for file sizes that made digital photography viable in the 1990s. The artifacts are well understood. The generation loss is real. The green drift happens.

Yet JPEG persists for the same reason QWERTY persists: the switching cost is higher than the pain of staying.

The practical path forward is layered:

  1. Keep high-quality masters. Archive originals as PNG, TIFF, or quality 95+ JPEG. Never re-edit from a quality 75 web export.
  2. Serve modern formats dynamically. Use a CDN or image service that negotiates AVIF, WebP, or JPEG XL based on the browser's Accept header. Store one master; let the edge convert on demand.
  3. Do not batch-re-encode JPEGs. Each generation destroys data. If you need smaller files, re-encode from the master, not from another JPEG.

JPEG will not die suddenly. It will fade the same way GIF faded: still supported everywhere, still opening in every viewer, but increasingly outclassed by formats that do the same job with fewer bytes and fewer artifacts. The difference is that this time, the replacement formats are actually winning.

More blog posts to read