JPEG from Zero to Hero: How JPEG Compression Works, Its Variants, and an Encoder You Build in Python

Editorial note: This article was developed with AI-assisted research and drafting, followed by technical review and editorial input. The illustrations were AI-generated. It was originally published on devsdaily.com, where the animations play as video.

In short: We follow one espresso photo through JPEG. Part 1 tells the story of how JPEG compression works: what it keeps, what it quietly throws away, and why your eyes don't notice. Part 2 meets the rest of the family, from progressive JPEG to JPEG XL and JPEG AI, measured on the same photo. Part 3 is the tutorial: you build a baseline JPEG encoder in about 350 lines of Python that writes real .jpg files and lands within 1% of libjpeg-turbo's file size.

Here is a photo of an espresso, sitting on a worn wooden table. It is 600 pixels wide and 400 tall, and every pixel is three numbers: how much red, green and blue it holds. That is 720,000 bytes.

Save it as a JPEG at quality 75, the default in libjpeg and Pillow, and it shrinks to 40,908 bytes, seventeen times smaller. Put the two side by side and you will struggle to say which is which.

That should feel slightly suspicious. Nearly 680,000 bytes have gone, and you can't see where. This article is the story of where they went. First we follow the photo through JPEG one step at a time, then we meet the formats that grew out of it, and finally you write an encoder that does the whole journey yourself.

Part 1: How JPEG compression works

JPEG doesn't store your photo. It stores what you will notice.

JPEG was designed in the late 1980s, when sending our espresso photo over a 2,400 bits-per-second modem would have taken 40 minutes. A lossless squeeze, the kind ZIP or PNG does, saves only about 40% on a photo like this, because photos are full of fine noise that never repeats exactly. So the committee behind JPEG asked a different question: not "how do we store every pixel?" but "which parts of this picture would a person actually miss?"

The answer came from two habits of the human eye, and the whole format is built on them.

Your eye cares more about brightness than about colour. You notice a slight change in how light or dark something is far more easily than a slight change in its hue, especially in small details.

Your eye cares more about broad shapes than about fine texture. The curve of the cup matters to you; the exact grain of the wood under it, much less.

Keep those two habits in mind. Every step that follows exploits one of them.

Step 1: Split brightness from colour

A camera records red, green and blue, but those three mix brightness and colour together. So JPEG first rewrites each pixel as three different numbers: Y, the brightness (called luma), and two colour numbers, Cb (how blue it is) and Cr (how red it is). Nothing is lost yet; it is the same information, sorted differently.

Look at what that does to our photo:

Illustration: the espresso photo split into brightness and two colour layers. AI-generated.

All the detail lives in Y: the crema, the wood grain, the reflections in the spoon. The two colour layers are soft, almost blurry.

Step 2: Keep less colour

Since your eye barely uses colour detail, JPEG stores the colour layers at a lower resolution. The usual choice, called 4:2:0, keeps one colour sample for every 2×2 square of pixels, so each colour layer shrinks to a quarter of its size.

This is the first step that loses information, and on a photo it is almost free: our three full layers become one full layer plus two quarter layers, half the numbers we started with.

Step 3: Describe each tile as a mix of patterns

Now JPEG cuts each layer into tiles of 8×8 pixels and does something that sounds odd. Instead of storing the 64 pixel values of a tile, it describes the tile as a mix of 64 fixed patterns, from a perfectly flat square to a fine checkerboard, and stores how much of each pattern the tile needs.

Illustration: the 64 patterns of an 8×8 DCT. AI-generated.

This rewrite is the discrete cosine transform, the DCT. Any 8×8 tile is exactly some mix of these patterns, so nothing is lost. What changes is the order of things. Most tiles in a photo are mostly smooth, so most of their information lands on the few coarse patterns at the top left, and the fine patterns get tiny weights.

Watch it happen on one real tile, taken from the rim of the cup. The animation adds the patterns one at a time, coarse to fine:

Animation: one tile from the cup's rim, rebuilt from its real DCT weights in zig-zag order. AI-generated.

A handful of patterns already give you the edge and the shading. Everything after that is fine texture, the part your eye cares least about. That is the opening JPEG needs.

Step 4: Round the weights (this is the quality slider)

Here is where JPEG throws information away on purpose. It divides each pattern's weight by a step size and rounds the result to a whole number. Coarse patterns get small steps, so they are kept accurately. Fine patterns get large steps, so their small weights round to zero.

The 64 step sizes form the quantisation table, and the quality setting in your image library simply scales that table up or down. Quality 90 means small steps and little rounding. Quality 10 means huge steps, and most of every tile rounds to nothing. When you type quality=85, this is the knob you are turning.

Step 5: Pack what's left

After rounding, most weights in most tiles are zero. JPEG reads each tile in a zig-zag from coarse to fine patterns, so the zeros pile up at the end, where a single "end of tile" symbol can stand in for all of them. Then Huffman coding gives the most common symbols the shortest bit codes, the same idea as Morse code giving the letter E a single dot.

Here is the whole journey in numbers for our photo at quality 75:

Illustration: where the photo's bytes go at quality 75. AI-generated.

The bottom strip is the whole trick in one picture. After rounding, 84% of the numbers are zero, and zeros cost almost nothing to store.

The way back

Opening a JPEG runs the journey in reverse: unpack the symbols, multiply the weights back up, turn the patterns back into pixels, enlarge the colour layers and convert to red, green and blue. Only two steps can't be undone, because they threw something away: shrinking the colour in step 2 and rounding in step 4. Everything else comes back exactly.

How far can you push it? A 1991 description of the standard by Gregory Wallace, who chaired the committee, gave a rough guide for colour photos: 0.25 to 0.5 bits per pixel for moderate to good quality, 0.75 to 1.5 for excellent, and 1.5 to 2 for results usually indistinguishable from the original. Our quality 75 file works out at 1.36 bits per pixel.

Where JPEG breaks

Once you know the journey, you can predict where it goes wrong:

• Text, line art and screenshots. Sharp edges need lots of fine patterns, exactly what rounding removes, so every letter grows a faint ripple and coloured text smears. PNG or WebP lossless are better tools here.
• Transparency. The classic JPEG has no alpha channel.
• Very low quality. When nearly every fine weight rounds away, each tile collapses to its flat average and the 8×8 grid shows through.
• Editing loops. Every save rounds again, against a tile grid that shifts when you crop. Part 3 measures how much that costs.

Part 2: The JPEG family

"JPEG" today means more than one format. The 1992 standard has hidden modes most people never meet, a set of wrappers that carry metadata, better encoders that still write the same format, and a line of successors. Here is the whole family tree at a glance:

Illustration: how JPEG grew up, 1974 to 2026. AI-generated.

Two moments on that timeline shape everything below. The free libjpeg code of 1991 is a big reason JPEG ended up on every device, and the patents on one optional part of the 1992 standard are the reason a better coder sat unused for thirty years. You will meet both in a moment.

Inside the 1992 standard: the modes

The standard describes more than the JPEG you know:

I encoded the espresso photo in each mode today's libraries support, at quality 75 (the scripts behind every number in this part are in the appendix):

Progressive JPEG stores exactly the same rounded weights, just in a different order. A baseline file is stored tile by tile from top to bottom, so a slow download paints the picture like a roller blind. A progressive file sends the average of every tile first, then the coarse patterns, then the fine ones, so you see a blurry version of the whole photo almost at once. This animation decodes the real bytes of both files as they arrive:

Animation: baseline and progressive versions of the same photo, decoded from their real bytes as they arrive. AI-generated.

libjpeg-turbo split our photo into ten passes. After the first 9% of the file you have the whole picture at 22.95 dB, and after a third of it 28.03 dB. When the last pass lands, the pixels match the baseline file exactly.

Arithmetic coding is the road not taken. It saved 8.0% for free, with identical pixels, yet almost nobody uses it. In the 1990s it was covered by patents; the Independent JPEG Group removed it from libjpeg and warned that it "cannot legally be used without obtaining one or more licenses". The patents have long since expired, but thirty years of software grew up without it. That history still bites: in Chromium 153 (the build bundled with Playwright), arithmetic-coded and 12-bit files report a successful decode and then draw a completely transparent image. There is no error to catch; your users just see nothing.

Around the image data: the wrappers

A .jpg file is a list of tagged segments, and decoders skip the segments they don't understand. That one design choice let people add features for thirty years without breaking old viewers:

• JFIF (1992) says which colour space the numbers use; it is what made .jpg files interchangeable.
• Exif (1995) carries camera settings, GPS, a thumbnail and the orientation flag. If you strip metadata from uploads, rotate the pixels first (in Pillow, ImageOps.exif_transpose), or portrait photos come out sideways.
• ICC profiles describe exactly what the colour numbers mean, for wide-gamut screens and print.
• Gain maps hide an HDR version inside an ordinary JPEG. Android 14 introduced Ultra HDR in 2023, and ISO 21496-1 standardised the metadata in 2025; old viewers just show the normal picture.

Same format, smarter encoders

The standard fixes what a decoder must do but leaves the encoder free to choose its tables and its rounding. So encoders kept improving without anyone else changing anything. libjpeg-turbo (2010) made the original library 2 to 6 times faster, by its maintainers' measure, and it now sits inside most browsers and Android. Mozilla's MozJPEG (2014), Google's Guetzli (2017) and Google's jpegli (2024) all produce ordinary JPEGs that are smaller at the same visual quality; Google reported about 35% for both of its encoders, on its own tests and, for jpegli, at high quality settings. If you serve a lot of JPEGs, switching the encoder is the cheapest saving available.

The successors

The committee kept going where JPEG could not stretch:

• JPEG-LS (1999): simple, fast lossless and near-lossless coding.
• JPEG 2000 (2000): wavelets instead of 8×8 tiles, so no blocking; it became the format of digital cinema and many medical and archive systems.
• JPEG XR (2009): Microsoft's HD Photo, with HDR and alpha; it never spread far beyond Windows.
• JPEG XT (2015): HDR, alpha and lossless as extensions that old JPEG decoders can still show.
• JPEG XS (2019): not for files but for live video in broadcast studios, with a delay of only a few lines of pixels.
• JPEG XL (2022): the modern all-rounder, which can also repack an existing JPEG into a smaller file and restore the original bytes exactly.
• JPEG AI (2025): the first where the transform is a trained neural network.

Here is how they compare on our photo. First, lossless compression, where every file decodes back to exactly the original pixels:

Thirty years of work bought about 28% over the 1992 lossless mode. Photos are noisy, and noise can't be squeezed losslessly, which is exactly why JPEG chose to be lossy.

JPEG XL's repacking trick did better than any of these on its own terms: it turned our 41,606 byte JPEG into a 36,041 byte JPEG XL file, 13.4% smaller, and decoding it gave back the original JPEG byte for byte.

Then lossy compression. I tuned each encoder to produce a file the same size as our quality 75 JPEG and measured the result (WebP and AVIF are not JPEG standards, but you will be choosing between them):

PSNR and SSIM compare pixel values, and higher is better for both. Take the ranking with a pinch of salt: it is one photo, and the encoders aim at different targets (JPEG 2000's encoder here maximises PSNR directly, while JPEG XL's is tuned for how people see, which pixel metrics undervalue). The safe conclusion is the headline: at the same size, every successor beats the 1992 format clearly.

And yet the 1992 format is still everywhere, because formats spread through decoders, not encoders. My Chromium 153 test decoded WebP and AVIF but none of JPEG-LS, JPEG 2000, JPEG XR or JPEG XL. That is finally changing for JPEG XL: Safari has decoded it since version 17 (caniuse marks the support as partial), and Firefox 158 and Chrome 155 both turned it on in October 2026. When I wrote this, caniuse showed no Edge version with it enabled, so serve JPEG XL with a JPEG fallback in a <picture> element.

JPEG AI: when the patterns are learned

Every format so far uses patterns designed by people. JPEG AI, published in September 2025 as ISO/IEC 6048-1, replaces them with a trained neural network. The journey from Part 1 maps across almost step for step:

Learned patterns can describe textures and edges far better than 64 fixed cosines, so the same quality needs fewer bits. The committee describes roughly a 30% saving over the best conventional codecs at the same subjective quality, and an overview paper by Esenlik and colleagues (IEEE TCSVT, 2026) measures 14% to 27% against VVC Intra, the still-image mode of the H.266 video codec, depending on which encoder and decoder are used. The costs are compute and caution: decoding runs neural networks (the standard defines three decoders, from about 14 to 215 thousand multiply-accumulates per pixel, all reading the same file), and the paper notes weaker results on screenshots with text.

When I wrote this, no browser decoded JPEG AI, so it is not a web format yet. It makes sense where you control both ends, such as photo archives and camera-to-cloud pipelines, and the reference software is linked from jpeg.org if you want to measure it on your own images.

Choosing a format today

Part 3: Build a JPEG encoder in Python

Reading about the journey is one thing; making it happen is another. In this part you write the five steps from Part 1 as code, ending with a real .jpg file that any viewer opens.

All the encoder code goes into one file, tinyjpeg.py. Each code block in this part is the next piece of that file, so paste them in order. It needs only NumPy; Pillow and scikit-image are for testing it.

Colour: from RGB to YCbCr

These are the full-range formulas from JFIF (ITU-T T.871), which nearly every JPEG decoder assumes. The weights in the Y line reflect how bright each primary looks to us; green counts for almost 60%.

"""tinyjpeg: a baseline JPEG encoder in plain Python + NumPy, written to be read. It writes real .jpg files (baseline DCT, 4:2:0, optimised Huffman tables) that any image viewer can open. It is slow and skips every speed trick on purpose. """ import heapq import struct import numpy as np def rgb_to_ycbcr(rgb): """JFIF full-range conversion. Input uint8 HxWx3, output float HxWx3.""" rgb = rgb.astype(np.float64) r, g, b = rgb[..., 0], rgb[..., 1], rgb[..., 2] y = 0.299 * r + 0.587 * g + 0.114 * b cb = -0.168736 * r - 0.331264 * g + 0.5 * b + 128 cr = 0.5 * r - 0.418688 * g - 0.081312 * b + 128 return np.stack([y, cb, cr], axis=-1) def ycbcr_to_rgb(ycc): y, cb, cr = ycc[..., 0], ycc[..., 1] - 128, ycc[..., 2] - 128 r = y + 1.402 * cr g = y - 0.344136 * cb - 0.714136 * cr b = y + 1.772 * cb return np.clip(np.round(np.stack([r, g, b], axis=-1)), 0, 255).astype(np.uint8) 

The conversion is a 3×3 matrix and an offset, and the decoder runs the inverse, so nothing is lost here beyond rounding.

Chroma subsampling: the first lossy step

def pad_to_multiple(plane, m): """Repeat the last row/column until both sides are a multiple of m.""" h, w = plane.shape return np.pad(plane, ((0, (-h) % m), (0, (-w) % m)), mode="edge") def downsample_2x2(plane): """4:2:0 chroma: average each 2x2 square into one sample.""" h, w = plane.shape return plane.reshape(h // 2, 2, w // 2, 2).mean(axis=(1, 3)) def upsample_2x2(plane): return plane.repeat(2, axis=0).repeat(2, axis=1) 

JPEG cuts planes into whole blocks, so we pad every plane by repeating its edge. With 4:2:0 the encoder works in 16×16 pixel units, so our 600×400 photo is padded to 608×400. The decoder knows the real size from the header and crops the padding off.

How much does subsampling buy? With Pillow at quality 75, keeping full-resolution colour (4:4:4) gives a 51,481 byte file at 33.41 dB PSNR. With 4:2:0 the file drops to 40,865 bytes at 32.43 dB: 21% smaller for a loss you would struggle to spot in a photo. For screenshots with thin coloured text it is a bad deal; ask your library for 4:4:4. Every figure in this article is a JPEG saved with 4:4:4 for exactly that reason: they are full of text.

The DCT in code

The DCT is a single 8×8 matrix. Row k of that matrix is the k-th cosine pattern in one dimension, and multiplying on both sides applies it to the rows and the columns of a block:

def dct_matrix(n=8): """Row k is the k-th cosine basis vector, scaled so the matrix is orthonormal.""" c = np.zeros((n, n)) for k in range(n): alpha = np.sqrt(1 / n) if k == 0 else np.sqrt(2 / n) for x in range(n): c[k, x] = alpha * np.cos((2 * x + 1) * k * np.pi / (2 * n)) return c C = dct_matrix() def dct2(block): return C @ block @ C.T def idct2(coeffs): return C.T @ coeffs @ C 

The matrix is orthonormal, so its transpose is its inverse; that is why idct2 is the same expression with C.T and C swapped. Before the transform, JPEG subtracts 128 from every sample so values sit around zero.

Let's watch a real block: the same tile from the cup's rim that the animation in Part 1 rebuilt. Save this next to tinyjpeg.py as one_block.py. It uses functions from the next three sections, so run it once tinyjpeg.py is complete; I show its output in two halves as we go.

import numpy as np from skimage import data import tinyjpeg as tj np.set_printoptions(linewidth=120, suppress=True) rgb = data.coffee() y = tj.rgb_to_ycbcr(rgb)[..., 0] block = y[96:104, 264:272] # 8x8 on the cup's inner rim print("pixels (Y):\n", np.round(block).astype(int)) coeffs = tj.dct2(block - 128) print("DCT coefficients:\n", np.round(coeffs).astype(int)) table = tj.scale_table(tj.LUMA_Q, 75) q = tj.quantize(coeffs, table) print("quantised (q=75):\n", q) zz = tj.to_zigzag(q) print("zig-zag:", zz.tolist()) dc, ac = tj.block_symbols(zz, prev_dc=0) print("symbols:", [(hex(s), b, n) for s, b, n in dc + ac]) back = tj.idct2(tj.dequantize(q, table)) + 128 print("decoded pixels:\n", np.round(back).astype(int)) print("max pixel error:", int(np.abs(np.round(back) - np.round(block)).max())) 

The first half of what it prints:

pixels (Y): [[133 124 128 126 129 119 117 118] [105 92 94 94 98 94 95 102] [ 92 93 96 96 92 96 97 96] [ 93 91 93 93 91 91 95 96] [ 91 90 88 87 80 87 91 85] [ 81 91 83 82 76 76 84 77] [ 83 84 83 80 80 81 73 81] [ 76 72 79 75 75 77 77 80]] DCT coefficients: [[-290 7 5 -3 1 3 2 1] [ 97 2 -1 3 8 1 1 4] [ 27 3 -3 2 8 -4 6 4] [ 29 14 0 1 -2 0 -5 0] [ 27 0 -1 1 1 3 1 2] [ 26 4 -6 -1 1 -4 0 -1] [ 9 0 -4 -3 -7 2 -5 0] [ 8 2 -2 2 -2 0 3 -2]] 

The top row of pixels is brighter than the rest, and brightness fades downward. That is a horizontal edge, and the DCT says so: the big values sit in the first column, which holds the patterns with horizontal stripes. The -290 in the corner is the DC coefficient, eight times the block's average after the level shift. Most other entries are single digits.

Quantisation: where the quality slider lives

The JPEG standard's Annex K gives example quantisation tables, tuned on how visible each pattern is to people. They are not mandatory, but nearly every encoder starts from them. The quality number you pass to a library is not part of the standard at all. It is libjpeg's convention for scaling these tables: quality 50 uses them as printed, lower values multiply them up, higher values shrink them, and 100 sets every step to 1.

# Example tables from Annex K of the JPEG standard (ITU-T T.81), in row order. LUMA_Q = np.array([ 16, 11, 10, 16, 24, 40, 51, 61, 12, 12, 14, 19, 26, 58, 60, 55, 14, 13, 16, 24, 40, 57, 69, 56, 14, 17, 22, 29, 51, 87, 80, 62, 18, 22, 37, 56, 68, 109, 103, Source：https://dev.to/iamsunny/jpeg-from-zero-to-hero-how-jpeg-compression-works-its-variants-and-an-encoder-you-build-in-python-2ajn
My favorites

Related posts

Chart typography: axis labels, annotations, and the 8px crime scene

Chart typography: axis labels, annotations, and the 8px crime scene Charts are typography under constraints: every pixel

0dev.to Design

Editing Text Inside Finished Images Without the Original Design File

Changing a date, price, or label in a finished image is different from editing a document: the original font and design

0dev.to Design

ZeroDay Marketplace (zdmarket.at) Exposed: Inside a Clearnet Carding Hub With Fullz, CVVs and Cashout Guides

Disclaimer: This article is written strictly for educational, research, and cybersecurity awareness purposes. The author

0dev.to Discuss

Building HelpBLESS: A Specific Request Is a Better Interface Than a General Appeal — What Worked, What Broke

Most help requests fail before anyone decides whether to help, and the failure is an interface failure. A general appeal

0dev.to Discuss

Building TaskFamily: A Guest Visit Is a System, Not a Host's Memory — What Worked, What Broke

Every household product eventually meets the guest visit, and the guest visit is unforgiving in a specific way: it conce

0dev.to Discuss

Smart Collections Without a Query Engine: Rules That Write Plain Membership Rows

Collections in most shops start as hand-picked lists. Someone adds products one by one, and the list goes stale as soon

0dev.to C#