Content-addressed keys for transformed images in object storage

Name every transformed image after what made it: a SHA-256 of the source bytes, plus a hash of the canonical options and the encoder version. The same input then always lands on the same object key, so a cache check is one read. On my Mac a repeat 640 px AVIF from a 9.8 MB camera JPEG dropped from 296.9 ms to 5.1 ms.

Why not key the cache on the URL or the filename?

A URL names where an image came from, not what it is. Two users upload the same IMG_0001.jpg with different pixels. One user uploads the same photo twice under two names. A key built from the name gets the first case wrong and misses the second.

A key built from the bytes gets both right. Identical bytes share one hash. Different bytes never do, in any practical sense.

The options matter just as much. A 640 px AVIF at quality 50 is a different object from a 1200 px WebP of the same photo. So the key has two halves: the source and the recipe.

What goes into the key?

Three things:

1. The source hash. SHA-256 of the uploaded bytes, before any decode.
2. The canonical options. Width, format, quality, and anything else that changes the output, serialized with sorted keys.
3. The encoder version. A sharp or libvips upgrade can change the bytes it writes. Put the version in the recipe, and an upgrade gets fresh keys instead of a mix of old and new output.

The canonical step is the one people skip. JSON.stringify keeps insertion order, so two option objects with the same values can hash differently:

const a = { format: 'avif', quality: 50, width: 640 }; const b = { width: 640, quality: 50, format: 'avif' }; const canon = (o) => JSON.stringify(Object.keys(o).sort().reduce((acc, k) => ((acc[k] = o[k]), acc), {})); console.log(JSON.stringify(a) === JSON.stringify(b), canon(a) === canon(b)); 

false true 

Note: Without the sorted keys, the same request from two code paths writes two copies of one image. Nothing fails, so you only notice it on the storage bill.

The code

This is the function I measured. Local disk stands in for object storage; on S3 or R2 the readFile becomes a GetObject (or a HeadObject when you only need to know it exists), and the writeFile a PutObject.

const sharp = require('sharp'); const { createHash } = require('crypto'); const fs = require('fs/promises'); const path = require('path'); const sha = (b) => createHash('sha256').update(b).digest('hex'); const canon = (o) => JSON.stringify(Object.keys(o).sort().reduce((a, k) => ((a[k] = o[k]), a), {})); const STORE = path.join(__dirname, 'store'); async function transformed(src, opts) { const recipe = sha(canon({ ...opts, sharp: sharp.versions.sharp })).slice(0, 16); const key = `${sha(src)}/${recipe}.${opts.format}`; const file = path.join(STORE, key); try { return { key, hit: true, body: await fs.readFile(file) }; } catch {} const body = await sharp(src) .resize({ width: opts.width, withoutEnlargement: true }) .toFormat(opts.format, { quality: opts.quality }) .toBuffer(); await fs.mkdir(path.dirname(file), { recursive: true }); await fs.writeFile(file, body); return { key, hit: false, body }; } 

The source hash is the folder, so every variant of one upload sits under one prefix. Deleting an upload and all its variants is one prefix delete.

How much does a cache hit save?

I ran the camera photo from the Blinkimg site (image-compressor-sample-camera-photo.jpeg, 4964 × 2782, 9,804,300 bytes) through transformed() with { format: 'avif', quality: 50, width: 640 }. Each run cleared the store, called once (a miss), then called again (a hit). Median of 5 runs:

key 7adfa6878004d5238d88... bytes 12558 miss median ms 296.9 hit median ms 5.1 

The hit still pays for hashing 9.8 MB. That is the floor of this design: you cannot know the key without reading the bytes.

Is hashing the source too slow?

No. Here is SHA-256 next to the encode it guards, median of 5 runs each:

On the large file the hash costs about 2% of one encode. On the small file it is noise.

If you already store uploads under their hash, you hash once at upload time and never again. The transform key then needs only the recipe half.

Does sharp give the same bytes for the same input?

It has to, or the key would lie. A content-addressed key promises that the object under it is what that recipe produces. I encoded each file 3 times in parallel with the same settings and compared the SHA-256 of the outputs:

photo.jpg 188259 B sha256 ms 0.08 encode ms 152.5 ratio 1840 out 25770 identical outputs true key 9b6aacdf80cdc900/ccddc4468306a9ce.avif image-compressor-sample-camera-photo.jpeg 9804300 B sha256 ms 4.29 encode ms 215.3 ratio 50 out 12558 identical outputs true key 7adfa6878004d523/ccddc4468306a9ce.avif 

Identical, on one machine and one sharp build. Notice the recipe half (ccddc4468306a9ce) is the same for both photos: same options, same encoder. Only the source half differs.

Different machines are a different question. A libvips built with other SIMD paths could write different bytes for the same recipe. That is fine for a cache: both outputs are valid answers to the recipe. It only matters if you also use the output hash as an integrity check across machines.

What about two workers racing on the same key?

Both miss, both encode, both write. Because the key is content-addressed, both writes carry an equivalent object, so the race wastes CPU and corrupts nothing. If the waste matters, take a short lock in Redis on the key before encoding. S3 also supports conditional writes with If-None-Match: * on PutObject, which makes the second write fail with 412 Precondition Failed instead of overwriting.

Where this pattern stops

• Expiry. A content-addressed object never goes stale, so it never needs invalidation. It does need a lifecycle rule, or the bucket grows forever.
• Privacy. A hash of the source is a stable identifier for that file. Do not put it in public URLs if users would not want two uploads of the same photo linked.
• Metadata. Two files with the same pixels but different EXIF hash differently. Strip metadata before hashing if you want them to share a key.

If you would rather not run the encode side at all, the Blinkimg API returns a signed link to each result, and how we measure explains where its byte counts come from.

How I measured

• Machine: Apple M1 Pro, macOS 15.7.3, on 2026-10-17.
• Node 24.22.0, sharp 0.35.4, libvips 8.18.6.
• Files: packages/engine/tests/fixtures/photo.jpg (800 × 600) and the site's image-compressor-sample-camera-photo.jpeg (4964 × 2782).
• Times are the median of 5 runs, with performance.now().
• The store is a folder on the local SSD standing in for S3 or R2.

What I did not test

I did not run the code against S3 or R2, so a network round trip would replace the 5.1 ms hit on a real bucket. I did not compare outputs across two machines.

I build Blinkimg, an image compressor and converter with a developer API. I wrote this article with help from an AI assistant (Claude), which ran the code on my machine; every number in it comes from those runs.