I maintain YouTube Hider, an open source extension that hides watched videos, Shorts and other clutter from YouTube. It has a Channel Whitelist and a Channel Blacklist, and like most extensions I stored them in chrome.storage.sync, so they follow the user to every device.
It worked fine, until it didn't. Past a few hundred channels, a newly added channel looked saved, stayed on screen, and then disappeared on the next page load. No error message, no warning, nothing in the console of a production build.
This post is about why that happens, and the five changes that took the list from 499 channels to over 10,000 in the same storage area. Every number below is measured on the extension's real code.
The quotas nobody reads until it is too late
chrome.storage.sync has more limits than its 100 KB headline suggests:
| Limit | Value |
|---|---|
Total size (QUOTA_BYTES) | 102,400 bytes |
Size of a single item (QUOTA_BYTES_PER_ITEM) | 8,192 bytes |
Number of items (MAX_ITEMS) | 512 |
| Writes per minute / per hour | 120 / 1,800 |
The one that bites is the second. An item's size is its key plus the JSON.stringify of its value, so a list stored as one array in one key can never grow past 8 KB, no matter how empty the rest of the 100 KB is.
My lists were exactly that: one key per list, an array of channel paths like /@veritasium. With an average handle of 13 characters, each entry costs about 16.5 bytes once JSON adds quotes and commas. That is 499 channels before the item is full.
Why nobody saw an error
When a set call exceeds the quota, Chrome does not throw. It reports the failure through chrome.runtime.lastError in the callback, or rejects the promise if you use the promise form. My storage helper did read it:
function safeStorageSet(area, data) { try { chrome.storage[area].set(data, () => { if (chrome.runtime.lastError) { logger.warn('Storage set failed:', chrome.runtime.lastError.message); } }); } catch (e) { logger.warn('Storage unavailable:', e); } } But logger only prints in development builds. Meanwhile the in-memory state had already been updated, so the UI showed the new channel. The write failed, the page kept the illusion, and the next reload read the old list back from storage.
If you take one thing from this post: check the result of every storage.sync write that grows with user input, and tell the user when it fails. A silent quota error is indistinguishable from data loss.
Change 1: make storage.local the source of truth
storage.local has a 10 MB quota and no per-item limit. So the lists now live there, and sync became a mirror instead of the database:
- Content scripts, the popup and the settings page read and write the lists only in
storage.local. - The service worker watches those keys and mirrors them into
storage.syncas one encoded payload. - It also watches
storage.syncand brings other devices' edits back intolocal.
The immediate effect: a list can grow without limit on the device where it is edited, and only the synced copy is bounded. Hitting the sync ceiling no longer means losing a channel. More on that in change 5.
Change 2: split the payload into chunks that cannot be mixed up
To get past 8 KB per item, the encoded payload is split into chunks of 8,000 characters, each in its own key, plus an index key that says how to put them back together.
The naive version of this has a nasty failure mode. Chrome sync resolves conflicts per key. If two devices write at the same time, one device's chunk 0 can win while the other device's chunk 1 wins, and you end up decoding half of one list glued to half of another.
The fix is to put a generation id in every chunk key:
function getChannelListChunkKey(gen, index) { return 'chl.' + gen + '.' + index; } A write produces chl.<gen>.0, chl.<gen>.1, ... and an index pointing to that generation:
items.channelListsIndex = { v: 2, gen, p: parent, n: count, len: encoded.length, ts: now }; Two concurrent writes now use different keys for every chunk, so they can only disagree on the single index key, and whichever index wins points to a complete set of chunks. The reader also checks that the chunk count and total length match the index before decoding, and reports the payload as incomplete otherwise. Old chunks are removed after the new ones are written, or before, only when both generations would not fit in the quota at the same time.
Change 3: merge without ever losing a channel
Mirroring between devices means merging, and a careless merge is how sync features delete people's data. Two rules keep this one safe.
Removals are applied only when they are trustworthy. The index carries p, the generation the write was derived from. A device applies another device's removals only if that p equals the generation it last synced itself, which proves the remote write started from the same base. Then it is a proper three-way merge:
function mergeListEntries(listKey, local, base, remote) { const toKey = entry => listEntryKey(listKey, entry); const baseKeys = new Set(base ? base.map(toKey) : []); const remoteKeys = new Set(remote.map(toKey)); const removed = base ? new Set(base.map(toKey).filter(key => !remoteKeys.has(key))) : new Set(); const kept = local.filter(entry => !removed.has(toKey(entry))); const keptKeys = new Set(kept.map(toKey)); const added = remote.filter(entry => !baseKeys.has(toKey(entry)) && !keptKeys.has(toKey(entry))); return normalizeListEntries(listKey, [...kept, ...added]); } Without a common base, it degrades to a union. The worst case is a channel the user removed coming back, never a channel the user added going missing.
A device with nothing synced and empty lists never writes. A fresh install has empty lists, and its storage.sync may not have downloaded yet. If it published, it would overwrite the real lists with nothing.
One more trap: the synced copy is sorted (you will see why in a moment) while the local one keeps insertion order. Comparing them with an order-sensitive check makes two devices push to each other forever, so list equality compares sorted entries.
Change 4: make every entry cheaper
Chunking raised the ceiling from 8 KB to 100 KB. The rest of the gain comes from spending fewer bytes per channel. Here is the same list of 5,000 random handles at each step:
| Step | Bytes per channel |
|---|---|
| JSON array | 16.45 |
| Front-coded text | 11.70 |
| Front-coded, then deflated | 7.78 |
| v1 on the wire: JSON, deflate, base64 | 11.95 |
| v2 on the wire: front-coded, deflate, base91 | 9.57 |
Drop what every entry repeats
Almost every entry starts with /@. The serializer drops it, writes /channel/UC... ids as #UC..., and keeps anything else verbatim behind a !, so the format stays lossless.
Front coding
Sort the list, then store each entry as "how many characters it shares with the previous one" plus the rest. The shared length is a single character, offset from ! (0x21), so it is printable and never collides with the separators:
function frontCodeEntries(entries) { const sorted = [...entries].sort(); let previous = ''; return sorted .map(entry => { const shared = sharedPrefixLength(previous, entry); previous = entry; return String.fromCharCode(0x21 + shared) + entry.slice(shared); }) .join('\n'); } For /@verge, /@veritasium, /@vsauce, /@vsauce2 the output is:
!verge $itasium "sauce '2 $ means "reuse 3 characters" (ver), ' means "reuse 6" (vsauce). Two details matter: the shared length is capped so the marker stays a single printable character, and it never ends in the middle of a UTF-16 surrogate pair, or a handle with an emoji would come back corrupted.
Deflate, without a library
CompressionStream is available in service workers, so there is no need to bundle a compression library:
async function pipeBytes(bytes, transform) { const stream = new Blob([bytes]).stream().pipeThrough(transform); return new Uint8Array(await new Response(stream).arrayBuffer()); } const compressed = await pipeBytes(new TextEncoder().encode(text), new CompressionStream('deflate-raw')); Front coding before deflate still pays off: deflate finds repeated substrings, but it cannot exploit sort order the way front coding does. 8.96 bytes per channel for deflated JSON against 7.78 for deflated front-coded text.
base91 instead of base64, with a twist
storage.sync stores JSON, so binary data has to become a string. base64 inflates bytes by 33%. basE91 inflates them by roughly 23%.
The twist is that the quota counts JSON.stringify(value). A standard base91 alphabet contains " and \, and JSON.stringify escapes each of them into two characters, eating part of the gain. So the alphabet here is every printable ASCII character from ! to ~ except ", \ and the backtick: exactly 91 characters, none of which JSON.stringify ever escapes.
const BASE91_ALPHABET = (() => { let alphabet = ''; for (let code = 0x21; code <= 0x7e; code++) { const char = String.fromCharCode(code); if (char !== '"' && char !== '\\' && char !== '`') alphabet += char; } return alphabet; })(); The result
How many random handles fit in storage.sync, next to the extension's other settings, with 4 KB kept free for them:
| Format | Channels that fit |
|---|---|
| One JSON array per key | 499 |
| v1: chunked, JSON, deflate, base64 | 8,211 |
| v2: chunked, front coding, deflate, base91 | 10,483 |
Random handles are the worst case: they share few prefixes and compress poorly. Real handles share far more, and across the four lists the extension now syncs (two channel lists and two lists of title words) the measured capacity is roughly 12,000 to 14,000 entries.
Change 5: when it still does not fit, pause instead of refusing
Every quota has an end. What matters is what happens there.
Before writing, the service worker plans the write: it adds up the bytes of every other key, the new chunks and the stale ones, and checks the result against the quota minus a reserve for the other settings. If the lists do not fit, it does not write, and it does not refuse the channel either:
- the lists keep working and growing in
storage.localon that device; - other devices keep the last version that fit;
- the UI shows a "Sync paused" notice that starts with the most important fact: nothing is lost;
- the last synced content is kept as the merge base, so edits made elsewhere in the meantime still merge three-way;
- sync resumes on its own once the lists are small enough again.
Versioning the format
Changing an on-disk format that several devices share needs one more rule: the index carries a format version (v: 2), new code reads both versions and rewrites v1 as v2, and a device that finds a version it cannot read stops writing rather than overwriting it. An old device meeting a newer format shows its sync as outdated until the browser updates it, instead of replacing everyone's lists with its own.
Checklist
If your extension syncs anything the user builds up over time:
- Treat 8 KB per item as the real limit, not 100 KB.
- Check every write's result and surface failures to the user.
- Keep the data in
storage.localand mirror it intostorage.sync. - Put a generation id in chunk keys so concurrent writes cannot be spliced together.
- Apply remote removals only from a known common base; otherwise union.
- Never let a fresh, empty install publish over existing data.
- Measure bytes per entry. Sorting, front coding, deflate and a JSON-safe base91 add up to a lot.
- Plan for the full quota: pause and explain, never drop data silently.
The whole implementation is pure functions with no chrome.* calls, so it runs and is tested under Node. It is on GitHub, in channel-list-sync.js and its tests. If you have solved the same problem differently, I would like to hear how.

