SQLite beat raw files 3.3x on blob reads. I measured it
SQLite read small blobs 3.3x faster than the filesystem on my machine — 47,481 ops/s against 14,163. Sharding writes across 16 SQLite files bought another 2.7x. Both numbers come from re-running the core claims in Zerodha's Blobasaur post, which hit Hacker News today.
Zerodha built Blobasaur because they store trading-platform UI state — chart layouts, indicator templates, widget arrangements — as JSON blobs from a few KB to several MB, read and written thousands of times a second. Their stack: Rust, the Redis protocol, SQLite as the storage engine, one database file per shard, zstd compression before write. The post makes four testable claims, and I tested all four on a 4-vCPU VM with Python's sqlite3 (3.45.1), zstd level 3, WAL mode, synchronous=NORMAL, batch commits of 50, and 16 concurrent threads.
The four claims, and what I measured
Zerodha's numbers, from their post:
Claim 1: 16 shards write 3.3x faster than 1 shard (7,740 -> 25,324 HSET/s, 64 KiB payloads) Claim 2: reads are flat across shard counts (67,128 -> 70,884 HGET/s) Claim 3: zstd compresses production JSON ~11x on average Claim 4: SQLite is up to 35% faster than the filesystem for small blobs My numbers:
Writes, 64 KiB payload, 4,000 ops, 16 threads: sqlite 1 shard : 2,133 ops/s p50 0.01 ms p99 41.21 ms sqlite 16 shards: 5,805 ops/s p50 0.01 ms p99 66.19 ms -> 2.7x filesystem : 8,689 ops/s p50 1.04 ms p99 22.49 ms Reads, 64 KiB payload, 4,000 ops, 16 threads: sqlite 1 shard : 43,738 ops/s p50 0.01 ms p99 8.07 ms sqlite 16 shards: 26,124 ops/s p50 0.37 ms p99 2.70 ms Reads, ~0.8 KiB blobs, 4,000 ops, 16 threads, two runs: sqlite 1 shard: 47,481 / 29,656 ops/s p50 0.004 / 0.029 ms filesystem : 14,163 / 12,083 ops/s p50 0.701 / 0.824 ms -> 2.5-3.3x Writes, ~2 KiB blobs, 4,000 ops, 16 threads: sqlite 1 shard: 5,256 ops/s filesystem : 12,241 ops/s Writes, 1.5 MB payload, 200 ops, 8 threads: sqlite 1 shard: 128 ops/s, 196 MB/s raw throughput, p99 1,135 ms Claim 1 holds in direction, not in magnitude: 2.7x on my box versus their 3.3x. The gap is easy to explain and worth stating plainly — they ran Rust on a 16-core laptop with tmpfs; I ran Python on 4 vCPUs with a real disk. The mechanism is the same either way: WAL mode allows one writer per database file, so more files means more concurrent writers. The p99s tell the rest of the story: median write latency is 0.01 ms in both setups, but the tail is all lock stalls — 41 ms at 1 shard, 66 ms at 16.
Claim 2 did not reproduce for me. Reads got slower with 16 shards on 64 KiB payloads: 43,738 down to 26,124 ops/s. WAL already serves concurrent readers without contention, so sharding adds connection overhead and buys nothing on the read path — at least on a 4-vCPU box with the page cache hot. Zerodha's reads were flat, mine tilted the other way; either way, sharding is a write-side optimization, not a read-side one.
The filesystem beat SQLite on writes at every size I tried — 8,689 vs 2,133 ops/s at 64 KiB, 12,241 vs 5,256 at 2 KiB. That is not a contradiction of the famous sqlite.org claim, because that claim is about reads. Unsynced file writes are nearly free: the kernel buffers them and goes home. SQLite pays for a WAL append and a commit per batch. If your workload is write-heavy and you don't need transactions, the filesystem is still faster. The honest version of the claim is "SQLite reads small blobs faster than files; it writes them slower."
Claim 4 held, and then some. On sub-1 KiB blobs SQLite read 2.5–3.3x faster than one-file-per-blob across two runs — well past sqlite.org's "up to 35% faster." The p50 gap is the whole story: 0.004–0.029 ms against 0.70–0.82 ms. Opening a file costs a syscall, a path lookup, and an inode; reading a row out of a B-tree that's already in the page cache costs almost nothing. The filesystem pays the open cost on every single read, and at 4,000 random reads that dominates everything.
Compression: the ratio is a property of your data
zstd level 3 on my synthetic chart-layout JSON: 2,190 bytes -> 709 bytes 3.1x 70,656 bytes -> 21,572 bytes 3.3x 1,522,372 bytes -> 484,127 bytes 3.1x Same widget template repeated 400x (177,627 bytes): 177,627 bytes -> 248 bytes 716.2x My synthetic JSON compresses 3.1–3.3x; the same data with repetition compresses 716x. Zerodha's production average of 11x sits between those two poles, which is exactly where it should sit: their real layouts repeat settings blocks, indicator definitions, and symbol names far more than my random floats do. The 11x is not a codec guarantee. If your blobs are mostly random coordinates and timestamps, expect ~3x; if they're templated UI state, expect far more. Either way, compression is what makes the 100 KiB crossover work in SQLite's favor — a 64 KiB JSON blob lands in the database as ~21 KiB, safely under the size where raw files start winning.
The 1.5 MB write test is the cautionary footnote: 128 ops/s with a p99 of 1,135 ms. One writer per shard means a single multi-megabyte write can stall every other writer queued behind it. Zerodha's answer is more shards and smaller payloads; the number to watch in your own deployment is p99 write latency, not throughput.
What I would actually take from this
The architecture is sound and the mechanism checks out: shard to parallelize the single-writer lock, compress to stay under the size where SQLite beats files, and accept that the filesystem still wins pure writes. The two claims that matter most survived contact with my machine — sharding multiplies write throughput, and SQLite reads small blobs several times faster than files. The one that needs a footnote is compression: 11x is a fact about Zerodha's data, not about zstd.
The core of the benchmark, for anyone who wants to argue with it:
import sqlite3, threading, time def worker(tid, shard_path, keys, comp, batch=50): # one connection per thread: sqlite3 connections are not # safe for concurrent use from multiple threads conn = sqlite3.connect(shard_path, check_same_thread=False) conn.execute("PRAGMA journal_mode=WAL") conn.execute("PRAGMA synchronous=NORMAL") cur = conn.cursor() for i, key in enumerate(keys): cur.execute("INSERT OR REPLACE INTO blobs VALUES (?,?,NULL)", (key, comp)) if i % batch == 0: conn.commit() conn.commit() Limitations, stated plainly: Python, not Rust; 4 vCPUs, not 16 cores; data on a real disk, not tmpfs; reads served from a hot page cache (no root to drop caches, so no cold-cache numbers); synthetic JSON, not production layouts. The direction of every result matches Zerodha's; the magnitudes are mine.
Sources: Zerodha Tech, "Distributed, high-performance blob storage on SQLite" (Nov 19, 2026); sqlite.org, "35% Faster Than The Filesystem". All benchmarks run locally Nov 19, 2026: Python 3.12, sqlite 3.45.1, zstandard 0.25.0, 4-vCPU VM.