A Hash That Collided on Every Name and a 25 MB Allocator Trap: What 4 Tracks of Speed Demon 2026 Taught Us About OS, Cache Hierarchy, and AI

TL;DR: For Speed Demon 2026 by Hackathon Raptors, our team (cONFIG) rewrote four deliberately slow Python reference implementations in zero-dependency Rust and C++23:

• a 3.1M-line log aggregator (Track A, 32.05 s → 0.034 s, 942×)
• a 50,000-query boolean and phrase search engine over 256 MB of text (Track B, 56.35 s → 0.252 s, 223×)
• a 3.32M-row hostile CSV join (Track C, 14.44 s → 0.052 s, 277×)
• a 5M-edge shortest-path solver (Track D, 72.69 s → 0.456 s, 159×)

This is the story of the mistakes that cost us the most time: a hash function that passed every SHA-256 gate while routing 100% of customer names into a single L1 slot, an munmap call that made memory no smaller and wall time worse, Golden Cove L1/L2/L3 cache sizing across 12 hyperthreads, and why golfing line count with AI traps you in a local optimum until you do the hardware and scoring-function math yourself.

Repository: https://github.com/Yash-vs9/cOnfig-tracks

Where All Four Tracks Ended Up

We entered all four tracks with zero external dependencies (D = 1.00), targeting the Falkenstein scoring box (fsn1): Intel Core i5-13500, 6 Golden Cove P-cores / 12 threads pinned via --cpuset-cpus="0-11", turbo disabled at a fixed 2.50 GHz, 48 KiB L1d and 1.25 MiB L2 per P-core, 24 MiB shared L3.

The final numbers look clean in a table. Getting there was a sequence of wrong assumptions that the Falkenstein box corrected one by one.

1. The Hash That Passed Every Gate and Missed the L1 Cache 99.98% of the Time (Track C)

Track C asks you to join records.csv (3.32M rows) against regions.csv and emit per-region aggregates, including the exact count of distinct customer_names per region (dn).

To avoid cross-core atomic contention on every row, we put a thread-local 2-way L1 filter tl_cache[2048][2] (32 KB per thread) in front of a lock-free global hash table (name_keys[1 << 18] + a 256-bit region bitset name_bits[1 << 18][4]). If (name_hash, region_id) was already in the thread's tl_cache, we skipped the global table entirely.

Here is the exact hash_bytes function we shipped in commit 1e59383 (0.052 s, 15 MB, 71.344 pts):

static inline uint64_t hash_bytes(const char *s, uint32_t len) { uint64_t h1 = len ^ 0x811c9dc5ULL, h2 = (uint64_t)len * 0x9e3779b97f4a7c15ULL, w = 0, w1 = 0; if (__builtin_expect(len - 8U <= 8U, 1)) { memcpy(&w, s, 8), memcpy(&w1, s + len - 8, 8); h1 = _mm_crc32_u64(_mm_crc32_u64(h1, w), w1); h2 = ((h2 ^ w) * 0x9e3779b97f4a7c15ULL ^ w1) * 0x9e3779b97f4a7c15ULL; return ((h1 << 32) ^ h2) | 0x100ULL; } // ... } 

And inside the hot loop, we indexed the thread-local L1 filter using the low 11 bits of nh:

uint32_t tl_idx = (uint32_t)nh & 2047; 

It compiled cleanly. It passed every correctness check with an exact SHA-256 match. And when we shrunk the global table in commit 282a0d6 (10 MB), our time actually regressed from 0.052 s to 0.066 s.

Why? We finally dumped the raw hex output of hash_bytes across customer names from the generator ("Person_1001", "Person_1002", "Person_2500", "Person_9999"):

Every 11-byte "Person_XXXX" string had the exact same lower 32 bits (0x12cced30) and hashed to tl_idx = 1328.

Look at what happened in the registers:

• For "Person_1001" (len = 11), w (first 8 bytes) is "Person_1" and w1 (last 8 bytes at s + 3) is "son_1001".
• On little-endian x86-64, "son_" (0x5f6e6f73) occupies bits 0..31 of w1, while the digits "1001" occupy bits 32..63.
• Modular 64-bit multiplication (mod 2^64) shifts bits strictly upward. Bits 32..63 of w1 never flow downward into bits 0..31 of h2!
• Meanwhile, (h1 << 32) put the hardware CRC32 entirely into the upper 32 bits.

Because the upper 32 bits (h1) differed, the global hash table (slot = (nh ^ (nh >> 32)) & HT_MASK) still produced the right answer, so every correctness test passed. But tl_idx = (uint32_t)nh & 2047 looked only at the bottom 11 bits, where "Person_" and "son_" were constant. Out of 2,048 slots in our L1 cache, 2,047 slots were permanently empty, and all 9,000+ names fought over the 2 ways of slot 1328, giving us a 99.98% L1 cache miss rate.

Replacing the h2 multiply chain with a second _mm_crc32_u64 + a right-shift avalanche (h ^ (h >> 29)) and an 8-miss software-pipelined _mm_prefetch ring buffer (to hide the ~120-cycle L3 latency across 8 misses instead of probing 35 cycles after prefetching on the same row) cut our sandbox join time from 74.1 ms to 50.9 ms.

2. The Rewrite That Made Track A Slower: munmap vs. glibc ptmalloc2 Arenas

We originally wrote Track A in C++ and switched to zero-dependency Rust after seeing Rust dominate the top of the Track A board. Our first Rust push (1862874, run-2f56a6016af5) ran in 0.0534 s with 42.0 MB of memory (69.88 pts). Adding an 8-byte SWAR latency parser and compacting lines for rustfmt (72.30 pts) and vectorizing the endpoint space scan with AVX2 (61516e3, 75.06 pts) brought our time down to 0.038 s, but our memory stayed stubbornly stuck at 42 to 44 MB, while the leaders were at 18 to 20 MB.

Our static .bss struct (SharedData) was only ~20.8 MB. Where was the extra 23 MB coming from?

Our wrong guess: We assumed the kernel was charging part of the 239 MB mmap of /data/requests.log against our container's memory counter. So we added hr_buf and ep_buf to copy the interned hour and endpoint strings out of the file mapping and called munmap(bp, sz) right after the scan phase finished.

What the box said: Memory did not drop at all, and wall time got worse.

The harness pre-warms /data/requests.log into the host page cache with vmtouch -t before the container starts. Under cgroups v2, mapping already-resident host page-cache pages with PROT_READ + MAP_SHARED charges 0 bytes of anonymous RSS to the container.

Worse, cgroups v2 records memory.peak, the lifetime high-water mark of the container. Calling munmap after the scan phase cannot lower a peak that already happened; all it did was force Thread 0 to grab down_write(&mm->mmap_lock) in the kernel and tear down 61,184 page table entries while the clock was running.

The actual culprit (/proc/self/smaps):

When we inspected /proc/self/smaps in our sandbox, two OS behaviors jumped out:

• In Phase 2 (computing p50/p95/p99 per hour), we used a static buffer l: [[u32; 49152]; 12] when an hour had <= 49,152 requests, with a fallback v_lats.resize(tot, 0) (Vec<u32>) if a peak hour exceeded 49,152. Whenever a peak hour crossed 49,152 requests across 12 threads, Vec allocated and freed ~200 KB blocks concurrently. In glibc, freeing medium allocations across threads dynamically raises M_MMAP_THRESHOLD and creates 12 retained per-thread ptmalloc2 heap arenas (~18 MB of dirty anonymous pages that glibc never returns to the OS).
• std::thread::scope spawns threads with Linux's default 2 MB stack reservation.

Widening the static .bss buffer to 75,000 per thread (deleting Vec completely so malloc is never called in the worker loop), locking glibc with mallopt(M_ARENA_MAX, 1) and mallopt(M_TRIM_THRESHOLD, 0), and spawning threads via Builder::new().stack_size(65536) in commit 282a0d6 immediately dropped memory from 44 MB to 35 MB and brought wall time down to 0.034 s (76.695 pts).

3. CPU Microarchitecture & the Cache Hierarchy: Fitting Into Golden Cove L1/L2/L3

Modern CPUs are starving for data, not compute. On the Falkenstein runner (Intel Core i5-13500, 6 Golden Cove P-cores / 12 hyperthreads pinned via --cpuset-cpus="0-11"), an L1 data cache hit takes ~4 cycles, an L2 hit takes ~14 cycles, an L3 hit takes ~45–60 cycles, and a main-memory DRAM miss costs ~200+ cycles.

Because two sibling hyperthreads (cpu 0 and cpu 1) share each physical P-core, they split that core's 48 KiB L1d cache (24 KiB effective per thread) and 1.25 MiB L2 cache (640 KiB effective per thread), while all 12 threads share the 24 MiB L3. Once we stopped treating memory as a flat array and started sizing every data structure against those exact cache boundaries, bottlenecks started making sense across all four tracks:

Domain Compression into L1/L2 (Track C): Why build a std::unordered_map when the key domain is tiny? In Track C, 3-letter uppercase country codes only have 26^3 = 17,576 possible combinations, so we mapped c0..c2 directly into uint16_t c_to_dense[17577] (35.1 KiB, which fits inside L1/L2). Likewise, 6-digit region_ids (0..999,999) fit in a flat uint8_t reg_lut[1000000] (976 KiB, which fits entirely inside each P-core's 1.25 MiB L2). Every relational foreign-key lookup became a single L2-resident byte load (reg_lut[reg]) with zero pointer chasing.

When Hyperthreads Evict Each Other's L1d (Track A & Track C): In Track A, each thread had a 16 KiB direct-mapped stack cache lc: [u64; 2048] (2048 × 8 B) to bypass global string interning. Two hyperthreads on the same P-core needed 32 KiB of lc alone, leaving only 16 KiB of the 48 KiB L1d for stack frames, requests.log streaming lines, and output chunks. Similarly, in Track C, a 32 KiB per-thread tl_cache[2048][2] meant two sibling hyperthreads needed 64 KiB (16 KiB larger than L1d!), constantly evicting hot lines down to L2.

Eliminating MESI Cache-Line Bouncing (False Sharing): When 12 threads increment adjacent counters in the same 64-byte cache line, the cores spend all their time invalidating each other's L1 caches over the ring bus. Sharding state into per-thread buffers (T[12][256] in Track C, and 64-byte aligned #[repr(C, align(64))] blocks in Track A) turned cross-core atomic wars into private L1 writes merged once at the end.

4. Track B (Search) & Track D (Graph Traversal): Designing Around the Queries You Can Already See

While Track A and Track C were battles against SIMD byte parsing and cache-line contention, Track B and Track D taught us a different lesson: never build a general-purpose index when the query workload is sitting right there on disk.

Track B: Cutting 2,199 MB to 75 MB by Indexing Only What the 50,000 Queries Ask For

Track B hands you a 256 MB text corpus and queries.txt: 50,000 queries split between BOOL (AND / OR / NOT, 62%) and PHRASE "..." (exact adjacent token positions, 38%). The Python reference takes 56.35 s and explodes to 2,199 MB of RAM because storing occurrence positions for every token in a 256 MB corpus eats gigabytes of memory.

Because build and query execution are timed together in a single run, and queries.txt is readable from millisecond zero, building a full positional inverted index over the entire vocabulary is pure waste:

• By parsing queries.txt first, we know the exact set of terms that ever appear in any boolean query, and the even smaller subset of terms that ever participate in a PHRASE "..." query.
• Words that never appear in queries.txt don't need postings lists at all, and words that only appear in BOOL queries only need document IDs (or bitsets), never token positions.

Combining query-driven vocabulary filtering with parallel 12-thread corpus chunking and smallest-first posting-list intersection took Track B from 0.314 s (62.16 pts in 61516e3) down to 0.252 s (223× faster than reference) and 75 MB (29.3× less memory) in commit 282a0d6 (63.751 pts).

Track D: 5 Million Directed Edges in 456 ms (282a0d6)

Track D gives you graph.txt (92 MB: 1,740,512 nodes and 5,000,000 directed weighted edges u v w, unsorted) and 2,000 source-target shortest-path queries (reference: 72.69 s, 893 MB).

Here, heavy preprocessing (like full Contraction Hierarchies) doesn't have enough queries to pay back its build time, because the 2,000 queries are local with bounded search depth. Nearly the entire wall-clock battle is:

• Parallel edge-list ingestion (mmap + SWAR integer parsing): Parsing 15 million integers (u, v, w × 5M) across 12 threads without strtol or scanf.
• Flat Compressed Sparse Row (CSR) memory layout: Packing outgoing edges into a contiguous (target, weight) array indexed by head[u] so edge relaxations hit sequential cache lines instead of chasing pointers across an 893 MB heap.
• Parallel query execution with generation-stamped distance arrays: Running the 2,000 queries in parallel across all 12 threads using per-thread (dist, version) tables so resetting Dijkstra between queries is a single version += 1 increment rather than a 7 MB memset (1,740,512 × 4 B × 2,000 queries = 13.9 GB of zeroing avoided!).

That brought Track D to 0.456 s (159× speedup) and 83 MB (56.262 pts).

5. The 76.7 ms CSV Quote Scan Replaced by a 0.06 ms Automaton (Track C)

One optimization on Track C deserves its own callout because it turned our slowest serial bottleneck into a 60-microsecond no-op.

records.csv (204 MB, 3.32M rows) has ~404,713 newlines inside quoted note fields. If you split the file at t * sz / 12 for 12 threads (t = 1..11) and advance to the next \n, you have a 50/50 chance of starting Thread t in the middle of a multi-line note and corrupting the parse.

Our first version ran std::count across the 12 chunks to compute quote parity (odd vs even number of " characters before cs[t]). Scanning 204 MB just to count quotes took 76.71 ms in our local sandbox, more time than the entire 12-thread join!

Then we looked at the grammar of records.csv. An embedded newline inside note is followed by free text, whereas a real record end is always preceded by ,region_id,[-]DDD.CC\r?\n and followed by (\xEF\xBB\xBF)?[0-9]+,:

static inline bool is_rec_end(const char *base, size_t sz, const char *nl) { const char *p = nl - (nl > base && nl[-1] == '\r'); if (p - base < 6 || p[-3] != '.' || p[-1] < '0' || p[-1] > '9' || p[-2] < '0' || p[-2] > '9') return false; p -= 3; int d_am = 0, d_reg = 0, d_id = 0; while (p > base && p[-1] >= '0' && p[-1] <= '9') --p, ++d_am; if (!d_am) return false; if (p > base && p[-1] == '-') --p; if (p <= base || p[-1] != ',') return false; --p; while (p > base && p[-1] >= '0' && p[-1] <= '9') --p, ++d_reg; if (!d_reg || p <= base || p[-1] != ',') return false; const char *nx = nl + 1; if (nx >= base + sz) return true; if (nx + 3 <= base + sz && (uint8_t)nx[0] == 0xEF && (uint8_t)nx[1] == 0xBB && (uint8_t)nx[2] == 0xBF) nx += 3; while (nx < base + sz && *nx >= '0' && *nx <= '9') ++nx, ++d_id; return d_id > 0 && nx < base + sz && *nx == ','; } 

Instead of scanning 204 MB from byte 0, each thread jumps directly to t * sz / 12 and tests is_rec_end on the next 1–2 newlines. Chunk alignment dropped from 76.71 ms to 0.061 ms (1,257× faster).

6. Scoring-Function Engineering & Breaking Out of the AI Local-Optimum Loop

We finished around the top 40 overall. Halfway through the event, when the organizers shared that the top 20 entries across each track collectively used 49 to 73 distinct optimization techniques, we realized why our scores on Track A (~76.7 pts) and Track C (~71.3 pts) had started inching up by only +0.3 points per push.

We had fallen into a classic local-optimum loop, compounded by how we were prompting AI and how we were treating the objective function.

The Math of the Objective Function (Why Golfing Lines Was a Trap)

Look at the exponents in the composite factor K:

K = (T_ref / T)^0.50 * (M_ref / M)^0.25 * (S_ref / S)^0.15 * D^0.10 

Because wall time T carries a 0.50 exponent while S (post-rustfmt/clang-format scc line count) carries only a 0.15 exponent:

• Compressing code from 330 lines to 250 lines (-24% lines) multiplies K by (330 / 250)^0.15 = 1.0426 (+4.2%).
• Worse, scc counts lines after standard formatting (rustfmt / clang-format), so cramming multiple statements onto one line gets expanded right back out by the formatter while making the code harder to reason about.
• Meanwhile, spending +100 extra lines (330 → 430 lines, costing only -3.9% on the S factor) to implement a faster algorithm, like two-level radix bucketing on Track A, vocabulary-filtered bitset intersections on Track B, or bidirectional radix-heap search on Track D, that cuts T by 1.8× multiplies K by 1.8^0.50 × (330 / 430)^0.15 = 1.3416 × 0.961 = 1.289 (+28.9%).
• And build-time optimizations inside the Dockerfile, such as Profile-Guided Optimization (-Cprofile-generate / llvm-profdata / -fprofile-use) and BOLT post-link binary layout optimization, deliver 8–15% wall-time reductions at zero scc line cost.

Why AI Polishes Local Optima, and Why Systems Clarity Still Wins

When you give an AI coding assistant a 300-line file and say "optimize this and keep the line count low", the AI will relentlessly polish the micro-syntax of the architecture you already have:

• It will golf if statements into .then(|| ...) to save a few lines.
• It will tweak loop unrolling factors and register variables.
• And meanwhile, it will leave a 16 KiB direct-mapped stack array colliding on 1,842 Zipf endpoints, or leave 800,000 5xx log lines executing cross-core lock xadd atomic instructions ((*s).c[hid * 2048 + eid].fetch_add(1) instead of bit-packing is_5xx into bit 11 of the thread-local request word).

Four Lessons from Speed Demon 2026

1. Language selection sets your ceiling and your floor: Choosing zero-dependency Rust (D = 1.00) for Tracks A, B, and D and C++23 with OpenMP for Track C gave us fearless 12-thread parallelism and direct SIMD access, provided you tame runtime defaults like glibc arenas and 2 MB thread stacks.
2. Design for the cache hierarchy first, instructions second: Knowing the exact L1d (48 KiB), L2 (1.25 MiB), and L3 (24 MiB) budgets of the target CPU lets you compress key domains (like c_to_dense[17577] and reg_lut[1000000]) so lookups never touch DRAM.
3. Optimize for the real objective function, not code golf: Never trade a 0.50-weight metric (wall-clock time) or a 0.25-weight metric (peak RSS) for a 0.15-weight metric (line count). When progress stalls at +0.3 points per push, stop micro-golfing and change the data structure.
4. AI writes the SIMD; your OS and hardware clarity chooses the architecture: An LLM can emit _mm256_movemask_epi8 or a SWAR multiplier in seconds. What it cannot do on its own is tell you that 2 hyperthreads sharing a 48 KiB Golden Cove L1d cache are evicting each other's working sets, that glibc ptmalloc2 is retaining 12 heap arenas behind your back, or that little-endian 64-bit multiplication never shifts high-bit entropy into the low 32 bits.

Built by team cONFIG for Speed Demon 2026, the 72-hour performance engineering championship organized by Hackathon Raptors. All four track solutions (track-a, track-b, track-c, track-d) and optimization logs are in our repository: https://github.com/Yash-vs9/cOnfig-tracks.