Your RAG Finds Some of the Answer. I Tested 5 Chunking Strategies From This Month's Papers to See Who Finds All of It.

Most RAG dashboards report a number that sounds like success: "the right document was in the top 5." That is partial recall: at least one relevant passage made it into the context.

Real questions rarely need one passage. They need a rule and the definition it depends on, or a limit and its exception. If retrieval finds one of the two, the model writes a confident answer from half the evidence, and a hit-rate metric scores it as a success.

The metric that catches this is complete-evidence recall. For a question q with evidence spans E(q) and an assembled context C(q):

partial(q) = 1 if ∃ e ∈ E(q) : e ⊆ C(q) complete(q) = 1 if ∀ e ∈ E(q) : e ⊆ C(q) 

Between mid-September and the end of October 2026, several papers attacked this gap from the chunking side. Some optimise chunk boundaries directly, some follow the document's own structure, and some build a graph over chunks. I implemented the core idea of each one in plain JavaScript and added them as a new page to my browser-native Advanced Local RAG demo: the Chunking Lab.

Everything runs in your tab. There is no server, no API key and no model download. Retrieval is BM25 only, so every number in this post is deterministic and reproducible from a URL. A full run (parse, five chunkers, two retrieval modes, all metrics) takes about 11 ms in Node 22.

The research

QESChunker (arXiv 2610.36557, Oct 27) treats chunking as supervised optimisation. Given question–evidence (QE) pairs, human-written or LLM-generated, it picks the chunk boundaries that maximise how much evidence ends up retrievable. Without overlap, the objective decomposes over chunks and dynamic programming finds the exact optimum in polynomial time. With overlap, the objective is monotone submodular. Exact optimisation is then NP-hard, but greedy selection comes with a (1 − 1/√e) approximation guarantee.

StrucChunk (arXiv 2610.26313, Oct 19) uses the structure the author already wrote. It treats a document as a graph of provisions, never cuts across a clause boundary, resolves cross-references ("see Section 4.2") before embedding, and fuses BM25 with dense retrieval through Reciprocal Rank Fusion. On LegalBench-RAG, Char-Recall@8 goes from 23.59% to 48.18%. On one corpus, Recall@5 rises from 0.557 to 0.804, and the authors attribute 88% of that gain to chunking alone.

SpectRAG (arXiv 2610.36991, Oct 27) stops treating chunks as independent. It builds a weighted graph over chunks (adjacency, section hierarchy, shared entities, citations), seeds it from hybrid search, propagates relevance, and assembles a context under a fixed token budget. The goal is a context that is relevant, diverse, connected and sufficient.

The Bottleneck Is Retrieval, Not Reading (arXiv 2610.26951, Oct 18) supplies the diagnosis. Across ten frontier and three open-weight models, retrieval reliably surfaces some of the relevant chunks but not all of the chunks a multi-fact question needs. Chunking also strips the document-level context that resolves references between passages.

Three September papers frame the rest:

• Pre-retrieval Query Clustering for Adaptive Top-k (arXiv 2609.13489) picks retrieval depth per query cluster, reporting over 36% better F1 and 14% fewer tokens on easy clusters. Context size is a per-query variable.
• Parser, Chunking, and Embedding Interactions (arXiv 2609.31660) runs a 54-configuration factorial study and finds parser and chunker choices interact, with no single retriever winning everywhere.
• Re-ranking and Late Interaction Drive Retrieval Quality (arXiv 2609.38473) compares six pipelines over about 464,000 papers and finds the biggest levers after first-stage retrieval. The main demo already has a cross-encoder reranker for that.

The test bed

The corpus is a fictional observatory handbook, deliberately written like a policy document:

Each QE pair stores the question and the exact evidence sentences as strings. The lab maps each evidence string to a character span and to the range of text units it covers, using a binary search over unit start offsets:

const unitAt = pos => { let lo = 0, hi = units.length - 1 while (lo < hi) { const m = (lo + hi + 1) >> 1; if (units[m].start <= pos) lo = m; else hi = m - 1 } return lo } 

The parser splits sentences on ., ! or ? followed by whitespace, so "1.2-metre" and "Section 1.3" stay intact. A provision is any heading at level 3 or deeper, or any numbered x.y heading, plus the sentences under it. Each provision records its heading path, for example 1 Definitions › 1.3 Red-light zone.

Fixed conditions for every strategy: chunk length limit L = 450 characters, context budget B = 120 words, the same BM25 (k1 = 1.2, b = 0.75, stop-word removal and a light suffix stemmer that strips -s, -es, -ed and -ing from tokens longer than four characters), and the same context assembler.

The five chunkers

1. Fixed window

N characters with a 50-character overlap and step L − 50. This is the tutorial default. It cuts mid-word and mid-sentence.

2. Sentence window

The exact algorithm the main demo uses (chunkDocument() in ragEngine.js). It packs whole sentences up to L characters. When a chunk closes, it backtracks whole sentences from the end until the 50-character overlap budget is used, and those sentences start the next chunk.

3. Structure-aware (StrucChunk-style)

One chunk per provision. A chunk never crosses a provision boundary, and a provision longer than L is split on sentence boundaries. Each chunk gets its heading path as a prefix, and every "Section x.y" reference inside it is resolved at index time by appending up to 320 characters of the referenced provision:

for (const m of C.slice(start, end).matchAll(/Section (\d+(?:\.\d+)+)/g)) { const ti = byId.get(m[1]) if (ti === undefined || ti === pi || seen.has(ti)) continue seen.add(ti) refs.push({ id: m[1], prov: ti, text: bodyText(provisions[ti], refChars) }) } 

An indexed chunk then looks like this:

[2 Access and Arrival › 2.1 Gate hours] The lower gate opens at 17:00. It closes to arriving vehicles 30 minutes before the start of the dark window defined in Section 1.1. ... ↳ Section 1.1 Dark window: A dark window is any period of at least 90 minutes between the end of astronomical twilight and moonrise. ... 

The rule and the definition it depends on are now one retrievable unit.

4. QESChunker, exact dynamic programme

Treat the document as a sequence of n text units. A segmentation splits it into contiguous chunks, each at most L characters. The utility of a chunk covering units j..i−1 is:

u(j, i) = Σ_{e ∈ spans} 1[ e ⊆ [j, i) ] + Σ_{q ∈ multi-ev} 1[ ∀ e ∈ E(q) : e ⊆ [j, i) ] − μ (μ = 0.05, a small per-chunk cost) 

The first term rewards evidence sentences kept intact. The second rewards keeping all of a multi-fact question's evidence together. The objective is a sum over chunks, so the optimum satisfies a simple recurrence:

best[0] = 0 best[i] = max_{j < i, span(j, i) ≤ L} best[j] + u(j, i) 

for (let i = 1; i <= n; i++) { for (let j = i - 1; j >= 0; j--) { const span = U[i - 1].end - U[j].start if (span > L && j < i - 1) break const v = best[j] + util(j, i) if (v > best[i]) { best[i] = v; arg[i] = j } } } 

The inner loop stops as soon as a window exceeds L, so the cost is O(n · w · |QE|), where w is the maximum number of units per window. On this document the optimum has an objective of 13.35, and every training evidence span stays intact.

5. QESChunker, overlapping greedy

With overlap allowed, the lab picks a set of possibly overlapping windows under a total index budget of 1.35× the document length. The coverage function counts newly covered evidence spans, newly completed multi-fact questions, and α = 0.02 per newly covered unit, so the whole document gets indexed. It is monotone submodular, so the lab runs cost-benefit greedy:

while (true) { let bestC = null, bestR = 0 for (const c of cands) { if (spent + c.cost > budget) continue const g = gain(c) // marginal coverage given what is already chosen if (g <= 0) continue const r = g / c.cost // benefit per character if (r > bestR) { bestR = r; bestC = c } } if (!bestC) break chosen.push(bestC); spent += bestC.cost // mark spans, questions and units covered by bestC } 

One practical detail mattered. Without a floor on window size, cost-benefit greedy collapses into one-sentence chunks, because a single evidence sentence has the best gain per character. Candidates shorter than 0.5 · L are therefore excluded. Units still uncovered when the budget runs out are packed into gap chunks so they stay searchable.

The two retrieval modes

Flat top-k

Rank chunks by BM25, then fill the word budget in rank order. The assembler skips any chunk that would overflow the budget and any chunk whose tokens are less than 30% new (a near-duplicate filter that matters for overlapping chunkers):

for (const i of order) { const c = chunks[i] if (used + c.words > budget) continue const t = toks(c.text) const novel = t.filter(x => !seen.has(x)).length / Math.max(1, t.length) if (picked.length && novel < 0.3) continue picked.push(i); used += c.words t.forEach(x => seen.add(x)) } 

Note the continue rather than break: a large chunk that does not fit lets a smaller, lower-ranked chunk take the space. That detail explains one of the results below.

Chunk graph (SpectRAG-lite)

Build an undirected weighted graph over chunks with four edge types:

Then run personalised PageRank from the top-3 BM25 seeds, with the restart vector weighted by BM25 score, restart probability 0.5 and 25 iterations:

p₀[i] = s[i] / Σ_seed s for the 3 seeds, 0 elsewhere p ← 0.5 · p₀ + 0.5 · Wᵀ D⁻¹ p (25 iterations) score[i] = 0.6 · s[i]/max(s) + 0.4 · p[i]/max(p) 

The final ranking blends normalised BM25 with normalised PageRank mass and goes through the same budgeted assembler. The only thing that changes between modes is the order in which chunks are offered.

What each chunker builds

Before retrieval, the index itself already says a lot:

Every chunker keeps every evidence sentence intact somewhere, because the sentences are short compared with L = 450. That means all differences below come from findability, not from broken evidence. The columns that differ are granularity, alignment with headings, and whether cross-referenced definitions travel with the rule.

Result 1: Partial recall flatters

Held-out questions, L = 450, B = 120 words:

Every strategy finds something for 69–92% of questions. The default chunkers assemble the whole answer less than a quarter of the time. A pipeline evaluated on partial hits would look healthy while three out of four multi-fact answers were built on incomplete evidence.

The "avg context words" column shows a second effect. The windowed chunkers leave 30–48 words of the 120-word budget unused, because the next-ranked 60–70-word window does not fit. Small chunks pack the budget more efficiently.

Result 2: Structure wins when context is scarce

Provision-sized chunks average 285 characters, against about 400 for the windowed chunkers. At 120 words you can fit two or three provisions but only one large window. When the answer lives in two places, two small, precise pieces beat one big piece that happens to hold half the answer.

The budget sweep (held-out, flat retrieval, L = 450) makes the trade-off visible:

At 40 words, structure-aware is the only strategy above 10%. By 300 words the big-window chunkers have caught up, and by 400 they have passed it, while structure-aware levels off at 77%. With a generous budget, almost any chunker eventually drags the evidence in, because you are sending a large share of the document. The best chunker depends on the budget, and the right budget depends on the query, which is the adaptive top-k paper's argument from the other direction.

The structure-aware plateau at 77% is not a chunking failure. Its remaining misses are retrieval failures (see Result 5).

Chunk-size sweep

Holding B = 120 and varying L shows the same mechanism from the other side:

Two things stand out:

1. Structure-aware is flat at 62–69%. Almost every provision is shorter than 450 characters, so L barely changes its chunks. Its granularity comes from the document, not from a hyperparameter.
2. At L = 800, fixed and sentence windows score 0%. An 800-character window is about 130 words, which is larger than the 120-word budget, so the assembler cannot fit a single chunk. This is the failure you get when chunk size and context budget are tuned by different people.

At L = 250, every chunker gets better and the gap narrows. Smaller chunks behave more like provisions. The cost is a larger index and more fragments for any reader that needs surrounding context.

Result 3: The provably optimal chunker came last

QESChunker's exact DP is optimal for its objective. It still scored 15% on held-out questions and 58% on the twelve questions it was trained on, below plain fixed windows at 75%.

The boundary map shows the mechanism. The utility function rewards evidence sentences that sit fully inside a chunk, and says nothing about the text around them. The cheapest way to keep a rule intact is to cut immediately before it. In a handbook, the line immediately before a rule is its heading, ### 5.2 Emergency kit. The heading ends up at the end of the previous chunk, and the rule starts the next one. Evidence intact: 100%. But a keyword query about "emergency kit" now matches the chunk holding the stranded heading, not the chunk holding the rule. That is why the DP has 0% of chunks starting at a heading.

Ablation: does a heading-path prefix rescue it? The lab can prefix every chunk from every chunker with the heading path of each provision it touches (the "contextual headers" toggle). Result, held-out, flat:

It does not rescue the DP. The prefix tells BM25 which provision a chunk belongs to, but the chunk holding the stranded heading also gets that provision's name in its prefix, so both chunks still compete for the same query. The prefix also spends budget words on every chunk. Only the overlap variant, whose windows already tend to start at headings, gains from it.

This is not a flaw in the paper. QESChunker is evaluated with a fixed hybrid retriever, and dense embeddings are far less sensitive to where a heading lands. It is a lesson for anyone porting the idea: the optimisation objective has to model the retriever you actually run. Intact evidence and findable evidence are different goals, and a chunker only optimises the one you write down. A practical fix is to add a term to u(j, i) that scores whether the training question retrieves the chunk under your real retriever, or to forbid cuts between a heading and its first sentence.

The overlapping variant shows the other side. Overlap lets the greedy pass choose windows that start at the heading and cover the evidence. It reaches 46% held-out and 83% on training questions, without being told the document had structure. On documents without clean headings (transcripts, email threads, scanned PDFs), this is the version I would reach for. The training/held-out gap, 83% against 46%, is the caveat for every supervised chunker: the index is shaped around the questions you thought to ask.

Result 4: The chunk graph helps where definitions are hubs

Switching to graph retrieval moved structure-aware chunking from 62% to 69% and changed little else.

One question shows where that gain comes from:

"Is flash photography allowed on the dome floor?" needs two facts: the camera rule (§4.3, flash is banned everywhere in the red-light zone) and the definition (§1.3, the red-light zone covers the dome floor). Flat BM25 ranks the definition third. At 52 words it no longer fits after the first two picks, so the assembler's continue lets a shorter, useless chunk about the telescope take the space.

In graph mode, the §1.3 definition chunk is a hub: two other provisions cross-reference it, and it shares vocabulary with several more. PageRank mass flows into it from the seeds, its blended score passes the telescope chunk, and the answer becomes complete.

Ablation: turn off cross-reference resolution. Structure-aware graph mode drops from 69% back to 62%, the same as flat. The graph gain comes from the xref edges, which are the most trustworthy edges a document provides.

On windowed chunkers, the graph does not help and sometimes hurts. Sentence windows go from 23% to 15%. Lexical edges between large, mixed-topic windows mostly spread relevance to the neighbours of what was already found. With 40% of the final score coming from PageRank, a neighbour can outrank a correct BM25 pick and take its place in the budget. SpectRAG's own abstract notes trade-offs in early precision and domain dependence, and this toy reproduces that.

Result 5: What none of them fixed

"My 8-year-old is coming. What rule applies on the terrace?" Every strategy misses both evidence sentences. The rule says "children under 10" and the question says "8-year-old". After tokenisation the question is [8-year-old, com, rule, appli, terrace], and the rule sentence shares none of those tokens. Structure-aware chunking gets part of the way: the resolved §1.3 reference brings "terrace" into the children's chunk, so it scores 1.18 instead of zero. But "terrace" also appears in several other provisions, and they outrank it. No chunking strategy can close the gap between "8-year-old" and "under 10". That needs a dense or hybrid retriever, query rewriting, or a reranker that understands the question.

This is why the lab complements the main pipeline rather than replacing it. The main pipeline has MiniLM embeddings, hybrid fusion and a cross-encoder reranker. Chunking decides what can be retrieved. The retriever decides what is retrieved. This month's papers are mostly about the first half of that sentence.

What I would actually do

1. Measure complete-evidence recall, not hit rate. Write 20 multi-fact questions for your corpus, mark the evidence sentences, and track how often all of them reach the context. Report it next to partial recall. The gap between the two is the number to watch.
2. If your documents have structure, chunk on it. Split on headings and numbered clauses, prefix the heading path, resolve explicit cross-references at index time. It is deterministic, cheap and explainable, and here it was the biggest single win.
3. If they don't, use a supervised chunker with overlap. Generate QE pairs with an LLM, optimise boundaries against them, evaluate on different questions, and put your real retriever into the objective, or at least forbid cuts between a heading and its first line.
4. Tune chunk size and context budget together. Plot complete recall against budget for each chunk size. If a chunk can exceed the budget, it can never be retrieved. If the curves cross, your "best chunker" depends on how much context you send, and that should vary per query.
5. Add a chunk graph only where edges are trustworthy. Cross-references, citations and section hierarchy are good edges. Lexical similarity between large chunks mostly is not.

Try it

• Chunking Lab: vishalmysore.github.io/advancedRagDemo/chunking.html. Change chunk size and budget, switch retrieval modes and splits, toggle heading prefixes and cross-reference resolution, inspect any question, or paste your own markdown and QE JSON.
• Full RAG pipeline (hybrid search, reranking, WebLLM): vishalmysore.github.io/advancedRagDemo
• Code: github.com/vishalmysore/advancedRagDemo. The engine is src/utils/chunkLab.js, about 500 lines of dependency-free JavaScript. runLab(text, qe, { L, budget, evalSplit, headers, resolveRefs }) returns every number in this post, so you can run it in Node as well as in the browser.

Every configuration here can be opened from a URL. For example, chunking.html?mode=graph&q=7 opens the flash-photography case in graph mode.

Caveat: this is a toy. One 5,348-character fictional document, 25 questions, BM25 only. Each held-out question is worth about 7.7 percentage points, so one-question differences are noise. The value is in seeing the mechanisms the papers describe: stranded headings, budget crossovers, chunks that cannot fit, definition hubs. The rankings are not a benchmark. Run it on your own documents; that is what the paste box is for.

Papers referenced

• Zhao et al., QESChunker: A Single Objective Unifies Overlapping and Non-Overlapping Chunking for RAG, arXiv 2610.36557, October 2026
• Singh, Tripathi & Singh, StrucChunk: Structure-Aware Retrieval for Legal Document RAG, arXiv 2610.26313, October 2026
• Sahinoglu et al., SpectRAG: Weighted Chunk Graphs for Token-Budgeted Context Selection in Multi-Hop RAG, arXiv 2610.36991, October 2026
• Banerjee et al., The Bottleneck Is Retrieval, Not Reading, arXiv 2610.26951, October 2026
• Xia, Yamangil & Wang, Pre-retrieval Query Clustering for Adaptive Top-k Document Retrieval in RAG Systems, arXiv 2609.13489, September 2026
• Singh, Parser, Chunking, and Embedding Interactions in RAG over Indian Government Regulatory Documents, arXiv 2609.31660, September 2026
• Rathi, Chawla & Andreopoulos, Re-ranking and Late Interaction Drive Retrieval Quality, arXiv 2609.38473, September 2026

Views are my own. All code is MIT-licensed.