LangChain chunk_overlap=200: 2,181 of 4,155 Chunks Got No Overlap

This is the first line of a chunk from my test index:

That 14.2% is the number that explains the 32% crash rate.

14.2% of what? The answer sits at the end of the previous chunk, and the two chunks share zero characters. I had set LangChain chunk_overlap=200, the setting every RAG tutorial uses to stop exactly this.

So I stopped trusting the parameter and measured it. I ran RecursiveCharacterTextSplitter over 452 of my own blog posts (3.5M characters) and checked how much text each pair of neighboring chunks really shared. More than half shared nothing.

TL;DR

• LangChain chunk_overlap is a maximum, not a guarantee. RecursiveCharacterTextSplitter only carries over whole pieces, and those pieces are usually paragraphs.
• If the last paragraph of a chunk is longer than chunk_overlap, nothing carries over.
• At chunk_size=1000, chunk_overlap=200, 2,181 of 4,155 seams (52.5%) had 0 characters of overlap. The mean was 56, not 200.
• Sentence pairs that straddle a paragraph break were split apart 16.4% of the time, against 1.5% for a plain sliding window.
• Sentence-level separators or a 6-line tail function kept at least 99.5% of those pairs together.

Why does LangChain chunk_overlap produce no overlap?

LangChain chunk_overlap produces no overlap when the trailing piece of a chunk is bigger than the overlap budget. The splitter never cuts a piece to fit. It keeps a whole piece or drops it.

Here's the repro. Three paragraphs of 599 characters each:

from langchain_text_splitters import RecursiveCharacterTextSplitter para = lambda ch: " ".join([ch * 9] * 60) # 599 chars, one paragraph text = "\n\n".join([para("A"), para("B"), para("C")]) splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) print([(len(c), c[0]) for c in splitter.split_text(text)]) # [(599, 'A'), (599, 'B'), (599, 'C')] 

Three chunks, no shared text, no warning. The cause is in _merge_splits (I read it in langchain-text-splitters 0.3.11):

while total > self._chunk_overlap or (...): total -= self._length_function(current_doc[0]) + ... current_doc = current_doc[1:] 

After emitting a chunk, the splitter pops pieces off the front of its buffer until what remains fits inside chunk_overlap. Whatever survives becomes the start of the next chunk.

Those pieces come from the first separator that works, which is "\n\n" by default. The splitter only recurses to "\n", then " ", when a single piece exceeds chunk_size. A 350-character paragraph is fine by that rule, so it stays whole. It's also bigger than 200, so it gets popped, and the buffer ends up empty.

The mental model most of us carry is "the last 200 characters repeat." The real behavior is "the last paragraphs repeat, if they happen to total 200 characters or less."

How much overlap do you actually get?

On my corpus the average seam got 56 characters of overlap from a budget of 200, and the median seam got 0. Paragraph length explains it: the median paragraph is 188 characters, and 47% of my paragraphs run longer than 200.

Your number depends on your prose, so measure it. This takes a minute:

def real_overlaps(text, chunk_size=1000, chunk_overlap=200): splitter = RecursiveCharacterTextSplitter( chunk_size=chunk_size, chunk_overlap=chunk_overlap, add_start_index=True ) docs = splitter.create_documents([text]) spans = [(d.metadata["start_index"], d.metadata["start_index"] + len(d.page_content)) for d in docs] return [max(0, a[1] - b[0]) for a, b in zip(spans, spans[1:])] 

Of my 2,181 zero-overlap seams, 182 opened the next chunk with a word that points backward, such as That, This, It, They, So or But. A sample of what the retriever would hand your model cold:

• "That's right often enough to be dangerous."
• "That error only matters when it lands on a weight that contributes a lot to the output."
• "This is the single biggest source of 'rerere is broken' complaints, and it is not a bug."

Each sentence is fine in its post. Alone, each one is a pronoun with no referent.

Does missing overlap hurt RAG retrieval?

Less than I expected for simple lookups, and more for anything that spans a paragraph break. Those are two separate failures, so I tested them separately.

Test 1, answer retrieval. My posts use question-style H2 headers. I stripped every header from the text, used 1,000 of them as queries, and counted a hit when a BM25 top-5 result fully contained the first sentences under that header.

Test 2, seam damage. I took 8,299 pairs of sentences that sit on either side of a paragraph break (last sentence of one paragraph, first sentence of the next) and counted how many landed together in at least one chunk.

The sliding window rows are a plain character window snapped to word boundaries, for comparison.

Three things stand out.

Overlap did almost nothing for answer retrieval in the recursive splitter. The move from 40.1% to 40.6% is noise: 32 queries got worse and 37 got better.

Paragraph-aware splitting beat a sliding window by a clear margin. 120 queries were found only by the recursive splitter and 72 only by the sliding window, both at zero overlap. Read that with a caveat: my gold answers open a paragraph, so a paragraph splitter can never cut them in half. That test favors it by construction.

Seam damage is where the default loses. Recursive 1000/200 separated 16.4% of the pairs, about 1,360 of them. The sliding window with the same setting separated 1.5%. If your questions need the sentence before the break, the default config is the worst option on this table that still has overlap switched on.

All of this is BM25 on one author's markdown. I didn't test embeddings, and retrieval scores on a messy corpus of PDFs will look different.

Should you just raise chunk_overlap to 500?

No. At 1000/500 you still get 10.6% zero-overlap seams, and you pay for it twice. The index grows to 1.54x the corpus, and overlapping neighbors start crowding your results: 68% of queries had overlapping chunks in the top-5, leaving 4.05 distinct passages out of 5.

On a sliding window it gets worse. At 1000/500, top-5 results averaged 3.58 distinct passages and hit rate fell below the 300-overlap run (38.5% vs 39.3%). You've paid for 5 chunks of context and bought 3.6.

How do you get real overlap in LangChain?

Make the pieces smaller than the overlap, or add the overlap yourself after splitting. I tested both.

Option 1: split on sentences. Sentences fit inside a 200-character budget most of the time, so the pop loop keeps some:

splitter = RecursiveCharacterTextSplitter( chunk_size=1000, chunk_overlap=200, separators=[r"(?<=[.!?])\s+", " ", ""], is_separator_regex=True, ) 

Pairs kept went from 83.6% to 99.5% on a 1.14x index. Seams with no overlap dropped to 13.5%. Those remaining seams follow the same rule: a final sentence over 200 characters gets popped like anything else. You also lose paragraph-aligned chunks, though hit rate stayed within noise (39.6%).

Option 2: split on paragraphs, then stitch a tail on. This one gives the overlap everyone assumes they already have:

def with_tail(chunks, tail=200): out = chunks[:1] for prev, cur in zip(chunks, chunks[1:]): t = prev[-tail:] out.append(t[t.find(" ") + 1:] + "\n\n" + cur) return out chunks = with_tail( RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0).split_text(text) ) 

No zero-overlap seams, 99.7% of pairs kept, 39.9% hit rate. Against plain 1000/0 that's 40 queries lost and 38 gained, so noise again. This is the one I'd ship.

Two warnings from running it. Chunks can now reach chunk_size + tail (1,197 characters in my run), so check your embedding model's input limit. And don't shrink the base chunk to compensate. My first attempt used chunk_size=800 to stay under 1,000, and hit rate dropped to 34.6%, with 107 queries lost and 52 gained.

So why did chunk_overlap=200 give me no overlap?

LangChain chunk_overlap sets an upper bound on carried-over text, and RecursiveCharacterTextSplitter fills that bound with whole pieces only. Pieces are paragraphs by default, so any seam whose last paragraph exceeds 200 characters gets no overlap, silently. On my 452 posts that was 2,181 of 4,155 seams, with a mean of 56 characters. Run real_overlaps on your own corpus, and if the zero count is high, add the tail yourself with chunk_overlap=0 or switch to sentence separators.

Written by the developer behind Preterview, an interview prep platform.