Kavish builds a help assistant for his company's staff handbook. He tests it with 10 real questions. Seven answers are good. Three are wrong, and each one sounds sure of itself.

One teammate says, "Let's add a graph database." Another says, "Let's switch models." Kavish pauses. He does not yet know why those three failed, and fixing without knowing is guessing.

This is the most useful habit in retrieval work: before you change anything, look at which pages the system actually found for a failing question. A wrong answer has two parents: the AI was given the wrong pages, or it was given the right pages and misused them. You cannot tell which from the answer alone, but you can tell in about one minute.

The one-minute habit

For every failing question, write down the question and the page that should answer it. Then print the pages the system actually found, in order, with their rank. Ask three questions: was the right page there at all? If yes, at what rank? Was it complete?

Here is what that looks like on a real failure. Kavish asks, "How many days do I have to return an item for a refund?" The printout shows:

  1. (score 0.81) Refund policy, 2022 version: "You have 30 days from delivery to return an item for a full refund."
  2. (score 0.78) Returns FAQ: "Most items can be returned. See the refund policy for the time limits."
  3. (score 0.74) Refund policy, 2025 version: "You have 14 days from delivery to return an item for a full refund."

The assistant answers 30 days. The current rule, 14 days, was found at rank 3, but the old rule won at rank 1. Nothing here is a model problem. It is the wrong version, and it took one minute to see.

Six ways basic RAG breaks

Think of a hospital. Before treating anything, a doctor asks where it hurts. Here the body is a pipeline, and there are six places it can break.

A. The right page is never found. Kavish asks what error E-4410 means and gets general guides. The question uses different words than the page, or the chunks were cut badly. The AI cannot use a page it never gets. First fix: cut better chunks, then add hybrid search, meaning plus exact words, so a code like E-4410 matches by its letters.

B. The right page is found, but ranked too low. The page for part-time notice periods sits at rank 6, and only the top 3 reach the AI. First fix: a re-ranker, a slower second model that re-reads the top results and puts the best first.

C. The chunk lost its heading. A chunk reads, "You get 10 days." The heading that said who it applies to, "Part-time staff", was cut into another chunk. A full-time employee asks about leave and gets 10 days, which is wrong for them. A chunk is read alone, so it must carry its own context. First fix: keep the heading or title with every chunk.

D. The wrong version wins. An old policy outranks the new one, or a page shows up for someone who may not read it. Search ranks by closeness and knows nothing about dates or permissions. First fix: metadata filters. Store the version and the access list with each chunk, and filter before ranking, so a blocked or outdated page can never reach the AI.

E. The right page is on top, and the answer is still wrong. The AI added its own guess, or got lost in a long prompt. First fix: clearer instructions, "answer only from these pages, name them", and fewer, better pages.

F. No single page holds the answer. "Which suppliers make parts used by Ohio customers?" needs several pages joined together. Search returns pages one by one, not connections. First fix: an agentic loop, or a knowledge graph. These cost the most, so they come last.

The six are not equal in cost. A and B are often an afternoon of work. F can be weeks. A system often has two causes at once, such as A and D together.

Count first, then fix in order

Kavish prints what was retrieved for each of his three failing questions. Q4, the E-4410 error, never retrieves the right page: cause A. Q7, the part-time notice question, has the right page at rank 6 while only the top 3 are used: cause B. Q9, the Ohio suppliers question, has no single page with the answer: cause F.

Now the arithmetic. Today: 7 good answers out of 10, or 70%. Say hybrid search fixes Q4: 8 out of 10, or 80%. Say a re-ranker fixes Q7: 9 out of 10, or 90%. Q9 would still fail.

So fix A and B first. They lift the score from 70% to 90% with small, cheap changes. Q9 waits: its fix is the costliest, and only 1 question in 10 fails this way, so Kavish checks the logs for more evidence before building anything.

In tallies like this, "never found" and "wrong version" together are usually more than half of all failures. Fix those two before buying anything.

The order to try fixes, cheapest and most common first: look at the retrieved pages. Check the instructions and prompt. Fix the data and chunks. Add hybrid search. Add a re-ranker. Rewrite vague queries. Add metadata filters and permissions. Only then try an agentic loop or a graph.

Most teams reach for the expensive fix first and miss the cheap ones. Print the pages, find where it hurts, then fix that, and only that.

The full interactive chapter, with a clickable pipeline map of all six failures, a sorting exercise, and Kavish's tally worked out step by step, is free at https://deployedlab.dev/lessons/46-why-basic-rag-breaks. 11 chapters are free with no signup, and the full course is $5/month or $39/year at deployedlab.dev.