helve_dev left a comment on my trimming piece that I could not shake. He had seen the same thing in a different shape: web search and extraction results, full markdown and JSON payloads, making a small model anchor on whatever was longest or loudest instead of what answered the question. Trimming to the tight fields that actually carry the answer, he said, fixed accuracy and the token bill at the same time, and it mattered most on long loops, where old results pile up over 15 or 20 turns until the model is reading boilerplate instead of facts. He had watched it happen in real traces. He had never run a clean two-sided test with scored pass rates. So I ran one. The result is not what either of us expected.

The setup is the same shape as the GitHub run, moved onto the web. Twelve questions, five pages each, all written for this test. The model searches, reads pages, and answers with the exact value: a version, a price, a flag, a limit. One arm sees the full page text in the search results. The other sees only the title, the link, and the first 200 characters of each page, and has to open a page to read the rest. Six questions hide the decisive fact inside those first 200 characters. Six hide it further down, where only opening the page finds it. Same small model as before, same six-call limit, every arm run twice. Both second runs came back identical to the first, so I am counting 24 tasks per arm, not 48.

1. The accuracy gain does not travel. It reverses.

Full pages solved 22 of 24. Trimmed pages solved 20 of 24. On the GitHub shape, trimming nearly doubled the score. On the web shape, it costs two tasks.

Both tasks it loses are the ones where the fact sits past the cut. One asks the per-seat price of a team plan after a price change. The trimmed snippet shows the cheaper plan's price first ($8, for the solo plan), and the model answers $8 and never opens the page. The real answer, $12, is three sentences further down. The other asks which command flag skips the build cache. The snippet never shows the flag, so the model opens the wrong page, the caching guide, and answers with a cache command instead. The trimmed arm solved all twelve questions where the fact was visible in the snippet, and only eight of twelve where it was not. The full arm solved all twelve of the hidden ones.

That is the shape of the failure, and it is worth naming plainly. The model does not treat a snippet as a pointer to a longer page. It treats it as the page. When the visible text contains a plausible answer (another plan's price, a related command), it answers from that and stops. Trimming only helps when the trim keeps the answer. When it cuts the answer, the model does not go looking for it. It answers with whatever the trim left behind.

2. The saving is small, and the re-open eats it

Trimmed pages cost 513 tokens per turn against 556 for full pages, and 1,472 per task against 1,667. That is 12%, not the 63% the GitHub trim saved. Two reasons. First, the pages here are short, so the gap between a full result and a trimmed one is only about 70 tokens. Second, the trimmed arm opens pages more often (0.67 opens per task against 0.42), and every open drops a full page into the history. By the third turn, the trimmed arm's prompt is actually bigger than the full arm's: 791 tokens against 783. The saving exists on the first turn and is gone by the third.

This is the part helve_dev's traces got right and my test sharpens. The bill on a long loop is not set by the size of one result. It is set by what stays in the history. Every result the model sees stays there, turn after turn, and every page it opens to recover a trimmed fact stays there too. Trimming one result does not trim the history. It just moves tokens from the search result into the re-open.

3. Both arms balloon the same way

Turn one costs about 162 tokens in both arms. Turn two costs 722 for full pages and 651 for trimmed. Turn three costs 783 and 791. That is nearly a fivefold rise in three turns, whatever the payload size, because the history only grows. The trim does not bend the curve. It shifts it by one turn at most.

The honest limit here is that nothing in this test ran long. Every task ended in three turns, so the 15-to-20-turn loops helve_dev described never happened. The curve to turn three is real evidence; the curve to turn twenty is still his traces, not my numbers. But the direction of the evidence is against the trim: if the history grows fivefold in three turns in both arms, a longer loop does not fix that by trimming harder. It makes the re-opens pile up faster.

4. The two arms fail in opposite ways

The full arm's only failure is a question needing two facts, a version and a date. It answered the date and dropped the version. The trimmed arm solved that one, and all three two-fact questions. Short results seem to help the model hold two facts at once. That is three tasks, and I will not lean on it. But the pattern is neat enough to note: full pages cost you the second fact, trimmed pages cost you the hidden fact. Neither shape is free. The question is which fact your task hides.

So where does that leave helve_dev's claim? Half right, in a way that matters. Trimming does cut the bill on the web shape, a little, not a lot, and only until the model starts re-opening pages. And trimming does not fix accuracy here. It breaks it, exactly on the questions where the answer sits past the cut. The GitHub trim worked because every decisive fact sat inside the first 300 characters by design. The web does not promise that. A page puts the price of the wrong plan first and the right one third. A snippet is not a summary. It is the top of a page, and the top of a page is where the least important facts live.

The fix I am keeping is narrower than the one I went in with. Trim the search result, yes, but only when the snippet is built to carry the answer, not just the first 200 characters of whatever the page happens to start with. And if the answer might sit past the cut, the model needs a reason to open the page before it answers, not permission to. A snippet that shows a plausible wrong answer is worse than no snippet at all, because the model will take it and stop.

Have you watched a small model answer from a snippet instead of opening the page? What made it open the page — a better snippet, a different instruction, or just a bigger model? I am curious what worked, because trimming alone did not.

P.S. Everything I write stays free — that part doesn't change. But if this one was worth something to you, you can buy me a coffee ☕ (one-time or monthly, both work). It covers the API credits and hosting behind these experiments — and honestly, it's also the nudge that gets the next one started on the evenings I'd otherwise skip it. — Salam

Buy me a coffee

AheadKit on Gumroad — Beginner-Friendly Books & Packs