Recently, I came across a YouTube thumbnail that said, "Why Grep is better than Vector Search."

That caught my attention.

Over the past few weeks, I've also been running into several articles, videos, and discussions around search algorithms, code retrieval, and optimization techniques for AI agents.

It made me curious.

How much does the way we retrieve information actually impact an AI agent's performance? Are we overcomplicating code retrieval with sophisticated approaches when simpler, well-engineered search techniques could do the job better?

That curiosity wouldn't let me sit still. I spent hours reading, experimenting, and understanding different approaches to information retrieval.

Well, curiosity needed a victim, and this time it was my keyboard. I decided to build an efficient code search adapter for agentic AI.

But there was another thought that kept coming back to me while working on this project.

Remember the days when memory was scarce and developers were extremely conscious of how much memory their applications consumed?

Every allocation mattered. Developers thought carefully about data structures, algorithms, and unnecessary computation.

By the way, I genuinely believe that's one of the reasons much of the older software felt incredibly fast. Resource constraints forced engineers to be intentional about efficiency.

Today, we have significantly more computing power, but we're facing a different kind of constraint when building AI applications: context.

An AI agent might have access to an entire repository, but that doesn't mean we should dump hundreds of files into its context window.

We need to retrieve the right information, rank it intelligently, and provide it within a limited token budget.

And that brings me back to the fundamental question:

Can we build a lightweight, efficient code retrieval engine that gives AI agents exactly what they need, without overwhelming them with unnecessary information?

That's what I wanted to experiment with. Here's how I approached it.

Rather than jumping straight into the implementation, I started with a few small experiments.

Experiment 1: Building a Trigram Index

What is trigram?

My first experiment was to understand how trigram-based searching works.

I followed these steps:

  1. Created a function to extract trigrams from a given word.
  2. Generated a trigram-to-document mapping using a collection of sample sentences.
  3. Tested the mapping with simple search queries.
  4. Implemented an index-based search function using the generated trigram map.

The basic idea was to avoid scanning every document for every query.

Instead, the index would identify potential matching documents using their shared trigrams, reducing the search space before performing the actual verification.

Once the basic implementation worked, I wanted to understand how it behaved with a larger dataset.

Experiment 2: Index Search vs. Full Scan

I generated 100,000 random sentences and intentionally inserted a rare word, zebracrossing, at document position 54,321.

The objective was to compare my custom trigram index against a straightforward full scan.

Here are the results:

[54321] [54321] rare: scan 12 ms, index 0 ms, same result: true rare: scan 3 ms, index 0 ms, same result: true rare: scan 11 ms, index 0 ms, same result: true common: scan 14 ms, index 24 ms, same result: 32078 vs 32078 common: scan 3 ms, index 18 ms, same result: 32078 vs 32078 common: scan 3 ms, index 20 ms, same result: 32078 vs 32078

Before comparing performance, I first verified that both approaches returned exactly the same result. As expected, both the full scan and my trigram index located the rare word at position 54321. The same correctness check held for the common query, with both approaches returning 32,078 matches. This gave me confidence that I was comparing search performance rather than different search results.

An interesting observation emerged.

For rare terms, my custom index function clearly outperformed a full scan.

However, for common terms, my custom index lost the battle. Why?

For a common term, my custom index retuned a large candidate set. The search engine had to process and verify those candidate to identify the actual matches.

In this situation, the overhead of retriving and filtering candidates was the greater than simple scanning the documents.

My realization here is, an index doesn't automatically make the query faster

Experiment 3: The smart search way

What makes this search smarter is here I have implemented the concept called selectivity which is commonly used in database systems.

If trigram occurence is below 10% of the total documents, use the index.
If trigram occurrence exceeds 10% of the total documents, use the default scan.
Here my intention is to avoid spending more computation time on processing a larger candidate where the full scan will outperform.

Here are the results

---- smart search benchmarks ---- [54321] true true zebracrossing: scan 3 ms, index 0 ms, smart 0 ms zebracrossing: scan 4 ms, index 0 ms, smart 0 ms zebracrossing: scan 3 ms, index 0 ms, smart 0 ms retry: scan 5 ms, index 14 ms, smart 5 ms retry: scan 5 ms, index 14 ms, smart 5 ms retry: scan 6 ms, index 14 ms, smart 5 ms

The adaptive strategy worked as expected in this experiment. For rare terms it used the custom index and for common terms it switched to a full scan.

This gave me an interesting takeaway: the fallback threshold is a tuning parameter. It can be adjusted experimentally to find the point where index-based retrieval stops being beneficial.

This is closely related to selectivity estimation and access-path selection in database query optimizers.

Of course, 10% is currently just an experimental threshold, not a universally optimal value.

A small experiment, a bigger question

What started as a simple search experiment gradually turned into a much bigger engineering question.

How do we build a retrieval system that doesn't just find matching code, but also understands which results are worth returning to an AI agent?

Search latency is only one part of the problem. Retrieval quality, relevance, and context consumption matter just as much.

I started moving beyond synthetic datasets and began experimenting with an actual open-source repository: Spring Batch.

That's where things started getting interesting.

I'll cover those experiments, the benchmark results, and the engineering decisions behind the retrieval strategy in the next part.

For now, I'm leaving you with one question:

When building retrieval systems for AI agents, should we always reach for sophisticated search techniques, or should we first get the fundamentals right?

P.S. Please excuse any spelling mistakes or grammatical imperfections. I'm still learning, and writing is part of that journey! 😊