Recall vs reflect: what each one is actually for in an incident agent

Recall vs reflect: what each one is actually for in an incident agent

When I started wiring Hindsight into DejaVu, my incident-response agent, I treated memory as one thing: you put incidents in, you get relevant incidents out. It took a few wrong turns to see that Hindsight gives you two very different ways to read memory, and that an incident agent needs both, for different jobs.

Recall answers "which past incidents look like this one?" It returns evidence.
Reflect answers "what has our whole history taught us?" It returns a conclusion.

Mixing them up gives you either an agent that can't see the big picture or one that makes confident claims you can't check. Here's how I split them.

Repo: https://github.com/cybersleuth25/Deja-Vu

The setup

DejaVu watches simulated service traffic. When a detection rule fires (checkout returning 500s, a burst of failed logins), it opens an incident and tells the on-call engineer what to do first, and what not to try again because it failed before.

All resolved incidents live in a Hindsight memory bank. If you haven't used agent memory before, Vectorize's explainer What is agent memory is a good primer, and the Hindsight docs cover the API.

Writing memory so recall can find it

Before talking about reading, one thing about writing. Hindsight doesn't store your text as one blob. It extracts facts from it and consolidates them over time, so what comes back from recall is not always word-for-word what you stored.

That matters when you need to map a recalled fact back to an exact record. My fix was to repeat the incident id on every line of the text I retain:

def incident_to_text(inc: dict) -> str: """Render an incident as a postmortem paragraph. The id is repeated so every chunk keeps it.""" iid = inc["id"] lines = [ f"Incident {iid} ({inc['type']}, {inc.get('severity', 'SEV?')}) on service {inc['service']}, " f"opened {inc['opened_at']}, status {inc.get('status', 'resolved')}.", f"{iid} title: {inc['title']}.", f"{iid} signal: {inc.get('signal', '')}", ] 

It looks repetitive to a human. To the extractor it means every fact carries its source.

Recall: evidence for one incident

When a new alert fires, the agent calls recall with the alert's title and signal and asks for the raw chunks as well:

res = self._call(self.client.recall, bank_id=self.bank_id, query=query, max_tokens=max_tokens, budget="mid", include_chunks=True) 

It then pulls every INC-xxxx id out of the facts, the document ids, the metadata and the chunks. Those ids are all I take from recall. I don't let recall decide the answer. Each recalled incident gets a boost in a plain similarity score (same service, same files, same attacker network), and the fixes are ranked from the exact recorded outcomes.

That split is deliberate. Recall is very good at "this sounds like that", even when the wording differs: "QueuePool limit reached" and "connection pool exhausted" end up in the same place. But when I tell an engineer "this fix worked 3 of 4 times", I want that number counted from records, not estimated from fuzzy matches.

Before and after recall

Same alert, recall off and then on.

Without memory:

Checkout 500s: DB connection pool exhausted on payments-api. Standard first steps: Restart connection pool; Restart service; Increase pool size. No incident history was consulted.

With recall:

This outage on payments-api matches 5 past incidents: INC-0004, INC-0009, INC-0014, INC-0020, INC-0029. Roll back pool config worked 3 of 4 times; Restart connection pool failed 4 of 4.

[Screenshot 1: the incident view with memory on, showing the matched incident ids and the reason each one matched]

The second answer is specific to this team, and every claim in it points to an incident you can open.

Reflect: conclusions across everything

Recall is scoped to one alert. It will never tell you "your security incidents cluster on Tuesday evenings" or "restarting things is your team's reflex, and it rarely works". For that you need something that looks across the whole bank and reasons about it. That's reflect.

DejaVu calls it from the patterns panel, with a question that asks for a synthesis and for citations:

REFLECT_QUERY = ("Across all incidents, which remediation fixes keep failing, which fixes reliably work, " "and what recurring patterns (attackers, risky code areas, timing) should the team act on? " "Cite incident ids.") def reflect_summary(memory) -> dict: """Hindsight reflect synthesis. Never raises.""" if memory is None: return {"text": "", "error": "No memory backend configured."} try: return {"text": memory.reflect(REFLECT_QUERY), "error": None, "backend": memory.name} except Exception as exc: return {"text": "", "error": f"reflect failed: {exc}", "backend": memory.name} 

Two choices here are worth explaining.

Reflect never sits on the critical path. It's an LLM call on the Hindsight side and is much slower than recall, so it only runs when someone opens the patterns panel and clicks Ask Hindsight reflect. The result is cached for two minutes, and the cache is cleared whenever an incident is resolved or a verdict is recorded. A slow or failing reflect shows a warning; it never blocks the incident view.

Reflect sits next to computed patterns, not instead of them. The panel also shows patterns that DejaVu computes straight from the records, such as "8 of 10 security incidents started on a Tuesday evening (17:00-22:00)". Showing both side by side lets an engineer see where reflect's synthesis agrees with the hard numbers. Reflect is good at phrasing and connecting; the computed numbers are what you'd put in a postmortem.

[Screenshot 2: the patterns panel with the computed patterns on top and the Hindsight reflect summary below]

A rule of thumb

After building this, the line I'd draw is:

Neither one is where the numbers come from. That's the third piece: a small JSON store that's the system of record. Hindsight decides which incidents are relevant and what the overall story is; the records decide how many times something worked.

Limitations

• Reflect output varies. It's generated text, so two runs over the same bank can phrase things differently. That's why it's labelled as Hindsight's synthesis and shown next to the computed patterns rather than replacing them.
• Freshly seeded memory needs a minute. Hindsight extracts and consolidates in the background, so right after bulk-loading incidents, recall and reflect can return less than you expect. The seeding script tells you to wait about a minute and then run a recall check.
• The traffic is simulated. Detection runs on generated events, not real logs. The memory side is real; the log pipeline isn't built.

Takeaway

If your agent only uses recall, it's a good lookup tool that never learns anything general. If it only uses reflect, it's a confident narrator you can't check. Use recall for evidence, reflect for synthesis, and keep your hard numbers in records.

Hindsight docs: https://hindsight.vectorize.io/
Hindsight on GitHub: https://github.com/vectorize-io/hindsight
What is agent memory: https://vectorize.io/what-is-agent-memory

Tags: @Code.in