Code review has a blind spot, and it’s history. A reviewer looking at a three-line change to auth/session.py sees three reasonable lines. They don’t see that this file was in the middle of three outages in the last ten weeks, or that the obvious fix for those outages never worked.

I built a small bot for DejaVu, my incident-response agent with long-term memory, that puts that history into the pull request. If a PR touches a file with an incident history, the bot comments with every incident that file was part of. This post covers how it works and the two security problems I had to fix before I’d trust it.

Repo: https://github.com/cybersleuth25/Deja-Vu

What the comment looks like

Open a PR that changes auth/session.py and the bot posts this:

DejaVu: incident history for this pull request
⚠️ auth/session.py was involved in 3 outages
INC-0007 2026-08-27 SEV2: Users logged out at random: session store evictions
INC-0025 2026-10-27 SEV1: Login 500s: JWT verification failing after key rotation
INC-0029 2026-11-03 SEV2: Checkout 5xx: session lookup timeouts

Fixes that did not work for these incidents: Restart service (INC-0007, INC-0025); Scale up replicas (INC-0029)

Pattern: 4 of the last 6 outages were preceded by an auth-related change (auth/, middleware/jwt) within 48h. (INC-0025, INC-0028, INC-0029, INC-0032)

Recommendation only. Consider extra review and tests for these paths before merging.

[Screenshot 1: the bot's comment on a real GitHub pull request]

The diff is the same either way, but the review isn’t. Without the comment, the reviewer sees a small change. With it, they see that this file has broken production three times, that auth changes came before 4 of the last 6 outages, and that restarting the service didn’t fix any of them.

How it finds the history

The check is deliberately boring. For each changed file, it scans the incident records for two things: was the file involved in the incident, or was it changed shortly before it?

python
def check(changed_files: list[str], incidents: list[dict]) -> dict:
changed = [f for f in dict.fromkeys(_norm(f) for f in changed_files) if f]
findings = []
for f in changed:
hits = []
for inc in incidents:
role = None
if f in inc.get("files", []):
role = "involved"
elif f in ((inc.get("prior_change") or {}).get("files") or []):
role = "changed before"
...

It also collects which fixes failed and which worked. A fix only shows up as “did not work” if it never worked in any of those incidents.

There’s no LLM in this and no network call. The bot reads the incident records straight from the repo. I did that on purpose. A PR comment should be fast, identical on every run, and free of anything made up, and every line in it traces back to an incident record.

The AI lives in the rest of DejaVu. Hindsight memory recalls similar incidents during a live outage, and its reflect feature summarizes patterns across all of them. The PR bot uses the same incident history, but as a check that runs before the next outage. (If agent memory is new to you, Vectorize’s What is agent memory is a good primer, and the Hindsight docs cover the API.)

One comment, not fifty

The bot runs on every push to the PR. If it posted a fresh comment each time, a busy PR would end up buried in copies, so it finds its own earlier comment and updates that one.

My first version identified “its own” comment by a hidden marker in the body, <!-- dejavu-pr-warning -->. A security review caught the flaw: anyone can paste that marker into a comment. Their comment would get overwritten by the bot, or the bot would mistake it for the warning and never post the real one.

The fix is to check who wrote the comment as well as what it says:

python
BOT_LOGIN = os.environ.get("DEJAVU_BOT_LOGIN", "github-actions[bot]")

def is_ours(comment: dict, bot_login: str = BOT_LOGIN) -> bool:
"""Anyone can paste the marker into a comment, so it only counts when the bot wrote it."""
return ((comment.get("user") or {}).get("login") == bot_login
and (comment.get("body") or "").startswith(prcheck.MARKER))
Don’t let the PR grade itself

The second problem was harder to spot. My first workflow checked out the PR’s own code and ran the check from there. That meant a PR could edit the incident history file or the bot’s script and quietly remove its own warning.

Now the workflow checks out the base branch, which is the code that’s already been reviewed, and takes only the list of changed file names from the PR:

yaml

  • uses: actions/checkout@v4
    with:
    ref: ${{ github.event.pull_request.base.sha }}
    fetch-depth: 0

  • name: List changed files
    run: |
    git fetch --no-tags origin "+refs/pull/${PR_NUMBER}/head:refs/remotes/pr/head"
    git diff --name-only "${BASE_SHA}...pr/head" > changed.txt

The PR’s code never runs. All it controls is which file names get looked up.

It never blocks

The bot only comments. It doesn’t fail the build, block the merge or request changes, and the comment ends with “Recommendation only.” Incident history is context for a reviewer, not a verdict on the code. A bot that blocked merges over old outages would get switched off within a week.

Limitations
The history comes from a file in the repo. The bot reads incident records committed to the base branch, so an incident resolved in the live app only reaches it once someone commits it back. It doesn’t call Hindsight.
A PR can still edit the workflow itself. GitHub runs the workflow file from the PR, so a PR that changes it could change what the bot does. The change would show up in the diff, and requiring approval before workflows run on outside PRs closes the gap.
Matching is by exact path, so a renamed file loses its history.
The incident history is synthetic. I wrote the 32 incidents for this project and planted patterns in them on purpose, so I could check the output against what I knew was there.
Takeaway

Catching an outage at review time is cheaper than catching it in production, and you don’t need a model for that. The incident history your agent learns from can also become a plain, traceable warning, placed where risky changes get approved.

Hindsight docs: https://hindsight.vectorize.io/
Hindsight on GitHub: https://github.com/vectorize-io/hindsight
What is agent memory: https://vectorize.io/what-is-agent-memory

Tags: @Code.in