SWE-sweep
Most coding benchmarks hand the agent a bug report and ask for a fix. Real work does not. SWE-sweep, from Meta researchers including two of the creators of SWE-bench, gives a coding agent a real repository and no hint about what is broken, where it is, or what kind of bug it is. The agent has to find the bugs and repair them. Across 4,068 known bugs in 100 repositories and 22 languages, the best model tested fixed only 4.7 percent, verified by hidden tests. Discovery, not repair, is the hard part. If you build coding agents, this is the benchmark to watch, and the one to run your own agent against.
What you get
- A benchmark for teams building coding agents who want to measure discovery, not just repair
- A score for how many bugs an agent can find and fix in large codebases, verified by hidden tests
- A paper and site at swesweep.com, published 1 October 2026, with the full results
- A live repo with 246 stars, no release yet, and no stated licence
Do not trust a benchmark that hands the agent the bug report. Run your agent against SWE-sweep and see whether it can find the bug before it fixes it.
Repo: https://github.com/facebookresearch/swe-sweep
Quick poll
What would you use SWE-sweep for first?
- Running my own agent against it
- Reading the paper first
- Comparing models on it
- Watching it for now
Vote in the comments.
If you found SWE-sweep interesting, I write about tools like this every week in The AI-Native Engineering Leader, what actually works for engineering leaders building with AI agents, and what to skip.


