In informal Slack polls, Anthropic staff put the uplift they got from their most capable internal model at 12x by October 2026. METR, the AI evaluation nonprofit that ran a high-access assessment of Anthropic from August 12 to October 29, called that number "only very loose evidence".

Its own estimate, published on November 5, is smaller. It is still large. METR says Anthropic "is likely improving the capabilities of their models on well-specified tasks at around twice the rate" it managed from March 2024 to October 2025. It also guesses that most of that speed-up comes from AI tools used in that research.

The same report carries a second finding that matters as much. In METR's words, "Anthropic redacted a majority of the evidence we found most informative for these conclusions."

What METR found

METR staff, 2.5 full-time equivalents, worked from Anthropic's internal benchmark data, compute metrics, uplift surveys, code and inference-spend data, and about 15 staff interviews. The headline numbers:

  • Capability growth on well-specified tasks is likely around 2x the earlier rate, with high uncertainty (plausibly 1.3x to 3x).
  • On AECI, which METR describes as Anthropic's version of the Epoch Capabilities Index, growth since October 2025 is about 1.9x the prior rate (95% CI 1.5x to 2.8x).
  • "We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI."
  • The acceleration is not explained by faster growth in training compute. METR says it is confident on that point.

RuntimeWire reports METR also laid the findings out in a 19-post thread on X, and that the estimate gives outsiders a quantified claim they can test.

A black silhouetted hand passes a wrench to a robotic hand that is assembling a taller copy of itself from gears, with burnt-orange motion lines around the gears
METR's read: AI tools are now a major input into how fast the next model gets built.

What the report can't show you

The redactions are the reason to read the numbers with care. METR says the public version does not meet its own standard for transparency about evidence, so you have to lean on METR's judgment more than usual. Anthropic held redaction rights, and the cut material includes per-stage acceleration estimates, bottleneck analyses, forecasts for future models and key intermediate measures.

Three more limits are in the report itself:

  • "Anthropic changed the scope of this project midway to exclude compute verification." A planned check of how compute was split between capabilities and safety work did not happen.
  • "For this engagement, we did not request or receive access to sample from non-public models."
  • "Acceleration might be limited to well-specified tasks." METR adds that recent progress "may be overfitting to the types of tasks which are included in AECI."

The internal-only model METR calls Model 4 scores above 50% on all 15 benchmarks used, which METR flags as a measurement concern, though METR doesn't think it changes the acceleration finding much. When a model already scores above half on every benchmark you have, there's less room left to see it improve.

Anthropic didn't dispute the direction. Co-founder Sam McCandlish wrote on X that "AI is speeding up our research, but it's hard to say exactly how much," KTNV reported. The AP report adds that Anthropic plans to publish its own analysis in the coming weeks.

A document page covered almost entirely in thick black redaction bars under a large magnifying glass, with only two short burnt-orange highlighted lines left visible
Anthropic redacted a majority of the evidence METR found most informative.

Anthropic's researchers credit coding agents

The interview quotes METR kept in the report will sound familiar if you use Claude Code every day:

  • "Experimentation overhead is driven towards zero super fast."
  • "I do genuinely think Claude (claude code, [agent-scaffold]) has been fairly significant."
  • "Now only bottlenecked on ideas & compute, no longer bottlenecked on code."

Those quotes are testimony from the people doing Anthropic's internal model-training work, and the report does not measure what coding agents do for developers outside the company. METR also notes that researchers described "signs of life" for fully autonomous AI contributions to R&D, which it says are not yet large.

The 12x problem is yours too

METR gave three reasons for treating the 12x poll as loose evidence: the people who answered may not represent the staff, the answers reflected low-effort consideration, and self-reported uplift has overstated real impact before. That last gap is one METR has measured in developers.

In its early-2025 study of 16 experienced open-source developers, the developers took 19% longer with AI tools and still believed AI had sped them up by 20%. METR now marks those results as out of date. In a May 2026 survey of 349 technical workers, the median reported change in the value of their work was 1.4x to 2x, and METR warned that "survey results are not necessarily grounded in reality."

So when your team, a vendor or you yourself quote a multiplier, ask what was measured. A few things you can check without a research budget:

  1. Pick real tickets from your backlog. Agents look best on well-specified work, which is the caveat METR attached to Anthropic's numbers.
  2. Count the time to a merged change, including review and rework, not the time until the agent says it's done.
  3. Track what happens after merge: reverts, follow-up fixes, bugs traced back to the change.
  4. Write the result down before you form an opinion, and compare it with the number you would have guessed.

METR has a note on the benchmark side of this too: many SWE-bench-passing PRs would not be merged into main. A passing test and a change a maintainer will merge are two different bars.

Plan for the model under you to keep changing

The forecast in the report is the part that touches your roadmap. "We did not identify any convincing bottlenecks to continued rapid progress," METR writes, looking 12 months out. It also says its view of bottlenecks changed several times during the work, so it is not confident in that conclusion.

If METR is even roughly right, the model your product or your agent setup depends on will be replaced sooner than you'd plan for. That argues for a few habits:

  • Keep the model ID in one config value so a swap is one change.
  • Keep a small eval set built from your own tasks and run it every time you change models. Public leaderboards measure someone else's work.
  • After a swap, watch cost and quality per task as well as the test results.

These habits pay off even if METR's 80% guess is wrong. All they need is a fast release cadence, and METR found no convincing bottleneck that would slow it.

Does the 2x figure match what you see in your own work, or does it feel closer to the 12x poll, or to the 19% slowdown? Tell me in the comments.

If this was useful

I build Hermes IDE, so weigh this with that in mind. It runs Claude Code, Codex and other agents side by side, each task on its own branch and worktree, so you can hand two agents the same ticket and compare what lands on your own repo. It's free, source-available (BSL 1.1), and runs on macOS, Windows and Linux: download it or read the source on GitHub. It isn't affiliated with Anthropic or Nous Research.

Measuring whether a model got better for your own work is what Observability for LLM Applications is about: traces, evals, cost, and catching quality regressions when the model under you changes.

Observability for LLM Applications — the book