Independent audit: AI doubles pace of Anthropic's model gains

Summary

• METR estimates Anthropic's model capability gains on specified tasks now run roughly twice their earlier pace.
• The evaluator attributes most of that acceleration to AI assisting Anthropic's own researchers, not added training compute.
• It is the first independent measurement from inside a leading lab of AI speeding model development.

The latest

Anthropic's models are likely improving on well-specified tasks at about double their previous rate, according to an assessment published on 5 November 2026 by METR, the independent model evaluation group. METR said AI use most likely explains most of that acceleration. It also warned the finding carries substantial uncertainty, and that redaction of Anthropic's internal material removed much of the supporting evidence.

Details

• The measurement: METR tracked Anthropic's capability scores using the AECI index and found they grew 1.9 times faster since October 2025, with a 95% confidence interval of 1.5 to 2.8 times. The comparison set October 2026 against the pace recorded between March 2024 and October 2025, and included unreleased internal models.
• Robustness checks: Alternative modelling choices produced acceleration estimates ranging from 1.1 to 2.9 times, according to METR, a spread wide enough that the group stressed the central figure should be read as an estimate rather than a settled number.
• Ruling out compute: METR said growth in training compute does not account for the speed-up, leaving AI assistance as the leading explanation for the change in pace between the two periods it compared.
• Productivity claims: Informal internal surveys put the average self-reported productivity gain from Anthropic's strongest internal model at roughly 12 times in October 2026. METR described this as weak evidence and did not treat it as a basis for its central estimate.
• Limits of autonomy: Current impact comes mainly from AI assisting human researchers, METR said, and AI does not yet make fully independent contributions to the most important training work. Some Anthropic researchers told the evaluator that threshold could be crossed soon.
• Bottlenecks: METR found no convincing evidence that algorithmic progress, training compute, experiment compute or data will constrain progress over the next 12 months, while saying it holds low confidence in that conclusion. It called the risk of gains compounding into faster acceleration significant.
• Measurement caveats: AECI focuses on easily scored tasks, so gains may not translate into real-world impact, METR said, adding its estimate could run above or below reality. One model cleared 50% on all 15 benchmarks, raising saturation concerns.
• Access and scope: The work ran 11 weeks from 12 August to 29 October 2026 using about 2.5 full-time staff, covering internal benchmark results, pre-training and reinforcement learning metrics, internal documents and roughly 15 staff interviews.
• Redaction: Anthropic could accept or refuse information requests and redact non-public material, and METR said redaction removed most of the more informative evidence, including AI credit attribution details, algorithmic progress estimates, bottleneck analysis and forecasts for coming models.
• Independence: Anthropic said redaction protects confidential information that could speed rival developers. METR said it retained full freedom over methodology and conclusions, received no payment, and judged Anthropic's own public report in August to have understated the degree of acceleration.

Between the lines

METR's own framing sets the main tension: a measured doubling in pace sits alongside a robustness range running from 1.1 to 2.9 times, and the strongest supporting material was redacted. Its judgement that Anthropic's August report understated acceleration also places the lab's public account and the independent estimate on different footings.

What's next

Watch whether Anthropic releases unredacted figures on AI credit attribution and algorithmic progress, whether METR's 12-month bottleneck assessment holds, and whether researchers' expectation of fully independent AI contributions to training work materialises.

Source: METR