METR assesses internal data, judges capability gains at about 2 times
METR, a U.S. AI evaluation group, on Nov. 5 (Nov. 6 Japan time) released a report assessing how much AI is speeding up AI development inside Anthropic. METR analyzed Anthropic's internal data from Aug. 12 to Oct. 29 and interviewed about 15 key employees. It concluded that, as of October 2026, Anthropic is likely improving capabilities on well-defined tasks at about twice the pace of March 2024 to October 2025. It said it would not be surprised if the actual figure were closer to 1.3 times or 3 times.
What METR calls "well-defined tasks" are tasks that are easy to grade automatically or with a language model. The figure of about 2 times is METR's overall judgment, drawing on benchmark estimates as well as evidence that was redacted in the public version. The range of 1.3 to 3 times is also a judgment, and METR did not explain how it was calculated. The conclusion applies to Anthropic, and METR did not draw conclusions about the pace of development at other companies.
Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development. We are now publishing a report summarizing the results of this risk assessment.
About 1.9 times is the ratio of the pace of increase in the "AECI" metric
The widely reported figure of about 1.9 times compares the pace of increase in a metric called AECI. AECI is Anthropic's in-house version of a capability index from research group Epoch AI, combining internal and external benchmark results into a single score. In METR's estimate, AECI accelerated from 15 points a year before October 2025 to 29 points a year after. The ratio is 1.9 times, with a 95% confidence interval of 1.5 to 2.8 times.
The calculation assumptions are as follows. It covers Anthropic's models, including internal-only models not released externally. The first model in the chart is "Claude 3 Opus" from early 2024, and the last is an internal model from October 2026. For each model's date, it used the date when most employees gained access instead of the release date. It estimated a single point where the slope changes, found that point to be Oct. 7, 2025, and took the ratio of the slopes of the 2 lines before and after.
The figure comes with caveats. The confidence interval includes only the uncertainty in each model's AECI, not the uncertainty from which models are included. Anthropic provided the data, and METR did the calculations using Epoch's public code. METR did not run the unreleased models itself, and whether it checked the benchmark results provided could not be confirmed from the report. The names of the benchmarks are redacted, and internal benchmarks are increasingly saturated. All 15 benchmarks used to evaluate the latest "Model 4" exceeded 50%.
AECI scores are in arbitrary units, and Anthropic fixes "Claude Sonnet 3.5" at 130. Because it uses internal results, it cannot be directly compared with the values of the index that Epoch publishes.
- Before October 2025
- After October 2025
- Anthropic models (main estimate)15 → 29
- Frontier models from all developers17 → 24
1.1 to 2.9 times depending on assumptions, 1.4 times including other companies
The 1.9 times figure moves considerably depending on calculation assumptions. Changing the formula that relates benchmark scores to a model's ability (the link function) moved it from 1.1 times to 2.9 times. On the other hand, checks that varied the benchmarks used, model versions and the strength of adjustments mostly stayed within 1.9 to 2.2 times.
Including frontier models from all developers, the increase is only 1.4 times, from 17 points to 24 points a year. In this estimate, the earlier slope is derived mainly from other companies' models and the later slope mainly from Anthropic's models. METR offered 2 interpretations and did not decide which is correct. One is that Anthropic's models did not reach the industry frontier in 2025, making past progress look low, so 1.9 times is too high and 1.4 times is closer to accurate. The other is that if other companies' models were optimized for benchmarks, the past slope would look high, so 1.4 times is too low and 1.9 times is closer to accurate.
The system card for Anthropic's "Claude Haiku 5.5" had shown 1.7 times. METR said the difference from 1.9 times was mainly due to adding data from new, unreleased models. Anthropic's system card had cited a hypothesis that capability jumped once with Mythos Preview. METR said that, when the internal models added later are fitted, the view that the slope itself changed fits better. The errors were 1.9 points and 3.9 points, and the slope change came out ahead in 81% of resamples.
METR acknowledged that the 1.9 times figure could be an artifact of overfitting to benchmarks or of the AECI method. If growth in the baseline period was temporarily low, the acceleration would be overstated. If recent acceleration was missed, or if AECI understates the actual impact, the acceleration would be understated.
The 95% confidence intervals are 1.5 to 2.8 times for logistic, 1.3 to 2.7 times for probit, 2.0 to 4.1 times for cauchit, 0.8 to 1.5 times for complementary log-log, 2.1 to 4.9 times for log-log and 1.8 to 4.1 times for learned. The complementary log-log interval spans 1 times, meaning no acceleration. No interval is given for the all-developer estimate.
About 80% due to AI is a subjective judgment, with no range given
METR wrote of the acceleration as of October 2026 that it guesses a majority, perhaps around 80%, is due to AI. The figure is not the result of a calculation but a judgment METR calls a subjective estimate. It said it considered factors such as increased compute, hiring, data improvements, algorithmic discoveries and the easing of other constraints. While saying there is great uncertainty, it gave no range around the 80%. The body of the report does not clearly define what the percentage is a share of. An explanatory chart separates out the portion above the past trend, but the body does not explicitly link that chart to the 80%.
The roughly 80% refers to the acceleration as of October 2026. It covers a different period from the 1.9 times, which looks at the average since October 2025, and the report does not show a calculation combining the 2. METR explains that cumulative acceleration is usually smaller than the acceleration at a given point in time. It also said factors other than AI likely had a large influence on the acceleration in late 2025. Detailed estimates of AI's contribution and many of the factors in late 2025 were redacted.
Employees' views were divided. METR classified the statements of 11 of those interviewed by whether they supported acceleration of 2 times or more from Claude. The results were "yes" from 2, "lean yes" from 3, "unclear" from 4, "lean no" from 0 and "no" from 2. One researcher guessed that the portion due to AI was 15-20%. In an informal survey on the company's internal Slack, self-reported productivity gains from the most capable internal model averaged about 12 times. METR said the respondents were not representative of all employees and their answers could not be considered carefully thought out, describing the survey as very weak evidence.
It also indicated limits to AI's role. METR said it understands that AI has not yet reached the stage of making large, fully autonomous contributions to the most important parts of model training. Researchers, however, are describing signs of it, and it could happen soon, it said.
Models 2 to 4 are labels shared with METR; correspondence to products unknown
Anthropic's internal-only models appear in the report under labels. METR noted that these are the names that were shared with it, and that it understands they differ from internal names. Models 2 to 4 differ from Anthropic's internal names, and the report does not say which products they correspond to. Whether they are the same as released models or future products is unconfirmed.
| Label | When training finished |
|---|---|
| Unreleased 2022 | 2022 |
| Model 2 | July 2026 |
| Model 3 | August-September 2026 |
| Model 4 | September-October 2026 |
Most of the evidence redacted, and compute verification halted midway
Anthropic had the right to unilaterally redact nonpublic information in the report, and it redacted most of the evidence METR considered most useful. Anthropic said this was to remove confidential information that could accelerate other AI developers. METR acknowledged that the public version does not achieve transparency of evidence and that readers must rely on METR's judgment far more heavily than would be ideal. Forecasts for future models were also redacted. The full version, about 100 pages long, was given to Anthropic.
METR said it is confident that accelerating growth in training compute is not the cause, but the details of the basis for that are also redacted. Verification of compute allocation, which had been part of the assessment, was removed from the scope by Anthropic midway and halted before METR reached substantive conclusions. METR's team amounted to 2.5 people's worth of staffing, and it said it could have done a much deeper assessment with direct access equivalent to that of employees. Significant uncertainty and disagreement remained within METR's team, and many members see the figure as about 2 times.
METR received no payment from Anthropic, and the tokens it used were provided free of charge. It also disclosed that one team member had job discussions with Anthropic and declined an offer, and held shares as a former employee of a competitor. That member's work was reviewed by colleagues without conflicts of interest, it said.
Anthropic's public report says less than 2 times; METR says it understates
On the same day, METR also published a review of a risk report Anthropic released in August. In that report, Anthropic concluded that the pace of progress was below a threshold in its internal standards. The threshold refers to achieving 2 years' worth of progress at the pre-AI baseline in 1 year, primarily because of automation of research and development. Anthropic also wrote that this conclusion is uncertain and that recent acceleration is hard to measure.
METR's review said the public version, overall, understates the level of acceleration at the time. It said the threshold may have been exceeded by July 15, and reported that Anthropic's internal nonpublic version of the report also wrote that it may exceed 2x overall. Many of the researchers involved lean toward the view that it has not been exceeded, it said. METR agreed with Anthropic that the risk of catastrophe from models as of July 15 was low.
Because the threshold's conditions differ from AECI's 1.9 times, 1.9 times does not translate directly into a judgment on the threshold. Whether METR judged that the threshold had been exceeded as of October 2026 is unconfirmed within what could be checked.
According to The Associated Press, Anthropic co-founder Sam McCandlish posted on X that many of METR's conclusions match his own views. He wrote that AI is speeding up Anthropic's research, but that it is hard to say exactly by how much, and said the company would publish its own analysis within weeks. On the redactions, he said they were to protect commercial secrets and avoid handing other companies a roadmap to move faster.
No constraints seen for the next year; transparency in focus ahead of IPO
METR said it found no evidence that algorithmic progress, training compute, experiment compute or data would become a major constraint over the next 12 months. It said its confidence in this judgment is low because its views changed several times during the assessment. It cited AI-driven acceleration beginning to compound as a key concern.
According to the AP, Anthropic Chief Executive Officer Dario Amodei called on the industry to slow development in a Sept. 12 essay, supported the idea of placing outside evaluators inside companies and named METR. OpenAI CEO Sam Altman also endorsed adjusting the pace of development.
Anthropic made public its IPO filing on Oct. 26. The Wall Street Journal has reported the listing for the week of Nov. 16. The focus will be on the content of Anthropic's own analysis and how far third parties can verify the redacted evidence.
What the report does not confirm
Which products Models 2 to 4 correspond to, and whether there are plans to commercialize them, is unconfirmed. The names of the benchmarks used in AECI, and whether METR checked the results Anthropic provided, could not be confirmed either. For the roughly 80%, the basis of calculation, range and detailed estimates cannot be confirmed from the report. How the 1.3 to 3 times range was calculated, and whether similar acceleration is happening at other companies, are also unconfirmed. No material in which another organization verified METR's conclusions was found.
