All Articles

Anthropic Just Finished Training Three Models It Has Yet To Release, So How Many Of Your AI Decisions Rest On Tooling Behind The Frontier?

Anthropic Just Finished Training Three Models It Has Yet To Release, So How Many Of Your AI Decisions Rest On Tooling Behind The Frontier?

Anthropic has trained three models it has yet to release. METR refers to them as Model 2, Model 3, and Model 4, codenames the evaluation group says were shared with it, and METR notes the models have different internal codenames. They finished their training runs between July and October of this year and have not reached a single customer. Anthropic's researchers are using more advanced models than the rest of us, and an independent assessment from the evaluation group METR found that a lot of the acceleration and productivity gains it measured are attributable to unreleased internal models. That gap, between what a lab has already built and what it lets you buy, is the whole story of what it means to purchase the frontier today.

What the report actually showed

The METR team spent eleven weeks inside Anthropic, from Aug 12 to Oct 29, 2026, reviewing pretraining metrics, reinforcement learning, benchmark scores, and staff interviews. The headline finding: benchmarks for Anthropic's models accelerated about 1.9x since October 2025, which by METR's own framing means the company appears to have made roughly two years of historical progress in a single year. The assessment was also confident that the acceleration was not driven by the lab simply burning more compute to train bigger models, per The Verge.

Instead, METR traced much of the gain to a different cause. The group estimated that roughly 80% of the current acceleration traces back to the lab's use of AI inside its own research process. Most of that acceleration comes from AI helping human researchers do their jobs, not from AI working on its own. Staff using the most capable internal model self-reported being about 12 times more productive, roughly double the figure reported a few months earlier while the previous internal model was the newest tool, The Verge reported.

METR flagged clear limits on that evidence. The 12x figure came from low-effort Slack polls, and the assessment said AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of model training. There were only signs of life, meaning autonomous work could become important soon, and the group found no clear bottlenecks that would slow progress in the next twelve months, per METR.

The frontier is not a shelf you can read

For a business operator, the quiet implication is destabilizing. The commercial model you evaluate this quarter was already behind the frontier the moment it shipped, because Anthropic had already finished training a better version, and it has not released it. The release you get to benchmark, compare, and buy is a checkpoint, not the ceiling of what the lab can build.

METR made this concrete. The acceleration figure the group reported was higher than the number published in a recent Anthropic system card, about 1.7x, primarily because METR included the newer unreleased models in its calculation. In other words, the published numbers people quote to justify a purchase already understate what the leading lab has actually built. You may therefore be pricing your strategy against benchmarks that leave out exactly the progress that matters most.

How many of your decisions rest on it?

The honest answer to the question is not a number. It is every decision you have made about AI, because each one quietly assumes the model you can buy is close to the best that exists. Your tooling choices, your rollout timelines, your trust posture, your cost projections, even the decision of whether to try AI at all, lean on that assumption. METR's report gives a buyer reason to question it, and the group warns that the pace itself could keep speeding up, since it found no clear bottlenecks that would slow progress in the next twelve months.

The uncomfortable follow-up is that you cannot measure your exposure. The very thing you would want to see to size the risk is the thing Anthropic has kept locked down. The assessment group said Anthropic redacted a majority of the evidence METR found most informative, leaving the public to rely on METR's judgment rather than the underlying data, The Verge reported. When the source that would let you quantify the gap refuses to publish the gap, you are back on vendor confidence and your own best guess.

What you can actually do about it

You cannot pull the three hidden models out of the drawer. You can, however, stop pricing your strategy on the assumption that the shelf model is the frontier. The most practical move is to treat every commercial model as a moving target with a short half-life. If the leading lab kept its best work private while accelerating roughly twice as fast as the year before, the model you signed in Q1 will feel outdated long before the renewal arrives. Structure deals around the outcomes a model is supposed to produce, and insist the vendor proves it on your workloads rather than a benchmark pulled from a demo. Build in a second option and keep switching costs low, so an "our new model is much better" email never becomes a crisis.

There is also a second lesson. A lab that will withhold its strongest models even from independent researchers is signaling how it will treat you once you depend on it. The transparency you can no longer see inside Anthropic is a preview of the leverage behind every vendor relationship you build on frontier tooling.

So how many of your AI decisions rest on tooling behind the frontier? All of them, and the exact number does not matter. The model you can buy is a released checkpoint. The frontier it trails is where the real capability now lives, whether the lab chooses to share it with you or not.

Sources

Ready to Apply This?

Let's Put This Into Practice

Reading about automation is useful. Having us implement it for your specific business is transformative.