Anthropic Warns Against AI While Building It Faster Than Anyone
Image Source: depositphotos.com
On June 4, 2026, Anthropic published a document unlike anything a major AI lab had put in writing before.
Titled "When AI builds itself," and co-authored by Jack Clark (Anthropic's co-founder and head of policy) and Marina Favaro, who runs the Anthropic Institute, the piece argues that frontier AI development may need to slow down — or even stop — before humans lose the ability to control what comes next.
What separates this from previous AI safety statements is the data. Anthropic disclosed its internal metrics, and they are hard to dismiss. As of May 2026, more than 80% of the code merged into the company's codebase was written by Claude. And this is happening just before the Anthropic IPO.
Before Claude Code launched in February 2025, that figure sat in the low single digits. Engineers are now merging eight times as much code per day as they were in 2024. A March 2026 internal survey of 130 employees found that the median respondent estimated producing roughly four times more output with AI assistance than without it.
In April 2026 alone, Claude delivered over 800 bug fixes that reduced an entire class of API errors by a factor of a thousand. According to one overseeing engineer, the work would have taken a human four years to complete.
External benchmarks confirm the pace. The length of tasks that AI systems can complete autonomously has been doubling approximately every four months. Claude Opus 4.6 now handles tasks that would take a skilled engineer 12 hours.
These figures point toward what Anthropic calls "recursive self-improvement" — the point at which an AI system could design and train its own successors with no meaningful human involvement. The company is direct: this threshold has not been reached, and is not inevitable. But it warns that crossing the threshold "could come sooner than most institutions are prepared for."
The proposed solution is not a unilateral halt. Anthropic argues explicitly that a single-lab pause would only shift who leads the field, not create the broader deliberative process the situation demands. A credible slowdown, the paper states, would require several well-resourced labs across multiple countries to agree simultaneously. A suggestion that OpenAI, among others, may not support, considering its upcoming IPO.
The Geopolitical Issues
Anthropic draws a comparison to Cold War nuclear treaties, while acknowledging a key asymmetry: AI training runs are far easier to conceal than missile silos. Without a verification regime that includes China, any pause risks becoming an advantage for whoever ignores it.
That observation has also sharpened the skepticism. David Sacks, a venture capitalist and informal adviser to President Trump, has accused Anthropic of pursuing a "regulatory capture agenda", using safety arguments to advocate for regulations that would effectively block lower-cost open-source competitors.
Rob Enderle of the Enderle Group told SiliconANGLE the economic and national security stakes were simply too high for any superpower to voluntarily slow down.
Anthropic is the company producing the most compelling evidence that AI development is accelerating beyond easy control. It is also the company accelerating it. With a reported IPO target for fall 2026, that tension is structural.
Whether "When AI builds itself" is a sincere risk assessment, a strategic positioning move, or both remains an open question. The answer probably determines what, if anything, the world does about it.