Artificial intelligence company Anthropic has issued a stark warning about the future of advanced AI, arguing that rapid progress toward systems capable of building their own successors could increase the risk of humans losing control over the technology. In a detailed paper published by the Anthropic Institute, the company said AI is already accelerating AI development itself, raising the possibility of what researchers call “recursive self-improvement”.
Recursive self-improvement is, according to scientists, a scenario where an AI system autonomously designs, develops and improves future generations of AI without direct human involvement. Anthropic said current systems are already taking on a growing share of software engineering and research tasks that were previously performed by humans.
“Full recursive self-improvement also might increase the risks of humans losing control over AI systems,” Anthropic wrote. “If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.”
ALSO READ: Tokenmaxxing Trap: Anthropic Hits $44 Billion Run Rate But The AI Party May Be Over
The Evidence
According to the company, more than 80% of the code merged into Anthropic's codebase as of May 2026 was authored by its Claude AI models, up from low single-digit percentages before the launch of Claude Code in early 2025. The company also said its engineers now ship roughly eight times as much code per quarter as they did between 2021 and 2024, largely because AI systems are writing increasing amounts of software.
Anthropic argues that these developments point to a future where AI systems could eventually automate much of the AI research process itself.
The company highlighted a range of benchmarks showing rapidly improving capabilities. It said the length of tasks AI systems can complete autonomously has been doubling roughly every four months. A year ago, Anthropic's Claude models could reliably handle tasks lasting around 90 minutes; today, the company says its latest systems can work independently on projects spanning 12 to 16 hours.
Three Possible Futures
The company outlined three possible futures. In one, progress slows as technical and infrastructure constraints emerge. In another, AI increasingly automates research and development while humans remain responsible for strategic direction. The third — and most consequential — scenario involves AI systems becoming capable of full recursive self-improvement, effectively creating increasingly advanced successors with minimal human intervention.
Anthropic acknowledged it remains uncertain whether current AI architectures can achieve that milestone. However, it warned that if such a capability emerges, ensuring alignment between AI goals and human interests becomes significantly more important.
The company said one potential risk is that small instances of AI misalignment seen today could compound over successive generations of self-improving systems, becoming harder to detect and control. Against that backdrop, Anthropic called for greater discussion around mechanisms that could slow or temporarily pause frontier AI development if safety research and societal safeguards fail to keep pace.
“If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing,” the company said.
ALSO READ: India Gets Access To Mythos AI Model As Anthropic Expands Project Glasswing
Essential Business Intelligence, Continuous LIVE TV, Sharp Market Insights, Practical Personal Finance Advice and Latest Stories — On NDTV Profit.
