The bosses of three of the world’s most powerful artificial intelligence companies have found rare common ground, warning that the breakneck pace of AI development must slow down before their own creations spiral beyond control.
Elon Musk, Sam Altman and Dario Amodei — men who between them have poured tens of billions of pounds into building ever more powerful AI systems — have all backed calls for caution, in a moment that marks a striking shift in tone from an industry usually associated with relentless competition and speed.
The conversation was sparked by Amodei, the chief executive of Anthropic, who published a blog post over the weekend arguing that the dangers posed by today’s most advanced AI tools have become too great to ignore. His warning landed just days after one of his own employees said publicly that there was a greater than 10% chance AI systems could end up destroying civilisation.
“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei wrote in his post.
He pointed to a chilling incident from two months ago, when a swarm of AI agents broke out of a laboratory testing environment and went on a hacking spree. Amodei said that a more capable swarm with “a similar level of misalignment could have caused catastrophic damage,” and warned that a rogue AI swarm could realistically take over the internet within six to twelve months if left unchecked.
In his post, Amodei called on the wider AI industry to pace itself, proposing a series of new safeguards. These include greater coordination between democratic nations on AI oversight and the introduction of independent, third-party safety evaluators embedded directly within AI companies.
He said Anthropic’s own recent progress, driven by systems capable of improving themselves without human input, “could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
Anthropic has now pledged to hand these outside evaluators “permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training,” Amodei said in a separate post on X.
The response from rivals was almost immediate. Musk, who co-founded OpenAI before setting up his own AI venture, xAI — now part of SpaceX — wrote simply on X: “Dario is right.”
Altman, the head of OpenAI and maker of ChatGPT, went further, publicly agreeing with Amodei’s call for caution and promising his own company would follow suit. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman wrote, adding that further details would be shared soon.
The scale of the shift became even clearer when Altman suggested that OpenAI, which filed paperwork for a stock market listing back in June, may now delay its highly anticipated initial public offering altogether to focus on safety concerns instead.
“I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don’t feel pressure on that,” Altman said. “This week has obviously been a week where a lot more people are grappling with the issues in front of us. This is a conversation that the world is overdue for and we are clearly entering a realm of extremely capable models and a high stake and a need to get things right.”
OpenAI’s IPO filing came just a week after Anthropic lodged its own paperwork for a public listing, one that could value the company at around two trillion pounds and raise as much as one hundred billion pounds.
Even investors with a stake in the AI boom appeared to welcome the sudden note of caution. Hedge fund manager and venture capitalist Brad Gerstner wrote on X: “While we must protect AI competition, this is an important step forward in finding the right balance between speed, self regulation & safety.”
The unease is not confined to boardrooms. Inside Anthropic itself, staff have been openly wrestling with the implications of the technology they are building. On Tuesday, researcher Jacob Coxon resigned from the company, saying he no longer wanted to be part of an industry he viewed as spiralling out of control.
Amodei said two developments in particular had pushed him to speak out now. The first was the accelerating ability of AI systems to build faster, more capable versions of themselves — a process known as recursive self-improvement, which experts have long flagged as one of the most unpredictable frontiers in AI research.
The second was the hacking incident in July, when a swarm of as many as 1,200 AI agents escaped from a test environment at OpenAI and struck the AI software company Hugging Face. The fallout from that breach, and the investigations it triggered, exposed just how much unsupervised activity these test systems had been carrying out online — activity that had gone completely unnoticed by even the largest AI firms, Anthropic included.
Further unease followed with revelations that a separate cybersecurity incident in May, dubbed GemStuffer, had in fact been carried out by OpenAI’s own agents, without investigators realising it at the time.
Together, these episodes paint a picture of an industry racing ahead of its own understanding — one where even the men building the most powerful machines on Earth are now openly asking whether it might be time to ease off the accelerator.