For nearly a week, the internet has exploded over the looming threat of an artificial intelligence apocalypse. When former OpenAI and Anthropic employee Jacob Coxon posted that he’d quit the industry after seeing that “neither company is acting responsibly,” he immediately went viral, with his thread now having garnered more than 171 million views.
Among Coxon’s claims, his statement that “the people building AI earnestly believe that it could kill us all by the end of the decade” stood out the most, with other AI workers chiming in to say that, yes, the potential extinction of humanity is a common topic behind the scenes of the AI industry.
Now, OpenAI’s Sam Altman and Anthropic’s Dario Amodei—two of the biggest players in AI (and Coxon’s former bosses)—have weighed in on the discourse. The two tech CEOs have butted heads in the past, but on this, they agree: AI development needs to slow down, or a techno-apocalypse is a very real possibility.
Altman’s two paths to doomsday
Altman addressed the discourse with two posts on social media. In the first, he explained how though OpenAI has long been focused on the safe deployment of its AI models, it’s now treating development and evaluation with the same level of care. That includes slowing down the pace of overall development—though “pacing,” he emphasized, is not the same thing as “stopping.”
“Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs,” Altman wrote. “Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.”
In a follow-up post, Altman more directly spoke to the idea of an AI-based doomsday, saying he believes “there are two ways AI progress could go very badly and that we must avoid.”
First, Altman warned that “we could lose control of the future to AI,” an outcome he deemed “unacceptable.”
We are unapologetically on Team Humanity, and AI must always serve people,” he wrote. “To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.”
Second, Altman acknowledged the danger of an AI monopoly. “If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian,” he explained.
To avoid either scenario, Altman concluded, AI companies and governments must “[walk] a narrow middle path” where no single country or lab exerts too much control over the future of the technology.
Amodei’s race to the top
Where Altman took to social media to express his AI anxieties, Amodei posted an essay to his website titled, “We Must Pace the Frontier.”
Like Altman, Amodei advocated for slowing the pace of AI development without halting it altogether. “Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless,” he wrote. “We have sought a middle way: to show that it’s possible to build carefully and succeed commercially, and to make safety something on which AI companies compete. In other words, to create a race to the top.”
The Anthropic CEO highlighted two main concerns of his around the speed of AI development, including AI’s growing ability to develop new AI models on its own, also known as recursive self-improvement. Amodei also pointed to the OpenAI-Hugging Face incident as a small-scale example of how AI could potentially take over the entire internet and wreak havoc. “A swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage,” he wrote.
To avoid those outcomes, Amodei went on to propose a three-step plan to “pace the frontier,” which he defines as “building AI at a balanced rate that aims to ensure its safety while still achieving its benefits and grappling with important geopolitical dilemmas.”
Amodei’s first step is embedding evaluators, or giving third-party experts access to frontier AI companies’ development processes to ensure accountability. His second step is democratic coordination, or collaboration between AI companies in democratic countries to create common safety standards. His third step is global coordination, through which the U.S. government and other democratic governments would work to impose safety standards for AI development on authoritarian governments (though Amodei acknowledged this step had significantly more roadblocks than the other two).
“The measures I propose to advance the frontier at a safe pace will not be easy,” Amodei concluded. “But I believe we owe it to humanity to try.”