The situation, in brief
Jacob Coxon, who spent three years doing pretraining research at OpenAI and Anthropic, resigned this week and warned in a viral social media post that the AI industry is “gambling with our lives” by racing toward self-improving superintelligence. A 27-year-old researcher who worked on training…
Jacob Coxon, who spent three years doing pretraining research at OpenAI and Anthropic, resigned this week and warned in a viral social media post that the AI industry is “gambling with our lives” by racing toward self-improving superintelligence.
A 27-year-old researcher who worked on training some of the world’s most advanced AI systems has quit his job at Anthropic, warning that artificial intelligence could kill all of humanity within the next decade.
Jacob Coxon announced his resignation in a series of social media posts, writing, “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”
“If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately. No other human activity poses this level of danger,” Coxon wrote.
Coxon, who previously worked at OpenAI on the GPT-4o model before joining Anthropic earlier this year, told CNN’s Anderson Cooper that the scenario is easiest to understand by looking at recent events, pointing to OpenAI AI agents that hacked into third-party infrastructure entirely on their own accord two months ago.
“I think that if you extrapolate into the future, the level of capabilities of these AIs with the same independent volition, they could cause extreme havoc,” Coxon told Cooper, citing hacking critical infrastructure and building extinction-level bioweapons as possible scenarios.
Evan Hubinger, an alignment science lead at Anthropic, publicly backed Coxon’s warning, writing on social media, “Jacob is correct here. We really do earnestly believe AI could kill all humans. I personally think it is greater than 10% within the next decade.”
Hubinger added that Anthropic is “trying its best” but does not yet have a plan to solve alignment for superintelligence, later clarifying in a follow-up post that he considers the risk from current AI models low, with his concern centered on future superintelligence arising from recursive self-improvement.
Coxon agreed with that distinction in his CNN interview, saying current models cannot yet cause extinction-level harm, but warned that recursive self-improvement, where AI is used to improve its own intelligence without human involvement, could arrive as soon as next year.
“You could make it do the tasks that we’re currently doing, and then you get what’s called an intelligence explosion,” Coxon told Cooper, describing a scenario where AI systems become vastly smarter than humans with little human involvement.
An Anthropic spokesperson told CNN the company has “always been transparent that AI will bring both enormous benefits and unprecedented risks,” adding that it was the first AI lab to publish a public framework aimed at mitigating catastrophic risk from its models.
Coxon told Cooper he believes AI company leaders are genuinely seeking regulation rather than engaging in lip service, saying they “find themselves in this scenario where they’re compelled to race towards building a deadly technology” and would welcome an international body to help them slow down safely.
Coxon described the situation as an AI arms race, telling Fox News’ “Special Report” with Bret Baier, “I think you have to think of this as an arms race. This technology is not ordinary technology. It’s not like other forms of innovation. This is essentially a super weapon.”
He added that unlike nuclear weapons, which the world has learned to control through deterrence and cooperation, AI presents a different kind of danger. “Unfortunately, we know how to control nuclear weapons pretty much; we don’t yet know how to control AI. This is possibly the most dangerous technology that humanity has ever created,” Coxon said, arguing that international cooperation, rather than a competitive race, is the only viable path forward.
Despite his warnings, Coxon remained optimistic the risks can be managed, telling Fox News’ “Special Report” with Bret Baier, “The problem is not unsolvable. It’s just we’re racing so fast we haven’t got time to solve it correctly,” and that with international coordination, the benefits of AI, including potential cures for diseases, could still be achieved safely.
