‘People building AI believe it could kill us all’ — says researcher after quitting AI firm

n n

A former Anthropic researcher has warned that artificial intelligence (AI) could become powerful enough to threaten humanity, saying some people building the technology believe it could “kill us all by the end of the decade.”

n

Jacob Coxon, 27, made the remarks after leaving the company in a resignation thread on X.

n

“These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” Coxon wrote.

n

“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”

n

Coxon’s concerns were backed by Evan Hubinger, a senior Anthropic employee, who responded: “We really do earnestly believe AI could kill all humans!”

n

Hubinger said he personally puts the probability at under 10% over the next decade, while acknowledging concerns that Anthropic does not yet have a plan to prevent a future form of AI from becoming beyond human control.

n

He later clarified that Anthropic’s risk assessment says current AI systems have little chance of gaining such power.

n

Anthropic CEO Dario Amodei has also repeatedly warned about the risks associated with the rapid development of AI.

n

“This is a complex engineering problem and I think something will go wrong with someone’s AI system. Hopefully not ours,” Amodei told the New York Times in February.

n

The warnings come amid growing calls for tighter AI regulation. In July, nearly 1,400 AI company employees signed an open letter urging the US government to regulate the technology and slow its development.

n

OpenAI chief scientist Jakub Pachocki has also warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them.

n

Recall OpenAI CEO Sam Altman also acknowledged the need to consider slowing development, saying: “We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels.”

n n