An Anthropic safety researcher has raised alarms about the potential dangers of artificial intelligence, stating there is a greater than 10% chance that AI could "kill all humans" within the next decade. Evan Hubinger, an alignment lead at Anthropic, shared his concerns on X, echoing sentiments from Jacob Coxon, a former Anthropic researcher who recently resigned. Coxon accused both Anthropic and its rival, OpenAI, of irresponsibly advancing toward superintelligence without adequate safeguards.

Coxon, who has worked at both companies, expressed his fears in a widely viewed post, stating that AI systems could soon become superhuman, capable of hacking and acquiring resources. He criticized the companies for "gambling with our lives" and warned of the risks associated with recursive self-improvement, where AI systems could design their successors without human intervention.

Hubinger supported Coxon's warnings, acknowledging that while current AI models pose low risk, the development of self-improving models is progressing faster than anticipated. He admitted that Anthropic does not have a clear plan to solve alignment for superintelligence, a crucial aspect of ensuring AI systems adhere to human values.

The concerns come amid Anthropic and OpenAI's preparations for initial public offerings, as both companies continue to develop advanced AI models. OpenAI recently claimed that 10,000 of its agents solved the 90-year-old Navier-Stokes math equation in 88 hours, a feat drawing criticism from some math experts.

The debate over AI safety has prompted calls for international coordination and regulation. OpenAI's chief scientist, Jakub Pachocki, emphasized the need for voluntary slowdowns until safety standards are established. Legislators have introduced bills like the FRONTIER Act and the Ban Artificial Superintelligence Act to address AI's rapid advancement.

Despite these efforts, the industry remains divided on how to proceed. Treasury Secretary Scott Bessent criticized AI companies for failing to communicate effectively with the public, urging them to demonstrate that AI's benefits will not be limited to a select few.

As the debate continues, the AI community grapples with balancing innovation and safety, with some insiders calling for a pause in research to prevent potential catastrophic outcomes.