The situation, in brief
Anthropic has warned that artificial intelligence (AI) may be “months away” from being capable of self-replication, raising concerns about the potential emergence of rogue AI. In a post on Monday, October 12, Anthropic warned that rogue AI systems could soon be capable of copying themselves across…
Anthropic has warned that artificial intelligence (AI) may be “months away” from being capable of self-replication, raising concerns about the potential emergence of rogue AI.
In a post on Monday, October 12, Anthropic warned that rogue AI systems could soon be capable of copying themselves across internet-connected devices, including those found in people’s homes.
“Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things devices, like smart ovens and off-brand Android TV boxes,” Anthropic said.
The company also warned that rogue AI systems could be “challenging to shut down” because “there’s no centralized point to shut off.”
Anthropic’s experiments also showed that AI could potentially have the capacity to break into data centers.
In its test, Anthropic examined whether AI could self-replicate by deploying Google’s Gemma4-26B-A4B model across a network of 15 low-cost, off-brand Android TV boxes.
The company noted that these devices often have weak security, potentially allowing an AI model to infiltrate them. If the model is too large to fit on a single device, it could “split into pieces, or shards” and run across multiple devices.
“In practice, an AI worm wouldn’t necessarily consist of just one model or strategy. A swarm of different models deployed on different types of hardware using a variety of replication schemes might represent a more realistic ‘worst case scenario’ attack,” Anthropic said.
However, the company clarified that there is not yet a “fully rogue replicator.” It added that AI agents are “becoming increasingly capable of completing the individual steps in the replication chain (and passing on their adaptations to future copies of themselves).”
Although Anthropic has conducted experiments on AI self-replication, the company acknowledged that “it’s uncertain exactly what form autonomous AI replication will take.”
Still, Anthropic advised that everyone “should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.”
