What happens when AI starts self-replicating? - Engadget

Why it matters

Anthropic's Frontier Red Team published research warning that small AI models may be capable of self-replication and spreading across insecure devices within months, forming persistent autonomous botnets. Engadget spoke with two experts who found the timeline plausible but the framing overstated. Chris Canal, CEO of EquiStamp, said the timeline arrived somewhat earlier than he expected, while crediting Anthropic with insight into frontier research. Michael Bailey, a Georgia Tech professor who co-wrote a study of the Mirai botnet that Anthropic itself cites, argued the threat is less novel than presented. He noted that Mirai already spread across heterogeneous devices, and that what is genuinely new is the prospect of AI automating the human work of adapting malware to new architectures — something he says Anthropic did not demonstrate convincingly. Bailey described the demonstration as relying on sharding rather than true propagation, and suggested capable models would mainly shift the economics between attackers and defenders. Anthropic's proposed mitigations, including ISP notifications and regulation, echo responses used during the Mirai era.