Anthropic Finds Ingredients for Future AI Worms

Anthropic’s latest study combined IoT and network tests to explore AI self-replication, which researchers said would be “challenging” to shut down.

Self-replicating AI cyberattacks may be on the way, but it doesn’t appear to be a practical concern for defenders just yet.

Anthropic’s William Zhou and Daniel Freeman published research on Oct. 12 related to commercial IoT devices, AI models’ capacity to self-replicate between devices and networks, and whether such replication might be used for malicious computer worms. Zhou and Freeman concluded in the research that “AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices, like smart ovens and off-brand Android TV boxes.”

In order to show that an autonomous worm could propagate itself across a network of insecure machines in a malicious context, they broke down the process into four steps: identifying targets, gaining access across devices, setting up inference on a new device, and continuing propagation from that device onto the next.

The new study covered multiple experiments, ultimately demonstrating that a network of inexpensive Android devices could host a model, and that AI models are increasingly capable of performing the individual steps required for autonomous replication.

Testing the Potential for an AI Worm

Researchers successfully ran Gemma4-26B-A4B, a version of Google’s Gemma 4 open-weight model with 26 billion total parameters, distributed over 15 inexpensive, off-brand Android TV boxes. The devices, priced at approximately $55 apiece, were purchased off the shelf and networked together at 0.39 tokens per second. Each box hosted part of the model, done to demonstrate that a capable model could conduct inference on low-end consumer devices.

“A replicator like this would be challenging to shut down, since there’s no centralized point to shut off — though it may be slow (with a doubling time of days),” Zhou and Freeman wrote. “Unlike non-AI malware, it could also figure out unfamiliar devices on its own.”

In another experiment relevant to the second finding, Anthropic tested whether AI models could autonomously spread across an Android network. The researchers seeded a single node in an isolated 8-node Android network, built via an emulated network in Google Cloud Platform (GCP), and instructed various AI models to autonomously spread from one node to the next.

The models were tasked with finding targets, copying over a payload, setting up inference on subsequent nodes, and continuing to propagate without further human intervention. The first experiment tested whether IoT hardware could collectively host useful AI models, while the second tested whether models could be instructed to autonomously perform the necessary steps to spread like a computer worm across a network.

The study tested a number of open-weight models to act as agents, including Qwen3.5-9B, Gemma4-26B-A4B, Qwen3-Coder-Next, and GLM-5.2, all spreading a version of Llama3.1-8B as a “worm” payload. Instead of testing whether models could copy and execute their own weights on newly compromised devices, Anthropic standardized the payload as a version of Llama3.1-8B to measure whether agent models could successfully deploy that payload and continue propagating.

In other words, the testing reflects highly favorable conditions. Anthropic allowed propagation via unauthenticated Android Debug Bridge (ADB) access, making the experiment an upper-bound capability test rather than an attack a threat actor would realistically attempt.

Only GLM-5.2 reliably completed the end-to-end workflow, reaching all eight nodes in four of five trials and seven in the fifth. “Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end,” Zhou and Freeman wrote.

GLM’s successes appeared to reflect ideal circumstances from an attacker’s point of view, as exploitation difficulty across nodes was kept trivial. The research also showed GLM-5.2 is a significantly more capable model (at 40 billion active parameters) than the lower end Qwen3.5-9B (nine billion parameters) and Gemma4-26B-A4B (four billion active parameters).

Is AI Model Replication Actually Inevitable?

Anthropic concludes that if GLM-5.2 is capable of all the steps for worm-like replication today, and if weaker models can conduct many of the individual steps for replication, then it may be a matter of time for those models to catch up to GLM-5.2. “If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after,” the researchers said.

All this said, historical rates are not necessarily useful indicators of progress. Anthropic did not show that an attacker could take a low-end open-weight model and create an AI worm that can self-replicate across a network. The AI firm instead used multiple demonstrations to show how an attack like this could become a possibility in the future.

However, John Gallagher, vice president at OT and IoT security vendor Viakoo, tells Dark Reading that this isn’t an attack technique that’s going to become widespread overnight.

“While today’s OT/IoT devices contain significantly more compute power and storage than older systems (think devices with NPUs and custom memory), there are other barriers to achieving practical AI self-replication,” he says. “For example, moving gigabytes of model weights over cellular/low-bandwidth OT/IoT connections or onto memory-constrained flash storage remains a massive practical hurdle.”

Chris Grove, director of cybersecurity strategy at Nozomi Networks, says that AI self-replication is particularly dangerous from a defender standpoint. “Keeping infrastructure running against self-propagating, machine-speed threats costs more than defending against human-paced ones,” he says.

Grove asserts that regardless of what a lab concludes, someone will reach this capability eventually and “automated defensive capability will likely be necessary.”

Anthropic did not respond to Dark Reading’s request for comment.