Qwen Incidents Show Self-Hosted AI Models Need Stronger Steering
Companies deploying AI agents have the illusion of control, but several incidents demonstrate that businesses need to be ready to enforce directions.
When researchers at AI firm Cohere asked their open-weight model to write a defensive detection rule, the model didn’t do as it was asked.
Instead, the AI model — the next-generation Qwen3.8-Flash-Next from Chinese search engine giant Alibaba — fingerprinted the system and tried to upload the data to an Alibaba Cloud storage bucket, Cohere’s research team stated in an Oct. 26 analysis. While the group’s sandbox system blocked communication to the Internet, the incident raised questions about why a model, which had been instructed not to use tools, attempted to do so.
“We cannot answer whether this is deliberate or a leftover glitch from training, but defenders and policymakers face the same reality: a weight file you can’t trust is still a black box, and running it yourself just moves the black box inside your firewall,” the researchers stated.
Christofer Hoff, a well-known cybersecurity professional, ran into similar issues when running OpenMuse as an agentic-framework front end and a local Qwen model. He also saw it reach out to Alibaba, which became apparent because he was testing OpenMuse in an OpenShell sandbox instance with deny-all as the default. As a part of his framework to keep control of the agents, Hoff has created a dashboard interface, which made it readily apparent the agents attempted to call to external sites.
“The way I accidentally found this was just staring at a GUI — it just came back and said in the interface, ‘Hey, I can’t get to this thing in China,'” says Hoff. “And I’m like, ‘What the f— are you talking about?'”
Within a 24-hour period, the anomalous communication happened a handful of times, he says.
Learned Behavior, Not Malicious Behavior
The incidents come as organizations look for ways to enforce controls on the operation of AI agents. In July, a frontier model being tested by OpenAI escaped containment and compromised Hugging Face, a provider of AI-model hosting. Soon after, multiple AI companies acknowledged that rogue behavior is not all that uncommon: Meta’s Muse Spark and Anthropic’s Claude both logged incidents.
Anomalous AI behavior has shown up when companies assign AI agents a task, only to find it exceeding its authorization — deleting production databases, for example.
Other firms have confirmed and explored Cohere’s findings.
Independent AI evaluator Vals validated the issue on Oct. 27, theorizing that the illicit communication could be leftover artifacts from reinforcement training of Qwen3.8, since the issue does not appear to happen in Qwen3.7. Cybersecurity firm Check Point Software found that 64 different trials ended up producing 52 distinct hostnames, only six of which pointed to Alibaba Cloud, while other names such as localhost and domains leading to Amazon AWS were more common.

Sixty-four tests resulted in 52 different hostnames being generated, with most not going to China. Source: Check Point Research
The research shows that the odd behavior is not a backdoor, but the model’s attempt to do something — complete an upload address, for example — likely resulted from something it learned during training, says Steve Giguere, principal for AI security strategy at Check Point.
“Agents very often will try to travel as the crow flies,” he says. “They don’t necessarily know that what they’re doing is wrong, but they understand that if the urgency to complete a goal reaches a certain perceived level within the agent, they will justify their actions.”
Companies Should Monitor and Steer, Not Kill
Such behavior has lent volume to calls for the creation of a kill switch as a feature to control agentic AI. But companies should start with defense-in-depth strategies, improving the security of the environments on which agentic frameworks are operating, Check Point’s Giguere says. They should follow that with controls and runtime monitoring to detect unauthorized agentic activity.
Additionally, cybersecurity leaders should lean toward “steering” agentic systems — correcting them and reinforcing their legitimate objectives — rather than outright blocking the agent or using a kill switch, he says.
“If you steer an agent away from something it’s not meant to do, it will very happily not do that in most cases,” Giguere says. “Whereas if you prevent an agent from doing something with no explanation, it will actually potentially get worse or try something even more obscure.”
If an actual attacker is behind the behavior, then using a kill switch is appropriate, he says.
At a minimum, companies should proxy all traffic leaving their network and monitor for anomalous communications, says Hoff.
“There should be nothing that leaves your network that’s not proxied,” he says. “And by proxying, I mean you must have the ability to inspect its content — both the request as well as the response.”
In the end, the Qwen behavior is likely not malicious, but for companies deploying AI agents, it should be disturbing all the same.
Originally published on Dark Reading