Srsly Risky Biz: Phew! Some Regulatory Relief for AI Hacks
Your weekly dose of Seriously Risky Business news is written by Tom Uren and edited by Patrick Gray and Amberleigh Jack. This week’s edition is sponsored by Thinkst.
You can hear a podcast discussion of this newsletter by searching for “Risky Business News” in your podcatcher or subscribing via this RSS feed.
The Trump administration’s AI task force will encourage regulatory agencies to go easy on organisations that fall victim to AI-powered cyber attacks. Last week the task force said in a statement that AI has changed the risk environment, and therefore “prior standards for taking regulatory action in response to cyber incidents may be outdated”.
The idea here is to encourage good-faith, voluntary reporting of vulnerabilities and incidents because a wave of strict enforcement actions in the current environment could have a “chilling effect on reporting and information sharing”. The task force described such reporting as “critical to our broad risk mitigation efforts”. The suggested regulatory leniency would last for at least 12 months, depending upon how things pan out.
This makes sense, at least in the short-term. It would be perverse to punish victims for having security settings that were appropriate last year, but may not be today.
AI has changed the equilibrium between attackers and defenders, so policymakers are right to adjust standards and expectations to fit the new reality of cheap, easy and fast AI-driven attacks.
Rather than just giving victims a short-term, get-out-of-jail-free card to deal with the problem, however, we’d like to see more forward-looking policy that is aimed at encouraging organisations to adapt to the new normal.
The call for leniency, for example, could be expanded to allow organisations greater leeway to experiment with defensive AI efforts. If we want to find our way back to the acceptable equilibrium between attackers and defenders, organisations need some freedom to experiment with AI-enhanced defence. Take AI red teaming, for example. If a test were to go awry despite genuine efforts to be careful, a little government tolerance might be needed.
Governments should also encourage organisations to raise the priority of cyber security and devote more management attention and even resources to it.
Finally, governments need to flesh out their practical, AI-specific guidance. We are aware of some high-level advice, a joint Five Eyes cyber security agency statement from June and a slightly more detailed ASD document from July. But “one and done” blog posts from a couple of agencies isn’t enough. We’d hope to see these statements released by more governments and they should be living documents with advice that changes in real time with the technology.
Regulatory leniency will buy a small amount of breathing room, but ultimately it’s a temporary stopgap. Governments should be encouraging sensible experimentation, identifying approaches that work, and then prodding organisations to adopt them.
America’s Rules vs China’s Money: Fight!
On Monday, the Commerce Department’s Bureau of Industry and Security (BIS) published new AI and semiconductor export regulations that tighten controls on who can buy AI chips and what they can be used for. The plan is to stop China’s AI labs from being able to acquire Nvidia chips or rent compute from Southeast Asian data centres.
It’s a nice start, but this won’t stop Chinese AI firms using Nvidia’s chips without a lot more work.
America’s overall intent here is to maintain its lead in AI by preventing China’s firms from accessing advanced AI chips. The regulation aims to close several commonly exploited loopholes in the old rules.
For a start the new regulations explicitly make it harder to smuggle chips to China by requiring the US government to review export licenses to Singapore, Malaysia, Thailand, Indonesia, and Brazil. These countries are all large importers of Nvidia chips, and according to BIS, “present a higher-than-normal risk of diversion” to the PRC.
This week, for example, The Wall Street Journal reported that Singapore-based cloud company Megaspeed arranged to send nearly USD$12 billion worth of Nvidia chips to China-based companies over a 15 month period from March 2024.
The new regulation also cracks down on firms from China simply renting access to advanced AI chips hosted in Southeast Asian data centres. This was legal, and there are several reports indicating this is a common practice. Senior White House science policy advisor Michael Kratsios claimed Chinese firm Moonshot AI trained its Kimi K3 model in a data centre in Thailand.
Under the new BIS regulation, foreign data centres will be required to ensure customers in China and arms-embargoed countries cannot access remote compute resources above certain limits.
It’s great that the new regulation addresses these loopholes, but we expect it will merely make them a bit harder to squeeze through. Closing them properly will require a lot more work.
The fundamental problem is that the BIS’s regulations are asking companies in Southeast Asia to choose between America’s rules or China’s money. There’s an awful lot of yuan available for companies willing to skirt the BIS’s requirements, when it comes to both remote access and the physical movement of chips.
Chip smuggling will be a tough nut to crack. In March of this year, the Department of Justice charged the co-founder of American company Super Micro Computer and two Taiwanese co-conspirators with diverting USD$2.5 billion of servers containing Nvidia chips to China. The DoJ alleged that the trio falsified documents and repackaged servers into unmarked boxes to ship them abroad. Prosecutors said they used dummy servers to deceive auditors and that surveillance camera footage showed them removing serial number stickers with hair dryers.
There’s money in this. Even a small margin on billions of dollars worth of servers adds up to a significant payday and simply tightening the rules won’t stop people from sidestepping them.
When it comes to remote access, the BIS rule says that data centre operators “will be expected to perform robust due diligence on their customers”. Without strong oversight we suspect that “robust” will be a subjective term that quickly devolves to “bare minimum” without strict audits.
And here’s the rub. The BIS only has eleven export control officers worldwide. The sole export control officer based in Singapore is also responsible for chip control across Malaysia and Thailand. That’s a lot of territory for one person, even if they are receiving support from other agencies. This year BIS has asked Congress for additional funding to bolster its enforcement capabilities, but even if that is granted it will still take time to spin up its activities.
Preventing China from getting access to advanced AI chips won’t be achieved by simply publishing better regulations. Government agencies need to follow up with on-the-ground enforcement that ensures that the rules are being followed. China’s companies will always be looking for new workarounds, so these rules may need to be updated over time.
The real headscratcher here is that, for reasons we don’t understand, America has done nothing to enforce chip controls for the last 18 months or so. As a result, China’s AI sector has had a field day.
These firms have enduring reasons to maintain access to advanced American AI chips, at least for now. The US government needs to respond with a similar lasting effort, rather than sitting back and hoping that 132 pages of regulation will do.
Watch Amberleigh Jack and Tom Uren discuss this edition of the newsletter:
Three Reasons to Be Cheerful This Week:
- Endgame keeps kicking goals: Europol announced that coordinated action by law enforcement agencies and private sector partners resulted in the disruption of three malware networks, Gootloader, CastleRAT and StealC. These are all cybercrime-as-a-service offerings and are used by a variety of criminals to enable ransomware and data theft.
- US freezes $1.2 billion from Binance hack: The US Department of Justice announced that it had frozen the funds and also seized infrastructure used by the North Korean FaceStealer social engineering scheme. It says FaceStealer is “a highly effective social engineering scheme that uses imposter versions of popular video conferencing platforms” and targets companies and individuals in the cryptocurrency industry.
- Lockdown mode for Microsoft Edge: Microsoft has posted about improving the security of the Edge browser including by hardening by rewriting core parts of the browser in the memory-safe Rust language, improving the delivery of security updates and working on what they call Edge Lockdown mode. It goes further than Edge’s Enhanced Security Mode, which disables just-in-time javascript compilation, and also disables other browser features that are also used in attacks.
Sponsor Section
In this Risky Business sponsor interview, Catalin Cimpanu talks with Thinkst CTO Marco Slaviero about the state of the defensive AI cyber space and the company’s current agentic canaries, as well as the need for more experimentation.
Risky Biz Talks
You can find the audio edition of this newsletter and other fine podcasts and interviews in the Risky Biz News feed (RSS, iTunes or Spotify).
In our last “Between Two Nerds” discussion Tom Uren and The Grugq discuss why AI means that mercenary spyware is a dead industry. The pair discuss why governments would rather buy the cow than the milk.
Or watch it on YouTube!
From Risky Bulletin:
Hackers hijack Optimizely to deliver malware on 178 popular sites:
For about 16 hours on Monday, November 16, the Optimizely A/B testing platform delivered malicious code on at least 178 popular websites across the globe.
According to an Optimizely statement, the hacker exploited a vulnerability in its “Experimentation” platform to add custom code to certain customers.
According to technical write-ups of what happened—via Report-URI, Step Security, and Amirul Iman—the code went through different versions across the day.
[more on Risky Bulletin]
Hackers accessed Barracuda’s cloud: A threat actor has gained access to the cloud environment of security firm Barracuda in a security breach that took place earlier this month and lasted for almost two days.
In a post-mortem, the company says it traced back the entry point to a cloud access key it issued to a third-party AI service provider and which it forgot to revoke after it finished working with the platform.
The attacker used the key to access its cloud environment and then deployed the ARTEX AI penetration testing framework to escalate its access across its internal network and other cloud environments.
[more on Risky Bulletin]
Chinese APT uses SS7, Diameter attacks to spy on targets: A newly discovered Chinese cyber-espionage group is hacking telcos across Asia to exploit the SS7 and Diameter protocols and spy on carefully selected targets.
The new SilentAir group has been active for at least two years, since early 2025, according to telco security firm ENEA, which has been tracking its operations.
The group is hacking telco infrastructure and planting backdoors that run malicious SS7 and Diameter commands designed to collect device data, track a subscriber’s location, and even intercept calls and SMS traffic.
[more on Risky Bulletin]
Originally published on Risky Business News