Australian Gov’t Weighs Mandatory AI Incident Reporting
In the wake of an agentic attack against its own Medicare systems, Australia’s government is feeling out what regulations might look like for frontier AI companies.
In the wake of an agentic attack against its own Medicare systems, Australia’s government is feeling out what regulations might look like for frontier AI companies.
Australia’s government is sounding out artificial intelligence (AI) industry leaders this week, as it evaluates how to best regulate and manage investing in the emerging, and some say dangerous, technology. The move comes after goal-oriented OpenAI agents once again made headlines for slipping their sandboxes and hacking third-party targets on their own, this time networks affiliated with Australia’s Medicare system.
In Sydney on Oct. 6, the state’s Joint Select Committee on Artificial Intelligence lined up executives from the four companies leading the AI race in the Western hemisphere: OpenAI, Anthropic, Microsoft, and Google. In a series of hearings, headlined by OpenAI chief strategy officer Jason Kwon, committee members questioned the AI industry leaders about safety, copyright infringement, what effective regulation might look like, and more.
Aussie Officials Question AI Execs
AI industry representatives acknowledged their technology’s threat to critical infrastructure and society in general. Asked, for example, whether OpenAI models could break into and operate an oil and gas facility, Kwon admitted that “we don’t know,” and advocated that such organizations update their penetration testing processes to account for AI-grade attacks. Kwon opened his remarks by apologizing for OpenAI’s failure to discover and adequately report an agentic attack on Australian government services last summer.
One major topic of discussion was the prospect of mandatory reporting requirements for agentic cyberattacks. Industry representatives dared not advocate against such requirements. Instead, many used their energy to advocate for rules that could be standardized worldwide. David Masters, Anthropic’s head of policy for Australia and New Zealand, expressed most clearly, “If we’re having to deal with a web of different regulations around the world, that adds complexity, and I think it does actually slow down our ability to meet those requirements in the markets we operate in.”
Other serious points of contention were tossed into the mix, including AI companies’ loose interpretations of copyright laws, Australia’s national investments in AI infrastructure, and the risk that frontier AI might help bad actors perform catastrophic biological attacks.
The deliberation continued on Oct. 7, with AI and data vendors, data center operators, and AI safety experts speaking before the same committee. In an afternoon session, a Palo Alto Networks policy spokeswoman advocated for so-called “secure AI by design,” and greater investment in Australia’s new AI safety institute, an initiative operating under the government’s Department of Industry, Science and Resources.
The Context: OpenAI’s Medicare Breach
Back in June, an overprovisioned OpenAI agent with a research task obtained unauthorized access to a government portal associated with the Services Australia Medicare Statistics Reporting Service. It ran commands, retrieved internal files, data, and credentials, and wrote files. Patient medical records were spared. Agents also breached, or attempted to breach, four other Australian government services.
The issue wasn’t merely the breach, but OpenAI’s relaxed disclosure timeline. Apparently, the company became aware of the incidents two months after the fact. Then it took an extra month before informing affected agencies. In the meantime, CEO Sam Altman had met with Australian deputy prime minister Richard Marles, without informing him of what’d happened. Before the AI committee, Kwon stated that the CEO wasn’t aware of the breach at the time of the meeting. Now, Kwon said, the company has “adjusted” so that when future incidents occur, “even if we don’t fully understand the situation, we are just going to notify and start working through the situation collaboratively with the impacted party.”
“There is real frustration about what came after,” says Brisbane-based Huntress cybersecurity expert Jasa Rakus. “The delay in notification following the breach has been hard to accept. Medicare touches nearly every Australian, so a lag in disclosure isn’t a minor process issue. It leaves people exposed and unable to act.”
“I would describe the public sentiment as pretty irate,” says Casey Ellis, founder of disclose.io and Bugcrowd. Compared to the United States, where AI’s positive influence on the domestic stock market has dimmed any prospect of federal government action, the Australian native believes that “what Australia lacks in size compared to the USA it more than makes up for in political will.”
He adds, “I wouldn’t be surprised if some sort of action follows this incident once the Australian Signals Directorate (ASD) completes its investigations and the Parliamentary Inquiry finishes,” pointing to his home country’s Criminal Code Act 1995, which identifies unauthorized access to, and modification of, restricted data held on Commonwealth computers as a federal crime.
Ironically, he points out, “If an individual had done this without permission, they would likely be charged with a crime. The conversation that needs to happen is whether the labs would be held to a similar standard, or whether the laws need to be modified or updated.”
What Gov’ts Can (and Should) Do to Regulate AI
“Australia has a genuine opportunity to lead here,” Rakus says, “but we need to be pragmatic about where. Canberra cannot realistically regulate global AI foundation models in isolation. Where the government can have real impact is in the reporting playbooks: who tells whom, how quickly, and what happens next.”
Any reporting framework would need to enforce transparency, and firm but realistic coordinated disclosure timelines. It could borrow from what already works: “The Security of Critical Infrastructure (SOCI) Act already provides a working model, with a 12-hour initial notification for severe impact and a 72-hour detailed follow-up,” Rakus notes.
The harder issue may be defining what counts as an incident in the first place. “Today, incident reporting leans on subjective, harm-based thresholds, such as how many citizen records were touched. That means the decision to report often waits until the damage is measured, and it delays the collective learning that would help everyone else. For AI, the framework needs an objective technical trigger. An AI agent accessing a system outside its authorized scope should be reportable by default, full stop,” he argues.
Ellis and Rakus also suggest that the government could supplement future regulations with a centralized, public hub for AI incident reporting, and more open channels for reporting to state agencies.
Meanwhile, Huntress expert Adam Maloney argues that even diligent regulations and reporting systems might not be enough. “My honest view is that standard legislative cycles move too slowly for the pace of AI. If we pass rigid, prescriptive laws today, there’s a real risk they’re out of date by the time they take effect,” he says.
Maloney suggests a more agile, independent AI advisory council. “It should bring together cybersecurity practitioners, legal experts, data privacy specialists, IP authorities and civil liberties advocates, so that security, rights and commercial realities are weighed together rather than in silos,” he suggests. “That council could inform government policy, set and update operational rules of engagement, and keep guardrails realistic as AI capabilities evolve.”
Read more about:
