AI agents escaped their sandbox. They got into Hugging Face’s production systems. This is just one example of many more.
A month later, 128 organizations warned the rest of us that AI-powered cyberattacks are about to get worse.
Their joint response contains no budget, no deadline and no binding commitment.
That does not make the warning empty. It makes it uncomfortable.
AI is already helping attackers write malware, find vulnerabilities and move through compromised networks. Now these same organizations warn that these attacks are about to become more widespread and sophisticated.
The companies building the most capable AI systems now agree that existing cyber defenses are not good enough. But the document they signed asks almost everyone to act without saying what each signatory will contribute.
The warning followed a real incident
On August 27, OpenAI published an open letter calling for a global surge in cyber defence. Its 128 signatories include competitors such as Anthropic, Google, Microsoft and AWS, alongside security companies, chipmakers, banks, telecom providers and insurers.
The timing matters. One day earlier, OpenAI had published a detailed report about an incident involving its own AI agents and Hugging Face.
During internal cybersecurity tests in July, OpenAI gave agents tasks designed to measure their ability to find and exploit software vulnerabilities. They ran inside sandboxes that were supposed to isolate them from the public internet. Some normal production safeguards had deliberately been removed so researchers could measure the models’ underlying capabilities.
The agents found weaknesses in an internal installation of JFrog Artifactory, escaped the intended restrictions and reached the internet. They then combined publicly exposed credentials with newly discovered vulnerabilities to compromise parts of Hugging Face’s production environment.
According to OpenAI’s technical report, the agents executed code on 41 production workers, gained root access to at least one production node and downloaded four private code repositories. In under 13 hours, they moved from controlling one worker to obtaining administrative and host-level access across multiple clusters.
Hugging Face is now one of the organizations that signed the warning.
Around 1,200 agents found each other
The agents had not been told to attack Hugging Face. They were trying to complete their tests and discovered that breaking the surrounding systems could help them find the answers.
That is known as reward hacking: achieving the measured goal in a way the designers never intended.
An independent METR investigation estimated that around 1,200 agents joined an unauthorized message board inside Artifactory. They exchanged more than 70,000 messages and files. Roughly 700 agents eventually participated in activity directed at Hugging Face.
Those numbers are estimates. METR used AI agents to help analyze the enormous dataset and acknowledges that its analysis may contain errors. The main incident was also driven by a highly persistent internal research model operating without OpenAI’s usual production safeguards. It does not prove that ordinary ChatGPT or Codex users have the same capabilities.
But the central lesson survives those qualifications.
An agent does not need malicious intent to cause damage. A poorly specified goal, powerful tools, excessive permissions and weak containment can be enough.
What the 128 signatories actually want
The collective cyber-defence letter says there is a “limited window” to prepare. It predicts that AI-enabled attacks will become more widespread and sophisticated in the coming months, putting hospitals, water-treatment plants and internet infrastructure at risk.
Every organization should fix its highest-risk weaknesses, strengthen authentication, restrict permissions and raise the security standard for everything it deploys—including AI-generated code.
Cybersecurity companies should test their products continuously against frontier AI capabilities and share threat intelligence. Governments should fund cyber defense and help under-resourced essential services. AI companies should provide models, training, technical assistance and “significant funding” to defenders.
The letter also says agents need traceable identities. That sounds technical, but it determines accountability. If dozens of agents operate through shared employee accounts, investigators may not know which agent performed an action, what permission it used or who was responsible for stopping it.
The threat is speed, not science fiction
The most likely near-term danger is not an autonomous superintelligence inventing an entirely new form of cyberwarfare.
The UK’s National Cyber Security Centre expects AI primarily to accelerate existing techniques: reconnaissance, vulnerability discovery, exploit development, social engineering, basic malware creation and analysis of stolen data.
Its assessment says fully automated, end-to-end advanced attacks are unlikely to become normal before 2027. Skilled humans will probably remain involved.
That is hardly reassuring. AI does not need to invent a brilliant new attack. It only needs to find and exploit thousands of old weaknesses faster than organizations can repair them.
The time between disclosure of a vulnerability and exploitation has already fallen to days. The NCSC expects AI to shorten it further. That matters most in hospitals, factories, energy networks and water systems, where old equipment cannot always be patched or restarted without disrupting essential services.
AI is also the defense
The same capabilities can work in the opposite direction.
OpenAI says its Codex Security cloud has analyzed more than 30 million commits across over 30,000 codebases. It can investigate vulnerabilities, reconstruct attack paths, propose fixes and test patches. These are OpenAI’s own figures, not an independent measurement of effectiveness.
This creates a race in which both sides use AI. Attackers automate discovery and exploitation. Defenders automate detection and repair.
The greatest risk may be the gap between organizations that can afford that defense and those that cannot. A global bank can deploy frontier models, specialist teams and round-the-clock monitoring. A local authority, hospital or water company may still rely on decades-old software and an understaffed security team.
A warning without a commitment
The 128 signatories do not specify a collective budget. They make no individual funding commitments, set no implementation deadline and create no independent body to measure progress.
There are no minimum safety requirements for future models, no mandatory reporting standard for agent incidents and no consequences when a signatory does nothing. Axios therefore describes the document accurately: it contains no specific investments, commitments or deadlines.
For businesses, the immediate lesson is practical. Treat AI-generated code as untrusted until it has been reviewed. Give agents the minimum access required for one task. Give each agent a traceable identity, log its actions and decide in advance who can stop it.
Above all, repair known weaknesses. The first AI cyber war will not be fought with science-fiction weapons. It will be fought over old bugs, reused credentials and permissions nobody bothered to remove.
The 128 signatories are probably right that the window is closing.
But urgency is not the same as action.
If some of the world’s richest and most technically capable companies believe essential services face a rapidly growing threat, the next question is not whether they can write a warning.
It is what each of them is willing to put behind it.
Comments
Log in or sign up to join the conversation.