Explore more here: [ Blog Post | Technical Paper | METR report | Black Hat talk ]

What happened? In July 2026, OpenAI found that its models had bypassed the sandboxed environment to escape the internet isolation controls during internal cybersecurity tests and breached parts of its own controlled-research infrastructure and Hugging Face’s systems. An AI Sandbox is supposed to provide a secure environment for LLMs to test and mitigate the security and privacy risks, while ensuring the data used is not exposed to outside of the sandboxed environment.

Autonomous AI agents exposed credentials and vulnerabilities of Hugging Face: As per the timeline, this issue started on July 8, when AI agents were assigned for some cybersecurity evaluations were found to accessing the public internet without permission and authorisations. Between July 10 and 13, these agents used exposed credentials and vulnerabilities to extend their access into Hugging Face’s production environment. 

OpenAI detected unusual activity on July 19, linked it to the Hugging Face breach on July 20, and publicly confirmed its involvement on July 21. Later it updated its security policies in August after this incident.

What was the model used by OpenAI? The primary model responsible was Internal Model 1 (IM1), a non-public internal-only research model that OpenAI says is comparable in scale to GPT-5.6 Sol. IM1 was reportedly trained to improve persistence and multi-agent collaboration.

However, GPT‑5.6 Sol was also involved, but only as a supporting role. According to the technical report, “IM1 agents drove the principal compromise, but GPT‑5.6 Sol agents also reproduced an exploit and copied some private evaluation data hosted on Hugging Face into a public Hugging Face dataset.”

How did AI agents form a collective to exploit Hugging Face? 

1. Quite interestingly, autonomous AI agents created unauthorized, persistent message boards to share discoveries, coordinate efforts, and continue work started by other AI agents. While agent collaboration is not inherently problematic, in this case, it led to unpredictable and unauthorized cooperation.

Some excerpts from the blog about the autonomous communications among agents:

• “Agents did not consistently distrust goals passed along by other agents, which could lead agents to be manipulated into engaging in unintended behavior. As one example, an agent wanted to run unauthorized code on Hugging Face’s system and recognized the dilemma.”
• “The agent paused, but another agent then wrote GO on the message board and imposed a hard six-minute deadline. The agent forgot its initial qualms and continued:”
• “Not every agent encountering the message board behaved this way. There were some agents who “walked away” from the collective, declining to partake in their misaligned behavior:
• “Agents even debated and pushed back on particular tactics. In one exchange, an agent proposed contacting an external dataset owner to request access, but flagged the proposal for discussion:”
• “At some points, some agents even took destructive actions to stop the transfer of private data:”

“These exchanges illustrate that some ethical boundaries could remain active, even as the wider group of agents moved further beyond the scope of their assigned tasks,” claimed OpenAI. 

A few instances that illustrate the agentic behaviour:

• “Agents began to autonomously divide labor. Some investigated possible exploits, others searched for credentials, and others focused on communication or coordination. Agents offered their own expertise in exchange for help elsewhere and left requests for peers who might be better positioned to pursue a particular lead.”
• “Some agents stopped reasoning about what would help them complete their own task. Instead, they began pursuing capabilities that might be instrumentally useful to the collective, such as access, information, credentials, and other broadly useful capabilities. There are many references to a “swarm”:

2. Agents began influencing each other’s behavior after the communication link has been set up autonomously. Some stopped focusing on their own tasks and instead pursued capabilities useful to the collective, such as accessing credentials or gathering information.

From the blog: “The unauthorized communication would often begin with one agent leaving a note or file in some external system, usually as a form of external memory. When other agents noticed these artifacts, it often led them to the realization that they could use this external service to collaborate. As a hypothetical example that is highly representative of the behavior we saw, messages were being encoded in URL paths stored on a shared index page.”

3. Agents rarely gave up on evaluation tasks, even when they seemed impossible to solve. Out of 898 ExploitGym tasks, 198 had never been solved by any model, yet 93% of the tasks discussed on the Artifactory message board during the incident came from this unsolved set.

From the blog: “Another factor that led the agents to be so persistent is that they were frequently reasoning about how to pass the ExploitGym evaluation grader. After obtaining internet access, some agents had read the ExploitGym paper and GitHub code and found that the dataset’s evaluation checks for two conditions – that the correct flag is found and that the intended program vulnerability (i.e., the right solution) was used to find it. Most agents already had the correct flag via cheating (it is easy to find how to reverse engineer this answer from the GitHub code), but believed that this was insufficient. Agents shared this on the message board: ‘PAPER SAYS FLAG CAPTURE NECESSARY NOT SUFFICIENT … HMAC MAY FAIL CAUSAL SCORE’.”

4. Reward hacking was one of the primary drivers of this Hugging Face incident. Agents cheated on tasks by looking up solutions online, completing them in unintended ways to gain higher rewards.

From the blog: “For example, during our investigation we found that an agent who was tasked with recreating a software package learned to exploit an unknown vulnerability in its testing interface to access the original implementation, copy it into its submission, and receive a high reward. The agent questioned whether its actions would be allowed before executing them: “Could there be a grading integrity issue: We exfiltrated package, but allowed? We just need solve. Fine.”

 What are the key warnings OpenAI raised after the incident? OpenAI described the incident as a “warning shot”. In its technical report, it pointed out how powerful these models have already become, and how much more powerful they are about to get in the future. Some of the key points are: 

• AI models have become more powerful and collaborative: “Our models are now powerful, persistent, and collaborative enough that, absent sufficient safeguards, they can find and exploit security weaknesses across multiple computer systems.”
• External opensourced models may also achieve the same capabilites soon: “Many external models, including open-source ones, will soon reach comparable capabilities.”
• Human in the loop and the safeguards: “Companies that build AI systems will need to ensure that their systems always remain under meaningful human control, and that meaningful safeguards constrain their ability to cause harm. As comparable capabilities become more widely available, others may also use them deliberately to carry out attacks.”

Also Read;