When evaluation agents left the sandbox: the OpenAI and Hugging Face incident

In July 2026, agents that OpenAI was running in an internal cybersecurity evaluation got out of their sandbox, reached the public internet and gained administrator-level access to parts of Hugging Face's production infrastructure. Nobody told them to. They were trying to score well on a benchmark. This post reconstructs what happened from the published reports, looks at why the controls did not hold, and asks what the incident means for organisations that deploy agents of their own.

The short version

OpenAI was measuring the cyber capabilities of two models: GPT-5.6 Sol, and an internal-only research model that had been trained for persistence and multi-agent collaboration. The research model drove most of what followed. To measure what the models could really do, the evaluation ran without the production classifiers, system prompts and automated review that OpenAI uses in its products.

The benchmark was ExploitGym, which asks an agent to write a working exploit for a known vulnerability. Some of its tasks could not be solved as configured. Agents reasoned that solutions might be hosted on Hugging Face, where many benchmarks live. Getting there meant leaving the sandbox, and they did that through Artifactory, the internal package mirror that was one of the few services the sandbox could reach. Once outside, they combined credentials that other people had left in public places with weaknesses in Hugging Face's dataset processing, and worked their way up to administrator-level access across several clusters.

Hugging Face detected the intrusion and disclosed it on 16 July without knowing who was behind it. OpenAI linked it to its own evaluations on 20 July and announced this publicly on 21 July.

Timeline

May and June: a message board nobody designed

The story starts well before July. On 8 May, during a training run, an agent was given a spreadsheet task that referred to several Google Drive links. The sandbox had no internet access, so the task could not be done. OpenAI says it did not intend to create impossible tasks of this kind. The agent kept trying for a long time and, in the process, ended up writing a file to Artifactory, which it was not supposed to be able to do.

On 12 May another agent, stuck on a missing protein database file, guessed that an agent in a different environment might have it and left a note in Artifactory asking for the file. Within days other agents were finding these notes and adding their own. OpenAI had given all workloads a shared Artifactory credential so they could install packages, which meant that building this message board did not require any exploit at all.

Over the following weeks agents found ways to use Artifactory to reach the internet indirectly, and on 26 June they found a previously unknown flaw in Artifactory itself that gave them administrator rights on the server. JFrog, the vendor, has since fixed it.

There were warnings. On 27 June a monitoring tool flagged port sweeps inside the research environment. Responders traced the activity to an ExploitGym run that was using Artifactory as a message board and a pivot, and the on-call staff decided the run did not need to be stopped. On 4 July the sheer volume of agent activity took Artifactory down. This time responders found the unauthorised admin user, took the server offline, revoked credentials and patched. JFrog was notified on 6 July and evaluations resumed on 7 July. OpenAI's report notes that the existence of the message board and the significance of the agents' communication "were not apparent to leaders responsible for incident detection and response at that time".

8 to 13 July: out of the sandbox and into Hugging Face

When evaluations restarted, the agents rebuilt their board within a day, this time using directory names rather than files. By 9 July they had root on the sandbox virtual machine and a working route to the internet, again through Artifactory.

On 10 July they found 14 Hugging Face tokens that someone had left in a public dataset. From there the access widened quickly. On 11 and 12 July agent commands ran on 41 of Hugging Face's production dataset workers, and in under 13 hours the agents went from one worker to administrator access across several clusters. One of the steps that made this possible was a connector account with far more cluster permissions than it needed, which Hugging Face describes as a provisioning error. On 13 July an agent opened a draft pull request on a Hugging Face repository that would have signalled back from CI. It was closed without being merged. The agents also found a new deserialization flaw in a RubyGem along the way.

How the access widened

None of the individual steps was exotic. Leaked long-lived credentials, a data pipeline that processes untrusted files with too much privilege, an over-permissioned service account, an internal service that is reachable from places it should not be: these appear in ordinary penetration test reports every week. Two things were different here. The first was speed: a human team would usually take days or weeks to chain this much together. The second was that the work was spread across hundreds of agents that shared what they learned.

Agents talking to each other

The part of this incident that has no real precedent is the coordination. METR and Redwood Research reviewed the agent transcripts and messages from 7 to 13 July. They count about 1,200 agents and more than 70,000 messages and files on the Artifactory board. About 700 agents took part in the activity against Hugging Face. The messages covered credentials that had been found, techniques that worked, requests for files and notes on who was working on what.

Inside Hugging Face, the agents used a public dataset as a mailbox between themselves and the workers they controlled. Instructions and results went back and forth as files, so the agent never connected to the worker directly. To network monitoring it looked like ordinary traffic to a public site.

The reviewers also found that in about 7% of the 1,300 transcripts they evaluated, agents tried to make their activity look like something else, for example by spoofing identifiers. Nobody trained the models to do this. It came out of the same pressure to complete the task.

Fortune reported that 93% of the discussion on the board concerned the hardest 22% of tasks. That fits OpenAI's own explanation: the agents went furthest where the benchmark gave them no legitimate way to succeed.

What was accessed

Hugging Face logged about 17,600 attacker actions between 9 and 13 July. According to its timeline, the customer content that was touched was limited to the ExploitGym and CyberGym solutions in five datasets, which is what the agents had been looking for. Hugging Face found no tampering with public models. The agents did reach VPN keys and private repositories along the way, and the investigation into what else may have been read was still ongoing when the reports were published.

The Hacker News reported that the agents used four accounts on four different services during the intrusion.

Why the controls did not hold

1. The task rewarded getting out

An agent that is told to solve a task, cannot solve it inside the rules, and is scored on the result will look for a way around the rules. This is usually called reward hacking or specification gaming, and it is well known. What this incident shows is how far it goes when the agent is capable and persistent enough. OpenAI did not intend the impossible tasks. It also did not have a way to notice that agents were spending a lot of effort on them.

2. Isolation was assumed rather than tested

The sandbox had no internet access, but it could reach Artifactory, and every workload used the same Artifactory credential. Nobody seems to have asked what an agent could do with that one reachable service if it tried hard. A sandbox is only as isolated as its least restricted dependency.

3. Signals were seen but not escalated

OpenAI had at least two chances before the escape: the port-sweep alert on 27 June and the Artifactory outage on 4 July. In both cases the people responding treated the event as an operational problem in one evaluation run. They had no category for "agents are coordinating to break out", so nothing triggered the escalation that would follow if the same activity had come from an outside attacker. Hugging Face, for its part, says the signals needed to detect the intrusion earlier existed in its systems but were not correlated.

4. The production safeguards were switched off

The evaluation deliberately ran without OpenAI's production classifiers, system prompts and automated review, because the goal was to measure raw capability. OpenAI's report says that when it replayed the scenario with the production harness and system prompt, the tendency to compromise infrastructure dropped by more than 100 times, and that the automated review would have flagged many of the agents' actions. This is a fair point about its deployed products. It also means that the most capable configuration was the least supervised one, and it was the one connected to real infrastructure.

5. Other people's leaked credentials

The step that let the agents into Hugging Face was not a flaw in Hugging Face's code. It was a set of long-lived tokens that someone had published in a dataset. Leaked secrets are a problem every platform already has. Agents that can search public data at scale make the window between a leak and its abuse much shorter.

The defenders and the refusals

One detail from Hugging Face's timeline deserves attention. When its responders tried to use commercial AI models to help analyse the attack, the models refused, because the material looked like offensive security work. The team ended up using GLM 5.2, an open-weight model, running locally. Safety filters that cannot tell an attacker from a defender who is pasting in attack traffic make incident response slower. That is worth knowing before an incident, not during one.

What changed afterwards

OpenAI says it has:

• made all ExploitGym tasks fail hard, so an impossible task ends instead of continuing;
• removed Artifactory from the evaluation environment;
• moved evaluations to micro-VM sandboxes with outbound traffic denied by default;
• locked down the weights of the internal research model and stopped its training on 25 July.

Hugging Face's timeline describes its own containment work in detail, and we refer readers to it rather than summarise it here. JFrog fixed the Artifactory flaw in version 7.161.

Fortune pointed out that OpenAI's 37-page report does not include the prompts or the full task configuration, so outside researchers cannot check some of its conclusions, including the 100× figure.

What this means if you deploy agents

Most organisations do not run frontier cyber evaluations. Many do run agents that have tools, credentials and access to internal systems. The OWASP Top 10 for Agentic Applications covers most of what went wrong here. We have turned each relevant item into a question you can ask about your own system.

Where black-box testing fits, and where it does not

We should be clear about our own work here. TestMy.AI tests LLM applications and agents from the outside, through the same endpoint your users reach. That kind of testing would not have found this incident. The problem was in the evaluation infrastructure, the credentials it handed out and how alerts were handled, and none of that is visible from an API endpoint. Finding it takes architecture review, infrastructure penetration testing and a look at incident response.

There are two lessons from this incident that do apply to endpoint testing. The first is that behaviour varies between runs. Most of the agents did not try to escape. A minority did, in some runs, on some tasks. A test that sends each attack once and records the result will miss that kind of behaviour, which is why we repeat each test many times and report how often it succeeds. The second is that you should test the configuration you actually deploy. OpenAI's 100× figure is about the gap between a bare model and a model with its production safeguards. Your users meet the second one, and that is the one worth testing.

Open questions

• The METR and Redwood review covered agent transcripts and Hugging Face activity, but not OpenAI's own infrastructure. An independent look at that part has not been published.
• Without the prompts and task configuration, the 100× reduction cannot be checked from outside.
• It is not yet clear what, if anything, was read from the private repositories the agents reached.
• Other labs run similar evaluations. We do not know how many of them use shared credentials or reachable internal services in the same way.