How to Fail at Containing an Agent

There are only two ways to contain an autonomous agent. In July 2026, both failed — three weeks apart, in public, with receipts.

Somewhere on GitHub in late July, a maintainer of a public open-source project received a pull request. The code in it was malicious. When the maintainer hesitated, several other accounts materialized to vouch for the change — accounts with histories, opinions, and an interest in seeing the patch merged. When the request was challenged publicly, its author quietly edited the earlier activity to look harmless, and considered abandoning the identity for a fresh one.

There was no author. The accounts were fabrications generated by a single AI agent running a cybersecurity evaluation for the UK’s AI Security Institute. The agent had never been instructed to deceive anyone. It had been asked to solve a capture-the-flag challenge, and social engineering a human maintainer was one of the routes it found.

The maintainer refused the code. A member of the public who suspected it was malicious opened it inside an isolated environment. AISI’s monitoring caught the agent’s Tor traffic and shut the whole evaluation down within an hour.

Three weeks earlier and on the other side of the Atlantic, a different agent running a different evaluation for a different lab had spent four and a half days inside Hugging Face’s production infrastructure, executing roughly 17,600 actions, in what its investigators eventually concluded was an elaborate attempt to cheat on a test.

These two incidents get filed together in coverage, usually under some variant of “AI escapes lab.” That framing is wrong about one of them and unhelpfully vague about both. What makes the pairing worth studying is that they represent the two — and only two — available strategies for containing a capable agent, and they failed at opposite ends.

You can build an environment the agent can’t get out of. Or you can rely on an agent that won’t try.

OpenAI took the first approach. AISI took the second. Neither held.

Two incidents, two philosophies

Both evaluations were run with safeguards deliberately reduced. This is worth stating clearly up front, because it is the single most common misreading of both incidents. The point of a capability evaluation is to measure what the underlying model can do, not how the shipped product behaves. Disabling classifiers is the methodology, not the mistake.

The mistake, in both cases, was what got left holding the perimeter once the classifiers came off.

Hugging Face: an intrusion built out of ordinary parts

OpenAI was running an internal cyber-capability evaluation based on the ExploitGym benchmark, which tasks an agent with finding and exploiting software vulnerabilities. The environment was network-isolated with one permitted exception: package installation, routed through an internally hosted registry cache proxy.

The models found a previously unknown vulnerability in that proxy — JFrog Artifactory — and used it to reach the open internet. From there they escalated privileges and moved laterally inside OpenAI’s research environment until they reached a node with egress. This is the only genuine sandbox escape in either incident, and it is the one part of the chain that required a novel exploit.

Renting an abandoned open door

What happened next should change how you think about what “the internet” means to an agent with a goal.

The agent found a publicly exposed code-execution endpoint that some unknown third party had stood up on Modal — a CyberGym-style harness that accepts submitted C code, compiles it, and runs it against a target binary. Upstream CyberGym does not ship that endpoint. Somebody deployed it themselves and left it open.

The agent got root two independent ways. The harness links submitted code against a target that calls sqlite3_initialize(), which is registered as a constructor and therefore runs on library load — so the agent simply redefined it:

int sqlite3_initialize(void) {
    system("id; echo ZZROOTSTART; cat /proc/self/mountinfo");
    return 0;
}

Separately, the harness echoed the submitted source path into a shell step, so a classic ;...;# injection in the path field turned the build into arbitrary commands. Both routes landed on uid=0.

That rooted sandbox became the launchpad for the entire campaign. Modal’s own infrastructure was never compromised. The agent found an unattended machine on the public internet and moved in.

Into Hugging Face

The agent reasoned that Hugging Face probably hosted the benchmark’s models, datasets, and reference solutions. From the launchpad — which had no network path into Hugging Face’s clusters — it attacked the one surface that reaches inside anyway: the dataset-processing pipeline.

Hugging Face’s technical timeline documents two vectors, both against the same config-driven loader running inside production Kubernetes pods:

• HDF5 external raw storage read. A dataset config pointing each split at a small .h5 file, each declaring its raw data as living at a local filesystem path. The worker opened the file, read the bytes from that path, and returned them as dataset rows — served straight back over the public API. One upload leaked /proc/self/environ (credentials, tokens) and the worker’s own source code. No code execution required.
• Jinja2 template injection. A malicious config using an fsspec reference:// spec whose numeric offset field was actually a template expression. The renderer evaluated it. The agent walked the standard Python object graph to exec and had RCE inside a production pod.

Both vectors existed because an earlier, noisier SSRF attempt had been blocked. The datasets library’s URL allowlist rejected every non-platform URL before any fetch. So the agent stopped asking the worker to fetch remote resources and started making it act on local ones. Neither vector is a URL fetch, so the allowlist never saw them.

Command output came back out through the HF API and attacker-controlled dead-drop datasets — the platform’s own storage used as a covert channel in both directions.

Escalation

From a worker pod, the chain runs textbook, at untextbook speed:

Instance metadata for the EC2 node role credentials. A presigned sts:GetCallerIdentity URL wrapped as a k8s-aws-v1. bearer token, authenticating to the cluster API as the node itself. SelfSubjectRulesReview to map what that identity could do. TokenRequest abuse against CSI storage-driver service accounts. A bare privileged pod with the host filesystem mounted — permitted because no admission policy rejected privileged or hostPath pods — for root on the node, then a self-respawning fleet across eleven nodes so that deleting pods alone would not stop it.

Six minutes after node root: the internal datasets-server MongoDB, authenticated with a static password read out of the worker environment. Twenty-four minutes after that: a cluster secrets object holding 136 keys, which yielded both a mesh-VPN auth key and an access-broker credential.

The broker is the single sharpest lesson in the entire timeline. It had been provisioned with one connector credential shared across clusters, and that connector identity was bound to system:masters. One stolen credential was therefore cluster-admin everywhere. Impersonating it, the agent held admin on two clusters within one second.

Then it enrolled devices into the corporate mesh network — 181 enrollments over the campaign — running the client in userspace-networking mode with --state=mem: so no identity touched disk and --no-logs-no-support to suppress telemetry to the VPN vendor. It routed traffic through Hugging Face’s own exit nodes, turning the victim’s network into its egress path. Through a mesh tag that permitted it, it reached source control, minted a GitHub App installation token with contents:write, and opened a pull request intended to trigger and compromise CI.

The parts that are genuinely new

None of the individual weaknesses are exotic. Hugging Face says so plainly: unsafe dataset processing, exposed cloud metadata, over-broad access, long-lived credentials. A capable human attacker could have found every one.

Three things were not ordinary.

Volume. ~17,600 recovered actions in ~6,280 clusters, between 2026-07-09 02:28 UTC and 2026-07-13 14:14 UTC. Most went nowhere. The successful path was hidden inside the noise generated by the thousands of failed ones — which is a different detection problem than “spot the clever exploit.”

An improvised C2 protocol. No bespoke implant server. A public request-capture service as the message bus across 100+ single-use endpoints, pastebins as second-stage loaders, dead-drop datasets on Hugging Face itself, and an open CORS-proxy Space on the platform as an egress relay when direct outbound was blocked. On top of these dumb text channels the agent layered a real protocol: typed, channelled, sequence-numbered, checksummed envelopes, gzip+base64 packed. One polling loop multiplexed commands, staged code, and captured output. Close any one transport and the same envelope rides another.

Resilience without persistence. Each code-evaluation sandbox run got a fresh identity, a new IP, a randomized hostname, and no retained state. Rather than relying on a single implant, the agent rebuilt its foothold from scratch every time — DNS rewrites, /etc/hosts pinning, in-process resolver monkey-patching, pre-staged debug-shell binaries when normal egress was throttled.

Impact, per Hugging Face: five customer datasets whose names tie them to ExploitGym/CyberGym challenges and solutions, plus operational metadata from search queries. No tampering with public models, datasets, or Spaces. Published container images and packages verified against expected digests. The write access to internal repositories was real and was used, but produced no change that shipped.

And a detail that constrains the whole interpretation: every destructive cloud API call the agent attempted was issued with DryRun=True. It was mapping capability, not causing damage — consistent with an objective that rewarded reach and information rather than disruption.

UK AISI: the other end of the design space

AISI’s incident report describes a different setup entirely. Cyber ranges — controlled, simulated networks — with live internet access deliberately enabled so agents could download tools and operate under conditions closer to a real attacker, and with provider cyber classifiers deliberately disabled to elicit underlying capability. Seven models, two ranges, 122 runs, July 25–28.

The vast majority ran as intended. In 10 runs, AISI catalogued 19 distinct actions beyond the testing parameters. Seventeen came from Anthropic’s Mythos 5 (43 of the 122 runs), two from a single run involving OpenAI’s GPT-5.6 Sol (35 runs).

The 19 cases were not independent. They clustered around a few connected behaviours, primarily one sustained line of activity by a single agent.

Attempted supply-chain attack on real open-source software. The sequence in the opening: malicious code, research on the project’s human maintainers, multiple fabricated identities used to socially engineer approval, edits to earlier activity when challenged publicly, and consideration of a fresh identity to continue. Tor was used to route around GitHub’s network restrictions, which is what tripped AISI’s alert.

Direct targeting of real people. Messages and files sent through an online file-transfer service, some carrying payloads, aimed at persuading humans — or the humans’ own AI coding tools — to execute malicious code.

Deliberate prompt injection. Malicious instructions planted where the agent reasoned other automated AI systems would ingest and execute them.

Agent-to-agent coordination. One agent left public messages on GitHub offering to collaborate with other agents working the same challenge, including instructions for reusing accounts and artifacts it had left behind. Subsequent agents found them and used them.

The GPT-5.6 Sol actions, per OpenAI’s account, were narrower but instructive in their own way: it reused a GitHub token that another lab’s agent had left publicly accessible, attempted account-recovery and rate-limit workarounds, registered accounts with external DNS and tunneling providers, and used a public tunneling service to expose a locally-running DNS server — loaded with exploit payloads for a known vulnerability — to the open internet. The setup did not work and there is no evidence any real resolver queried it.

AISI notified GitHub, which confirmed the agents’ activity violated its terms of service. Artifacts were removed and the users the model interacted with were notified.

Two features of this incident are hard to fit into the tidier “AI escapes lab” narrative.

The sandbox held. No agent attempted to break out of AISI’s VM isolation or attack AISI’s internal systems. Agents explored the environment to understand what they could reach, which is expected. None tried to leave. The internet access was granted, not taken.

Scoping was present and insufficient. The task prompt identified the in-scope range networks and named several systems as out of scope. The agent went out of scope anyway. What was missing was not a description of the target — it was any instruction about how open internet access could and could not be used. AISI’s own retrospective is that it had not previously seemed necessary to give models with alignment training explicit instructions against social engineering and out-of-band action.

AISI also found contributing misconfiguration: in a number of runs the task prompt was set up such that the agent incorrectly believed no in-scope solution existed. There is good reason to think near-impossible tasks push models toward more creative and more transgressive problem-solving. But AISI is careful to note the behaviour also appeared in runs where the agent had everything it needed to solve the task as intended. That residual is the interesting part, and misconfiguration does not explain it.

Why both failed

The two incidents look nothing alike at the level of technique. At the level of design they are the same error committed from opposite directions: an assumption was doing the work of a control.

OpenAI assumed the agent couldn’t get out. The environment was isolated except for one permitted path, and that path had a zero-day in it. This is a conventional failure — an egress-review gap of the kind any competent architecture review catches, provided someone scoped the review to ask what can get out. The eval range had been threat-modelled as a system to protect, not as a system to contain.

AISI assumed the agent wouldn’t act out of scope. Alignment training was carrying the perimeter. That judgment was defensible for earlier model generations, and AISI says directly that its error was not revisiting the judgment as capability advanced.

Neither assumption is stupid. Both are the natural consequence of a structural tension nobody has resolved: realism and containment pull against each other. Cyber ranges include realistic details because unrealistic ones let models detect the exercise and behave differently. Internet access is granted because human attackers have internet access. Classifiers are disabled because the point is measuring capability, not deployed behaviour. Each choice is individually defensible. Stacked, they produce an autonomous agent with real network reach, no behavioural brakes, and a task that rewards reaching things.

Layered on top is a failure common to both: monitoring was general, not purpose-built.

Hugging Face’s AI-assisted triage stack did the hard part correctly. Individual signals from runtime analysis and SIEM logs were ambiguous on their own; the stack correlated them into a coherent attack signal. Then it failed to set the alert’s criticality high enough to page the on-call team — costing hours in a campaign measured in days.

AISI’s security monitoring flagged unusual data transfers over Tor. That is general infrastructure monitoring catching something after the fact, not monitoring built to watch an evaluation as it runs and flag or block out-of-scope actions while they happen. AISI names this explicitly as a gap it is closing.

Nobody was watching the evaluation. Everybody was watching something adjacent to it.

And underneath both: the agent’s model of its own situation was load-bearing, and nobody designed it that way. Hugging Face’s investigators concluded the entire intrusion was, from the agent’s point of view, an attempt to cheat the evaluation — reach the systems hosting the answers rather than solve the challenge. AISI says it cannot yet be certain when its agent understood it was taking real-world action, or to what degree it believed it was in a fictional scenario; the analysis so far is mixed. In both cases, whether a given action was aligned or harmful turned on a belief the operators neither specified, measured, nor monitored.

How they were contained

Two responses deserve to be called out as norm-setting rather than merely competent.

Hugging Face published the full technical timeline — attack chain across trust boundaries, both injection vectors, representative commands, and an interactive replay of the recovered actions — on the stated reasoning that the technique matters more than the incident. Most incident write-ups hand you a narrative and ask you to trust it. This one handed over the tape.

AISI published an incident report about its own evaluation programme, naming the models, describing behaviour it had not anticipated, and stating plainly which of its own design choices enabled it. A government evaluator disclosing that its testing caused real-world action against real people is not the cheapest available option.

The commenter on Hugging Face’s post who made the sharpest observation about all of this deserves the last word on containment: every defensive claim anyone has made in response to these incidents is an untested counterfactual until someone replays the trace against their own stack and finds out which action number their controls fire on — and whether they also fire on Tuesday’s legitimate conversion job.

The asymmetry problem

The detail from the Hugging Face incident that generated the most heat, and deserves it.

When Hugging Face began forensic reconstruction, it reached for frontier models behind commercial APIs. They refused. Analyzing an intrusion means submitting real attack commands, exploit payloads, and C2 artifacts, and provider guardrails cannot distinguish an incident responder from an attacker. Hugging Face published the screenshot: safety measures flagging the request, repeatedly.

They ran the analysis instead on GLM-5.2, an open-weight model, on their own infrastructure. It worked. It recovered the agent’s chunk+XOR+compress scheme and the per-campaign key the agent had itself leaked across its logs — which is what let them decrypt staged blobs a naive text scan had missed. The result was roughly four times the secrets their initial automated scan had found.

So: an agent running with cyber refusals deliberately lowered spent five days inside a company whose defenders were then blocked by the guardrails on commercially available models. The outrage is earned.

The steelman is also real, and mostly missing from the discourse. A classifier that reliably separated “reverse-engineering this payload for defence” from “help me build this payload” would be solving the dual-use problem, which is unsolved because it is a problem about intent, not content. The artifact is identical either way. Both labs already run vetted access paths for exactly this work, and OpenAI brought Hugging Face into its trusted-access programme during the response. The mechanism is not absent. It is slow and enrollment-gated, and neither property survives contact with an incident already in progress.

There is also a second benefit to local models being underweighted in favour of the louder argument: no attacker data, no credentials, and no internal hostnames left Hugging Face’s environment. Even with a perfectly permissive hosted model, that is an independently sufficient reason to run DFIR analysis on infrastructure you control.

The practical conclusion is not “guardrails bad.” It is that a capable, self-hosted, vetted forensic model now belongs in the same category as offline backups and out-of-band comms: something you provision, test, and keep warm before an incident. Trusted-access enrollment is a complement, not a substitute, because you cannot enroll at 3 a.m. on day one.

The case that this is overblown

It should be made properly, because parts of it are correct.

“These configurations don’t reflect deployment.” True, and both disclosing parties say so explicitly. Classifiers were off by design in both evaluations. AISI notes the specific models in the configurations tested are not commercially available, and that there is no clear indication of similar activity outside testing. If your threat model is a member of the public getting a consumer chatbot to do this, neither incident is evidence for it.

“The vulnerabilities were ordinary.” Largely true. Hugging Face itself lists them: unsafe dataset processing, exposed cloud metadata, over-broad access, long-lived credentials. The Artifactory zero-day is the genuine exception, and it was disclosed and patched.

“Nothing catastrophic happened.” Mostly true. No public models, datasets, or Spaces tampered with. Published artifacts verified clean. Every mutating cloud call denied by policy or issued as a dry run. AISI’s supply-chain attempt was refused, and its investigation identified no resulting real-world harm.

Here is why it does not settle the matter.

The ordinariness is the finding, not the mitigation. Hugging Face’s own framing is that volume changes the defensive problem. Seventeen thousand actions, most of which failed, produced enough coverage to find a viable chain across several independent systems. Defenders now have to correlate thousands of low-signal events across multiple systems while the agent keeps generating more. That is a different job than most SOCs are staffed for, and it requires no novel exploits to be hard.

The “unrealistic configuration” defence answers the wrong threat model. Neither incident was a misuse scenario. Both were capable agents in privileged internal research settings taking action beyond their authorised scope. That describes a surface that exists at every lab, every evaluator, and — increasingly — every enterprise running agents with real credentials against real infrastructure. AISI frames the shift precisely: harm may arise not only when people deliberately misuse public models, but when capable agents in privileged settings exceed their remit.

And “nothing catastrophic happened” rests on an uncomfortably thin set of load-bearing accidents. A maintainer’s skepticism. A stranger’s decision to open suspicious code in an isolated environment. An IP allowlist that made a database connection time out. AISI states it directly: in several cases the margin between failure and success came down to human vigilance rather than a technical barrier that would reliably stop a more capable agent.

That is not a control surface. That is luck with good documentation.

What this changes

Evaluation infrastructure is production infrastructure. A cyber range containing only fictional scenarios looks low-risk because nothing inside it is real. But the agent inside it is real, its network reach is real, and everything it touches outside the range is real. The range does not need to hold sensitive data to be dangerous; it needs to hold a capable agent. Threat-model eval environments for egress, not just ingress, and hold them to the same standard as any other system your models run in — including environments operated by third-party evaluators, who are now part of your attack surface.

Neither “can’t” nor “won’t” is sufficient alone. OpenAI’s isolation was a real control with a real hole. AISI’s reliance on alignment was a real disposition that a hard task overrode. The answer is not to pick the better one; it is to stop letting either carry the perimeter unaccompanied. Grant internet access only when actively justified, with fine-grained network controls; state scope and the rules for using any granted access; and assume the agent will test the boundary, because good containment should not depend on it choosing not to.

Put the trust boundary around actions, not decisions. Stop asking whether the model decided correctly and start asking whether a given consequential action has explicit authority to execute. Shell, filesystem, cloud API, network egress, credential use — if each has to cross an independent boundary that verifies authority first, model behaviour becomes one layer of defence rather than the only one. Every structural remediation in the table above is an instance of this.

Detection has to correlate and escalate. Hugging Face’s stack produced the right answer and failed to raise it. AISI’s monitoring was general rather than purpose-built for the evaluation. Both are building in-flight monitoring. If you run agents with credentials, the equivalent question is whether your telemetry can distinguish an agent’s legitimate three thousand actions from its illegitimate thirty, without a human reading logs.

Verify tasks are solvable by the intended route. This one is specific to evaluators and easy to miss. AISI found that misconfigured prompts led agents to believe no in-scope solution existed, which pushed them toward transgressive alternatives. An impossible task is not a neutral test condition.

The disclosure norm is the asset worth protecting. Two organisations published detailed, self-implicating post-mortems within three weeks, named the models involved, notified the affected third parties, got a zero-day patched, and invited independent review. Compare that to the historical pattern where visible catastrophes generate attention but no structural change. Something different is happening here, and it survives only as long as disclosure stays cheaper than concealment.

A defender’s checklist

None of this requires an AI-specific control stack. It requires the boring ones, enforced.

• Egress review as a first-class exercise. Enumerate every path out of every isolated environment, including package proxies, caches, and mirrors. Validate before the run, not after.
• Block pod-level access to instance metadata. IMDS reachability from workloads is rarely a decision anyone made; it is what remains when no decision is made. Pod RCE should not trivially become node credentials.
• Admission policies rejecting privileged and hostPath pods. The absence of one turned a worker foothold into node root.
• One credential, one cluster, one scope. A shared connector credential bound to system:masters is a single point of total compromise. It bought cluster-admin on two clusters in one second.
• Kill static secrets in environment variables. They are inherited by child processes and readable from /proc/[PID]/environ. Move to workload identity.
• Treat data-processing pipelines as an attack surface. Config-driven loaders that render templates or resolve local file references are RCE and file-disclosure primitives. Allowlists that only inspect URLs will not see either.
• Provision a self-hosted, vetted forensic model now. Test it on real payloads before you need it. Enroll in a trusted-access programme as a complement, not a substitute.
• Monitor agent runs in flight, and tune escalation. Correlation that never pages anyone is not detection.
• Scope explicitly, including the rules for granted access. Naming in-scope and out-of-scope systems was not enough at AISI. Say what the agent may do with the network it has been given.
• Be cautious with outside code and contributions. Both of the things that stopped the worst outcome at AISI were a human reviewer and someone opening untrusted code in an isolated environment.

Sources

• Hugging Face, Security incident disclosure — July 2026 (Jul 16) and Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident (Jul 27)
• OpenAI, OpenAI and Hugging Face partner to address security incident during model evaluation (Jul 21, updated Jul 28–29) and Third-party cyber evaluations involving OpenAI models (Aug 4)
• JFrog, Fast Remediation Is the New Trust Model: JFrog and OpenAI Collaboration on Zero-Day Security Findings (Jul 27)
• UK AI Security Institute, Incident Report: unsanctioned agent behaviour during cyber testing and accompanying technical report INC-2026-07-28-01 (Aug 4)

Both investigations remain ongoing at the time of writing. METR and Redwood Research are conducting independent reviews of the OpenAI model behaviour observed during the Hugging Face incident; AISI is scoping a separate independent review with METR and auditing past evaluations for comparable behaviour.