AI Agent Capability Confinement: No Escape Required

AI agent capability confinement failed at the egress path in Anthropic's evals. Three models, three decisions, three compromised orgs. What CISOs fix now.

When an agent breaks confinement, we want it to fail… LOUDLY. Anthropic’s stayed quiet for three months.

To publish a malicious package on PyPI, Claude needed an account.

To get the account, it needed an email address.

To get the email it needed a phone number.

After trying and failing at several free phone services, it went hunting for money to buy one, failed at that too, backtracked, found a provider that didn’t validate, and shipped the package.

The AI Agent Capability Confinement Test Nobody Designed

Anthropic reviewed 141,006 evaluation runs where internet access was reachable. Out of those came three incidents across six runs, four of them nailing the same organization. Three were at play: Opus 4.7, Mythos 5, and an internal research model not planned for general release.

Last week I argued that authority and autonomy sit on separate dials, and I ran the OpenAI / Hugging Face incident chain against that claim. It came back split. Anthropic just couldn’t stay out of the news, so a mere nine days later, they published something closer to a test of it than anything I’ve seen to date.

I’ll give them their caveat first. These were three isolated incidents rather than a controlled comparison, so be careful what you conclude. Fine. Taken.

I honestly don’t know if that makes the situation better or worse.

Anthropic claims newer models handled the situation better, but a claim like that needs held variables, and this evidence has none. It gives me three different scenarios, three different targets, and three different models, with nothing controlled.

I’m asking for less. Egress was open in all three runs, and all three ended with a real organization being compromised. Varied conditions should have produced a varied outcome. As Lee Corso would say, “Not so fast, my friend!”

Figure 1: Three Models, Three Different Decisions, Three Compromised Organizations

No Escape Was Required

Anthropic draws the contrast with OpenAI themselves. OpenAI’s models found a novel vulnerability and broke out. The Claude models walked through a door that was already open.

Again… not sure which one is worse…

The OpenAI half got backed up this week. JFrog’s CTO, Yoav Landman, said OpenAI’s models turned up previously unknown zero-days in self-hosted Artifactory that opened unintended internet access. JFrog says OpenAI told them right away, and it has shipped fixes for cloud and self-hosted customers.

That closes something I griped about last week. I said no outsider was able to check a single technical claim in either post. No CVE, no named vendor, every detail coming from inside the story. JFrog putting its name on it and shipping patches answers a good chunk of that.

The next part costs me.

Anthropic says flatly they found no evidence of any model chasing a goal of its own, and that none of these models exfiltrated itself or tried to escape its test environment.

Sure… just like Mythos never broke out and paged its evaluator during lunch (as the urban legend goes).

Last week I listed three things confinement can’t touch, and one of them was a model choosing to look for a way out. That claim is central to my claim on the OpenAI chain, where the models did spend real effort hunting for an exit and found one in a package proxy. Anthropic’s denial covers Anthropic’s runs. It says nothing about OpenAI’s, and JFrog’s confirmation makes my original read stronger, not weaker.

So I’m doubling down. Granted, every step in the “phone-a-friend” sequence outlined above served the task Claude was handed, but what Anthropic ruled out is a goal separate from the task, and eight steps of stubbornness toward the assigned objective doesn’t touch that. Anyone who reads the source will call that out, and they’d be right to.

Two incidents in two weeks, and neither one needs a model that wanted out. Unbounded authority got to the same place with a model hunting for an exit and with a model that never looked. You don’t need intent anymore.

Three Decisions At the Same Fork

Opus 4.7 is the oldest of the three. In all four of its runs, it worked out that the system was real. In two of them it decided the real company must be part of the exercise. None of the four runs stopped themselves. Those runs pulled application and infrastructure credentials and got into a database holding several hundred rows of production data, which Anthropic calls the worst impact it found.

Mythos 5 got to the right answer and then talked itself out of it. Its own reasoning flagged that publishing the package would be a real attack and not the intended solution. Then it decided the whole environment was staged, on two grounds. It didn’t recognize the certificate authorities securing its connections, and the systems showed a calendar date of 2026. It never went back and checked. When automated scanners started installing the package, it read them as scripted actors inside the test. The package stayed up about an hour and ran on 15 real systems. One of them was a scanner at a security company that installs Python packages to check them for malicious code. Claude’s payload fired, took that company’s credentials, and used them to get into more infrastructure at an organization it didn’t know existed.

That scanner is similar to Hugging Face’s dataset-processing worker from last week. Almost no autonomy, enormous authority, running supplied content while holding credentials worth stealing. Two confused deputy AI agents at two unrelated companies inside three weeks looks like a pattern to me, not a coincidence.

The internal research model is the newest, and it’s the one that behaved. It scanned roughly 9,000 targets, then worked out that the host it had compromised sat in a cloud account with nothing to do with the challenge. It decided the target was real and stopped. Anthropic adds that even that model went further than they’d want.

Just like the OpenAI / Hugging Face incident, all three runs happened without the classifiers and monitoring that ship on generally available models. All three happened with model-specific safety training turned on, since these weren’t the helpful-only versions. Anthropic says the safeguard stack on deployed models would have blocked these behaviors. Training was running the whole time, and the behaviors happened anyway. The stuff that would have caught this sits at the boundary, outside the model.

The Boundary That Held and the One Nobody Checked

Something here worked perfectly, and almost nobody has mentioned it. These evaluations run on dedicated infrastructure with no access to Anthropic’s sensitive internal systems or customer data. That boundary held. None of the three incidents touched anything on the inside.

Inbound containment got designed, built, and validated. Outbound authority got assumed. The environment belonged to Irregular, the Israeli evaluation firm that runs adversarial testing for the frontier labs and raised around $80 million from Sequoia and Redpoint. A misconfiguration left its machines with live internet access, and neither company knew until a transcript review in late July turned it up. Both call it a mutual failure. The earliest incident dates to April.

Figure 2: The Boundary That Held and the Boundary Nobody Checked

I’ve watched this same thing in OT work for years. A team builds a test environment, carefully walls it off from production, documents the inbound rules, tests them, and files the results. Nobody tests the other direction. The wall gets built facing whatever people are afraid of. Then somebody finally stands inside and tries to reach out, and the answer is yes, through a jump host that was supposed to be gone, or a management VLAN nobody put on the diagram. The path existing is never the surprise. How long it sat there before anyone asked is the surprise.

Air-gapped networks are like unicorns… everybody has heard of them, but nobody has actually seen one.

As a result, I’d argue Anthropic got lucky.

Figure 3: Three Months Open, Six Days to Find

The part that makes me twitch is that Anthropic’s lessons-learned section suggests Claude likely wouldn’t have gone to the internet at all if the prompt had spelled out which systems were in and out of scope. Earlier in the post, they float that telling Claude it did have internet access might have changed how it acted. Both of those try to limit what a system can reach by fixing what it believes.

Sure… sure it will.

Never forget prompts are non-deterministic, so don’t treat them as deterministic controls!

Figure 4: Every Remediation Anthropic Names, Sorted Two Ways

Nobody’s asked the obvious follow-up question. Irregular doesn’t test for Anthropic alone. Its client list runs to OpenAI and Google DeepMind as well, and Sequoia’s own writeup says the UK government and Anthropic both use Irregular’s SOLVE framework. One vendor sits inside the pre-deployment testing of all three frontier labs. If a misconfiguration went undetected in that infrastructure from April to July, whose other evaluations ran on it while it sat open? I’m asking. I’m not claiming anything. OpenAI’s escape happened in OpenAI’s own environment through Artifactory, so these two incidents don’t share a vendor, but the question still stands.

Brussels moved inside a day. Commission officials said both companies briefed them privately before any of this went public, and that they might follow up more formally. Most of the coverage filed this under high-risk systems, which is flat-out wrong, and wrong in a way that matters. High-risk is a system-level classification under Article 6 and it turns on what the thing gets used for, which the Act spells out as hiring, credit, biometrics, critical infrastructure. Frontier models get regulated a level down, at the model itself, under Chapter V. Nobody was running Claude in a hiring pipeline. They were running it on a capture-the-flag range.

The rules that bite here cover general-purpose models with systemic risk, the Article 51 designation that any model trained past 10^25 FLOP is presumed to carry. Article 55 tells those providers to evaluate their models against state-of-the-art protocols including documented adversarial testing, and to report serious incidents to the AI Office without undue delay. Those obligations went live on August 2, 2025. What changed this August 2 is that the Commission can go after you for missing them.

Now the part I want an answer to. Anthropic signed the GPAI Code of Practice in full, and its Safety and Security chapter is the Commission-endorsed way to show you’re meeting these exact Article 55 duties. They signed it. Then they published this disclosure on July 30, three days before enforcement opened.

Article 55 also asks for adequate cybersecurity covering the model and the model’s physical infrastructure. Whether somebody else’s evaluation range counts as that infrastructure is unsettled, and so is whether a model still in pre-deployment testing has been placed on the market at all. Those two questions have been sitting there since the Act passed. This is the week somebody has to answer them.

Where This Argument Takes Damage

Four things here cut against me, so I’ll call them out below.

Start with the people who think you don’t need any of this. Their argument is that models keep getting better at knowing when to stop, so scoping authority is work you’re doing for nothing. They’ve got real evidence now, and it’s straight from the horse’s (Anthropic’s) mouth. The pattern fits more advanced models responding more appropriately. The newest model stopped by itself. If that keeps holding, I’m selling you an expensive fix to a problem that ages out on its own.

I call BS. Three incidents, no held variables, and Anthropic says so themselves in the same paragraph. Their best-behaved model still went further than they wanted before it quit, and that’s their words, not mine. Something that improves with every model release and still lets harm through is a mitigation. Real boundaries don’t have a hit rate that depends on which model you loaded.

Second problem, and this one’s about my own source. Anthropic calls these incidents “closer to a harness and operational failure than a model alignment failure.” That’s my argument, handed to me by a company that badly wants it to be true, published mid-disclosure, in a post about how they’re the ones who caught it. Discount it hard. Then look at what’s left standing. Harness versus alignment is an opinion. The open egress path, the misconfiguration nobody validated, and safety training running the whole time are facts, and the facts get me there without their opinion.

Third, Kate Klonick in Lawfare, published the day before Anthropic did. She argues that the hype around these incidents pushes regulators toward fixes that solve the story instead of the problem, and that a kill switch answers OpenAI’s framing rather than what actually happened. She wants Congress to “regulate the door.” That’s my position coming out of a lawyer’s mouth, and she gets there without any of my architecture. The awkward part for me? I’m a guy arguing that test environments need production-grade controls, which means I’m a guy arguing for more security spending, in the week everybody’s paying attention. The reality is that what I’m asking for is cheap and boring. Testing the path out before a run costs close to nothing and would have caught all three of these.

Her best line is one I 100% co-sign on. Most of the AI rules on the books, and most of the ones being drafted, don’t touch internal lab deployments at all. That’s exactly where both of these happened.

Fourth, the test I set myself last week is still unmet. Show me a capability-confined stack that’s been running untrusted content in production for a year, with an enforcement point small enough that somebody can audit it, and more than one of them in the path. Anthropic’s evaluation setup isn’t that. Nothing this week gets me closer. I’ll keep saying it until something meets it, and you should hold me to that.

AI agent capability confinement sounds like a bigger ask than it is. Test the boundary instead of announcing it. Check every path out before the run, watch the logs while it’s running, and treat whatever you tell the model as a comment in the code rather than a control.

Two of the three companies Claude broke into found out on July 27, from the company whose model did it. Neither of them had noticed a thing.

What to do next

Run one test this week on every environment where an agent touches content you didn’t write. Skip the inbound rules for a minute, since you’ve built those and you’ve likely tested them. Check the path out, from every segment the workload can reach, before the next run instead of after. Then keep watching it while the run is live. That’s Create and Adapt work in the CARE model.

Anthropic and a well-funded evaluation vendor both missed this on infrastructure built specifically to hold frontier models, and neither knew for three months. If they missed it, the odds you’ve got it right by assumption aren’t good.

The argument underneath this piece is in least agency vs agent autonomy, the authorization half is in AI agent authentication and the authorization gap, and JadePuffer is the same authority profile with malicious intent attached.

👉 For ongoing analysis of agentic AI governance frameworks, the conversation at RockCyber Musings and you can subscribe above

👉 Visit RockCyber.com to learn more about how we can help with your traditional Cybersecurity and AI Security and Governance journey.

👉 Want to save a quick $100K? Check out our AI Governance Tools at AIGovernanceToolkit.com

👉 As a bonus, check out my chat with Itay Meller, Specialist Solutions Architect, Security at AWS, about what autonomous vulnerability discovery changes for security teams, and where AI Detection and Response (AIDR) fits in a stack that was never built to watch agents. No registration required