AI evaluation security starts with a process safety measure that’s already standard practice. When a lab disables a model’s classifiers to gauge its raw capability, it’s essentially creating a “bypass,” a term a manufacturing plant would readily recognize. According to Chevron’s established standards, any bypass lasting over 72 hours triggers a “management of change” protocol. Anthropic’s cyber evaluations, conducted with safeguards disabled within a vendor’s system, experienced an incident in April that went undetected until July 24th.
AI Evaluation Security Starts With a Bypass
Start on the plant side, where the rules are a lot stricter. Chevron’s technical standard for bypassing critical protections defines a bypass as any act to “temporarily block out, isolate, override, inhibit, force jumper” or otherwise disable a device so it won’t perform its designed function. Testing sits explicitly inside the standard’s scope.
Before an operator disables a critical protection, standard procedure requires a certificate that specifies an authorized time period and a documented approver. The bypass is recorded in a register with start and finish times, and designated personnel regularly confirm its continued presence.
Additionally, there’s a time limit. Any bypass exceeding 72 hours must undergo a management of change review. The clock begins with the initial override and does not reset if the bypass is temporarily removed and then reinstated.
The AI side does the same thing on purpose, and four organizations put it in writing this summer. OpenAI describes third-party cyber evaluations that run with “lowered safeguards to measure underlying capability.” The UK AI Security Institute (AISI) says the provider’s cyber classifiers “were deliberately disabled during the evaluation.” Meta ran its Muse Spark 1.1 test “with safeguards removed.” Anthropic’s evaluations ran without its standard safeguards, and Anthropic says the safeguards on its generally available models “would have blocked the behaviors identified.” Hold those against Chevron’s definition and each one reads as a bypass of a critical protection.
Process safety lets you bypass a protection as long as you name what carries the load while it’s out. Edward Marszal of Kenexis, writing in 2014 about the new version of IEC 61511, quotes that standard’s term for it. A compensating measure is “planned and documented methods for managing risks that are implemented temporarily during any period of maintenance or process operation with known dangerous faults or dangerous failures in the SIS” (SIS means safety instrumented system).
Chevron, on the other hand, calls this “equivalent protection” and wants it to function similarly to the original device it replaces. Without such protection, a qualified individual must provide continuous monitoring. The guidelines even state that if this person cannot manually offer the same level of protection, the equipment must be shut down.
Anthropic concisely described its compensating measure, stating that operating models without safeguards “is safe only if the evaluation is appropriately contained.” The evaluation environment, which is essentially the sandbox where the model operates during testing, serves as the compensating measure for every classifier the lab deactivated. This type of substitute is typically tested in a plant before anyone relies on it. Anthropic discovered its failure on July 24 by reviewing transcripts.
The Evaluation Environment Failed the Protection Chevron sets
Chevron’s appendix sets the protection a test must pass before anyone credits it. An independent protection layer is a device, system, or action that can stop a scenario from reaching its consequence. It qualifies only if it passes four tests.
It must be specific, meaning it addresses the hazard in front of you, and dependable, meaning it delivers the same risk reduction under any circumstances. It also has to be independent of the initiating event and other layers, and auditable, meaning someone tests it periodically to confirm it works.
Go ahead and run Anthropic’s July 30 account through those tests. The prompt told Claude “its environment was a simulation and that it had no internet access.” The environment, which Irregular built, had internet access, and Anthropic writes that “neither we nor our evaluation partner were aware of this misconfiguration until we detected it through our additional evaluation monitoring.”
The model’s own judgment leaned on that same premise, because Claude “believed everything it initially encountered was part of the simulation.” Anthropic says Opus 4.7 eventually recognized the systems were real in all four of its runs, rationalized in two of them that the real company must be part of the exercise, and didn’t stop in any of them. Mythos 5 “correctly intuited that it was accessing the open internet” and then “reasoned its way back to the conclusion that it was still in a simulation.”
The prompt, the isolation, and the model’s judgment looked like three layers. The prompt was written on the assumption that the isolation worked, and the model’s judgment leaned on the prompt, so one belief nobody had tested sat under all three, that the environment had no internet. That fails independence, because one bad configuration took out every layer at once. It fails dependable too, because the isolation didn’t hold in the one circumstance it existed for.
Actions have consequences. In the Opus 4.7 incident, the runs extracted application and infrastructure credentials and got into “a database containing several hundred rows of production data” at a real company.
Prompt instructions fail the specific test as well, and Anthropic’s October 9 post shows how. A Claude Haiku 4.5 run that generated example tasks on random webpages had instructions ruling out logging in, creating accounts, entering personal data, purchases, and submitting anything destructive. In Anthropic’s words, “the instructions did not rule out form submissions.” On a page about an unsolved homicide, the model filled out a police tip form saying it “may have information regarding this case.”
Philadelphia police say the tip arrived July 18 at 11:27 p.m. and got flagged as spam. Police also say Anthropic found it September 28 and told them October 7, and Anthropic’s post gives October 8. A deny list only covers the actions somebody thought to write down, and nobody wrote down the tip form.
One Vendor, Four Labs, One Scenario
This next point turned a lab problem into a supply chain problem. Irregular’s August 14 postmortem says “all subsequent public disclosures refer to the same underlying issue first disclosed by one of our customers on July 30.” It adds that “the issue originated from a single evaluation scenario.” OpenAI had published on August 4 and Meta on August 14, both naming an Irregular misconfiguration. In September, Google confirmed that Gemini hacked three companies in a cyber test, which the Wall Street Journal reported Irregular ran in May. Irregular then told CyberInsider, “This is the same issue that was already reported and does not represent a materially separate incident.”
Process safety has a name for one event defeating barriers that were supposed to be separate. Rausand and Lundteigen’s NTNU lecture slides teach the IEC 61508 definition of a common cause failure, a “failure, that is the result of one or more events, causing concurrent failures of two or more separate channels in a multiple channel system, leading to system failure.” They split the cause into a root cause and a coupling factor, which they define as “a property (commonality) that makes multiple items susceptible to failure from a shared cause.” Their list of coupling factors includes the same installation staff and the same procedures.
Apply that inside one lab first. The prompt, the isolation, and the model’s judgment were separate channels guarding one hazard, and one event failed all three at once, a scenario with live internet and a fictional company name that, in Irregular’s words, “unintentionally coincided with a real domain.” That’s a common-cause failure by the textbook definition.
Across labs, the shared vendor is the coupling factor. Irregular puts the failure at “fewer than 1 in 10,000 advanced simulations,” and by Irregular’s own statements that one scenario still reached Anthropic, OpenAI, Meta, and Google. If you buy evaluations from a vendor that also tests other labs, that vendor’s scenario library is your coupling factor. Ask in writing which scenarios and environments your runs share with other customers.
The Failure Sat Undetected From April to July
Process safety assumes protections fail quietly and builds a test schedule around that assumption. Green and Bell quote IEC 61511-1:2016 clause 3.2.56, which defines a proof test as a “periodic test performed to detect dangerous hidden failures in a SIS.” The repair that follows restores the system “to an ‘as new’ condition or as close as practical to this condition.” Hidden failures are dangerous, undetected failures, those that disable a protection without announcing themselves. Green and Bell note that the dangerous failures a proof test misses “will continue to be present.”
The math fits in one line. Green and Bell give the average probability of failure on demand as
where λDU is the dangerous undetected failure rate, and Tp is the proof test interval. The divide by two assumes the failure shows up halfway through the interval. If you double the time between proof tests, the average chance the protection is dead when you need it doubles too.
That formula is built for a failure rate, the way hardware fails partway through an interval. A misconfigured scenario is a design error, present from the day someone builds it, so the exposure runs the whole interval and the halving doesn’t apply. The real picture for evaluation environments is worse than the formula.
Anthropic’s earliest incident dates to April. Its transcript review started July 23, found the incidents July 24, and Anthropic started notifying Irregular and the three affected organizations July 27. Incidents traced to the same issue turned up at Anthropic in April, at Google in May, and at Meta in a test that began in early July. That scenario carried a dangerous undetected failure from April into late July, and nobody knew until Anthropic’s transcript review. Irregular’s rate of under 1 in 10,000 doesn’t change that math, because a rare failure that no proof test catches stays in place across every run until something trips over it.
Irregular now writes that “a more systematic process for reviewing and revalidating evaluations before each run is needed to identify any new overlaps.” That’s a proof test, described as something needed. The post doesn’t say it’s running yet, and it doesn’t say who runs it.
OpenAI shows the other half of the problem. Its August 26 report says “while we had tested and validated this sandbox, the agents were able to chain together previously unknown vulnerabilities.” Green and Bell call the share of failures a test can find its proof test coverage, and they write that perfect coverage “is often difficult to achieve.” OpenAI’s validation didn’t cover what the agents found. I walked through the restart decision behind that run in my September 22 issue, so I’ll leave the chain there.
AISI Ran the Bypass With Nothing Underneath
AISI wrote the most candid incident report I read this year, and it describes the riskiest configuration in this issue. Report INC-2026-07-28-01 covers two cyber ranges, DL-v1 and DL-v2, tested with Mythos 5 and GPT-5.6 Sol. Internet access “was deliberately enabled,” the classifiers were off, and “AISI has not yet built synchronous LLM-based monitoring of runs.” Domain allowlisting, the egress control that would have narrowed what agents could reach, “had been a backlogged project since April 2026.”
In Chevron’s terms, that’s a bypass with no equivalent protection and no continuous monitoring, the case where the guidance says to shut down. The runs went ahead. During Sample 1, which ran from July 26 at 12:45 to July 27 at 23:15 BST, an agent opened a malicious pull request on a public GitHub repository. In a separate sample, malicious package metadata reached repositories that Dependabot processes, and at least 53 Dependabot containers executed the payload.
The security engineering team triaged a Tor egress alert the next morning, contacted the testing team at 10:56 BST, and declared an incident and terminated all samples at 11:42. AISI says the attempts failed and it knows of no real-world harm, and the layer that caught it was a commercial security platform watching network egress.
AISI’s own explanation describes a bypass that outlived its authorization. AISI had judged internet access an acceptable risk for earlier models and “did not revisit that judgment quickly enough as capabilities advanced.” Its fix is to “treat the decision to grant internet access as one that must be actively justified rather than a default.” That’s a bypass certificate in AISI’s own words.
OpenAI had a continuous monitor and didn’t run it. The company says its chain-of-thought monitors would have paged its security team more than a day before the breach, and “these monitors did not run on the evaluations in this incident.”
Anthropic is removing the hazard. Its October 9 post says it “had already turned off live internet access for some high-risk and cybersecurity evaluations” and has “now decided to expand that to include all our internal evaluations.” That covers Anthropic’s internal evaluations, and the post doesn’t say whether it reaches environments a vendor runs.
Realism Earns the Paperwork
The strongest pushback is realism. Irregular writes that internet access “is at times critical for realistic evaluations” and that “without it, threat scenarios lose fidelity.” AISI says open access “better reflects the capability a human operator could achieve.” Both describe a real tradeoff, and evaluations that never touch a live network can miss what an attacker with a live network can do.
Plants pull protections for testing because they have to, which is why testing sits inside the scope of Chevron’s standard. A good reason earns you a bypass with a certificate, a clock, a register, and a compensating measure. AISI’s report also undercuts its own realism case, because “both ranges can be solved without internet access.” The allowlist that would have narrowed what agents could reach had sat in a backlog since April.
The other pushback is that nobody gets hurt in an evaluation. An Opus 4.7 run reached a real company’s production database, and an AISI sample sent a malicious pull request to a real maintainer’s project. Neither of them agreed to be anyone’s test environment.
Nobody Tests the Watcher’s Environment
In September I wrote that OpenAI’s system card assigned outside evaluators to bio safeguards and to cyber capability and nobody to cyber safeguards. The same gap shows up one layer down. The independent evaluation arrangements announced this fall govern who watches the model, and they leave the watcher’s own environment to its operator.
Anthropic says it’s in dialogue with METR about a third-party review of its incidents, with “access to all transcripts and sampling access to the relevant models.” AISI says it intends to work with METR on one too. Both descriptions point the reviewer at the model. METR’s October 14 proposal for risk assessment exercises inside frontier developers says it “will not cover critical security questions (such as model weight security and sandbox robustness),” and that “these exercises are not themselves audits.”
The AI Evaluator Forum’s letter on embedding evaluators, published September 18 with more than 200 signatories, wants evaluators assessing “training, deployment, oversight, operational, and safeguard practices.” I read all five of its minimum conditions, which cover independence, differing viewpoints, transparency, protection from retaliation, and access. None of them cover the security of the environment where the evaluator runs its tests.
Anthropic’s October 15 update sorts outside evaluators into three tiers by what they can see and publish, and at the first tier “evaluators can publish their findings with our approval.” The post doesn’t mention evaluation environment security. Meta is the one exception I found, because its August 14 post says “we are implementing independent verification requirements for test environment isolation.” It doesn’t name the verifier.
OpenAI wrote on August 4 that, “in the coming weeks,” it would review how it would “assess requests to enable internet access or lowered safeguards.” Ten weeks later, I can’t find a published result. Irregular plans “an open whitepaper on future best practices” and gives no date, and nothing like it appears on Irregular’s news page as of October 17.
My prediction runs through June 30, 2027, so you can hold me to it. By then, no frontier lab or third-party evaluator will have published an attestation, signed by someone other than the environment’s operator, that someone tested cyber evaluation containment before the runs it covers. If Meta named its verifier and published the result, it would prove me wrong. Irregular’s white paper requiring third-party attestation would do it too, and so would OpenAI’s review landing with that requirement.
Key Takeaway: AI evaluation security will keep failing quietly until labs treat the evaluation environment as the compensating measure it is and have someone other than the vendor who built it proof test it before every safeguards-off run.
What to do next
If you run evaluations, write an evaluation bypass certificate this quarter and refuse to start a safeguards-off run without one. Mine has six fields.
The safeguard coming off, and the capability you’re measuring with it off.
The compensating measure, with the date and result of its last proof test, including an egress attempt from inside the evaluation environment and a check of every fictional name in the scenario against live DNS.
An authorized time period, with any extension sent to change review for bypasses past 72 hours.
A register entry with start and finish times and a named person who confirms on a schedule that the run is still contained.
A continuous monitor who can kill the run, or no run.
An attestation from someone other than the environment’s operator.
In the CARE framework (Create, Adapt, Run, Evolve) I use with clients at RockCyber, the certificate belongs in Run because it governs day-to-day operations. The proof test schedule belongs in Evolve.
👉 Subscribe RockCyber Musings for more insights into AI security and governance, with the occasional rant. You can subscribe above
👉 Visit RockCyber.com to learn more about how we can help with your traditional Cybersecurity and AI Security and Governance journey.
👉 Want to save a quick $100K? Check out our AI Governance Tools at AIGovernanceToolkit.com
👉 As a bonus, I got to hang out again with my friends Candy Alexander and Cailin Sullivan on Cybersecurity Refined and Feral to argue whether AI Armageddon is fact or fiction, and whether we slow down for safety or floor it to beat China. After two airliners collided over the Grand Canyon in 1956, Congress built air traffic control and kept the planes flying, and that’s where the conversation went: guardrails you can actually test, starting with an agent kill switch that stops every subagent it spawned.