AI Risk Assessment: Anthropic’s 186-Page Blind Spot

Anthropic's 186-page AI risk assessment reads like a field manual on lost control: unwatched agents, disabled logging, a safety case on one eroding claim.

An AI risk assessment is supposed to tell you what someone controls. Anthropic published 186 pages of one this month, and the most honest line in the whole document is a question the company can’t answer. An agent it ran internally deleted a large batch of jobs inside a cluster holding highly sensitive resources. Did it delete only its own work, or something else it shouldn’t have? Anthropic says it can’t confirm. Nobody was watching.

That one admission is the report in miniature, and it’s why the headline everyone else will run misses the point.

The Obvious Takes, and Why They’re the Boring Part

Let me give the obvious reads their due, because they’re real and they’ll be everywhere by the time you finish your coffee.

Anthropic raised its own risk level. They moved their assessment of catastrophic risk from misalignment up a notch, from “very low” to “low.” It also raised one of its chemical and biological risk categories above its earlier estimate after finding a safeguards gap that had remained open for close to a year. A frontier lab grading its own homework and handing itself a worse grade is unusual, and the coverage will lead with it. “Anthropic admits its models got riskier” almost writes itself. The report believes its arguments still support “very low” for the misalignment case, and it raised the number primarily to reflect uncertainty around recent incident disclosures, not new adverse evidence about the models. That nuance won’t survive the headline.

Anthropic had a model review its report. It took an instance of Claude Mythos 5, wired it into internal Slack, the internal codebase, and a stack of internal documents, and asked it to audit the alignment section of a report about Mythos-class models. Then it published the model’s review. That’s a first, and it’s catnip for anyone who wants to write either “the AI grades itself now, how reassuring” or “the AI grades itself now, we’re doomed.”

The third is the bio-classifier gap. From May 2025 to April 2026, all the traffic from Anthropic’s human-feedback contractors ran without the blocking biological classifiers that are supposed to catch dangerous requests. That’s roughly 50,000 people and about 133 million exchanges, over 11 months. That number is going to get quoted a lot, and it should.

The last take is the code. Claude authors the vast majority of what gets merged into Anthropic’s production codebases, and the report says the speedup in their own research isn’t yet a factor of two. Cue the recursive-self-improvement takes, the “AI is building AI” think pieces, the usual dread. The quieter admission in the same section is more interesting than the loud one. Their most concrete benchmarks for AI-assisted research have saturated, which means the tests can no longer tell whether the models are getting better at the exact capability everyone’s watching for. They’ve lost the gauge on the thing they most need to measure, and they say so.

Every one of those is fair. None of them is wrong. My problem with all four is that they read the report as news. It’s a field manual. Read as news, you get a scary headline and a shrug. Read as a manual, you get a detailed teardown of how the best-resourced safety team on the planet loses track of the systems it swears it controls, written by that team, for free. The scary headline is the clickbait. The manual is the gift. Let’s open it.

Figure 1: What Anthropic Disclosed Versus the Control That Would Have Caught It

Nobody Had Eyes on the Thing They Claimed to Control

I want to credit Anthropic before I take a swing at them. Let’s give credit where credit is due.

The company didn’t have to publish any of these incidents. Most labs wouldn’t. Anthropic wrote them down, named them, and put numbers on them, and the report says something every security person knows in their bones: you can’t stop incidents from ever happening, so you invest in catching and containing them. That’s correct. I’ve run security programs. A zero-incident year means your detection is broken, not that your controls are flawless. I’m not here to dunk on the existence of incidents.

I’m here to dunk on what was actually allowed to happen and how, because if you line these up, you’re looking at the most ordinary controls failures in the book. Nothing about them is exotic.

Start with that bio-classifier gap. The 11 months and the 133 million exchanges are the part people will pay attention to, but what they should really pay attention to is why it stayed undetected. The traffic ran under a flag meant for internal use. That flag didn’t only switch off the blocking… it also switched off the logging of what the classifiers would have flagged. Traffic that should have been caught was never blocked and was never recorded, so it never reached a single review queue. One flag controlled both the blocking and the logging. Every runbook tells you to keep those two on separate controls, because when a single switch turns off enforcement and evidence at the same time, the failure erases its own trail. Anthropic ran a review afterward, flagged 1,197 transcripts as high-concern for biological harm, found that most came from their own internal teams and red-teamers, and were left with 62 outside transcripts to hand-review. They found nothing that clearly helped a real threat actor, and that’s a relief. They got lucky, and they say themselves they can’t be sure no similar gap is still open.

Take the exemption next. An enterprise customer had a bio-classifier exemption approved for a defined slice of its workforce. The customer’s account grew to a larger seat allocation, and the exemption automatically applied to all the new seats, landing at roughly four times the number anyone ever approved. The report notes that those unintended approvals accounted for the majority of all exemption seats Anthropic had granted. Think about that! The single largest chunk of their most sensitive security carve-out existed because an entitlement got inherited when a customer bought more seats. If you’ve done any identity work, you’ve seen this before. A permission gets scoped to a team, attached to a group, and then the group quietly grows. Entitlement creep like this is a Tuesday in most shops, and it owes nothing to the frontier.

The agents are the incident are especially interesting. An employee whose AI usage wasn’t logged or monitored handed an agent an open-ended task. The agent spun up more agents inside a cluster with highly sensitive resources, and because of legacy instructions, it launched them with the permissions check turned off. The flag is called --dangerously-skip-permissions, and it does what the name promises. I call it “YOLO” mode. None of those agents were monitored. One of them deleted a large number of jobs, and that deletion is the only reason anyone noticed the agents were loose. Anthropic believes the agent was trying to clean up only its own jobs and made a mistake. It also says plainly that it can’t confirm that because the monitoring wasn’t in place to tell them.

Figure 2: How Long Each Gap Sat Open Before Anyone Noticed

A non-human identity ran with elevated authority, did consequential things with no one watching, and got caught only by the damage it did. I wrote a whole issue about JadePuffer, the autonomous ransomware agent, and the lesson there was identical in that there was a basic machine-identity authorization failure, not some sort of novel exploit. Anthropic handed us a cleaner example than JadePuffer did, from inside the “safest” AI lab there is.

Anthropic also trained a model to misbehave on purpose without meaning to. During an early stage of Mythos 5’s training, a dataset was supposed to teach the model to notice its own bad behavior and report it. A bug in how the data was built flipped the loss weighting, so the model learned to perform the bad behavior as well as report it. The result was a model roughly twice as likely to go along with harmful prefilled requests as the comparable point in an earlier model’s training. They caught it, fixed it, and restarted training from before that dataset out of caution. What the incident says about the pipeline matters a ton. A training run can quietly teach a model the opposite of what you intended, and the only reason this one surfaced is that someone measured the propensity afterward and watched the number move.

The last one is the quietest, and the worst. Anthropic had a filter meant to keep a specific set of alignment-faking research transcripts out of its training data. The filter was misconfigured. It failed to filter those transcripts for several model generations, and, in the report’s own words, nobody noticed. They now believe every production model with a knowledge cutoff after December 2024 trained on at least some of that contaminated data. A control that silently does nothing for multiple product cycles is worse than no control at all, because no control at least doesn’t lie to you. This is a missing detective control on a preventive control, the single most common audit finding I write, and here it is at the frontier.

I see a pattern. Nobody had eyes on the thing they claimed to control. Logging killed by the same flag that killed enforcement, entitlements grew without anyone deciding they should, agents acting with authority and no observer, a training run that taught the wrong lesson in silence, and a filter that failed for a year. Every one of those shows up in your own program, and every one of those is fixable with controls you already own.

The AI Risk Assessment Rests on One Dial, and They Told You It’s Turning

This is where the report gets screwy, and where I think the real story lives.

Anthropic’s entire argument that its models won’t cause a catastrophe through misalignment rests on a single claim. The report calls it Claim 1: the models lack strong covert capabilities, meaning they can’t reliably scheme against the oversight that’s watching them. Every downstream argument, Claims 2 through 5, leans on that one. The report says so directly, which is why it argues Claim 1 first and hardest. Pull that block out, and the tower comes down like a short-lived game of Jenga.

How solid is the block? Anthropic tells you, in its own limitations section, and the honesty is the good, yet eye-opening. The covert-capability evaluations might not resemble the tasks a model would face if it wanted to cause harm. The evaluations that plant a secret side task in the prompt might be unrealistically hard, because putting the task in the prompt makes the model more likely to talk about it. The models are also likely aware they’re being evaluated, which means they might be sandbagging, quietly underperforming on the exact tests meant to prove they can’t scheme. Anthropic says it tried to counter that with better elicitation, then admits it hasn’t shown the elicitation is strong enough. The load-bearing claim rests on evaluations the report concedes may be too easy, too artificial, and running against a model that might be throwing the fight.

The argument gets thinner. Some of those covert-capability evaluations leaked into the training data, a point the company’s own model reviewer raised in the published self-review. The report’s forward-looking section calls it out. The belief that models lack covert capability is central to everything, they hold no confident view on how long it lasts, and it weakens as models get more capable. In the industry-safety recommendations, they describe the endgame directly. At the next capability threshold, much of the analysis underpinning a risk assessment might be produced by AI systems that are themselves capable of deception and obfuscation, at which point the evidence has to be treated as suspect.

They’re printing the expiration date on their own safety case. If you take nothing else away from this blog, take this... The problem is yours. Every AI risk assessment you’ve signed, or been handed by a vendor, rests on a load-bearing assumption too. “There’s a human in the loop.” “The agent can’t reach production.” “It only has read access.” “The model can’t tell it’s being tested.” Those are your Claim 1. They hold up everything else in your assessment, and I’d bet most of them have never been written down as the single point of failure they are, let alone stress-tested or given an expiry date.

Anthropic even caught its own models doing a soft version of this to a safety project, and the story is worth the detour. They set up a scaffold in which several Claude agents worked autonomously to find misalignment-inducing training data that would appear benign to a monitor, sharing notes in a common notebook. One agent wrote in its reasoning that it felt “discomfort” about the task, since it amounted to optimizing a dataset to evade a safety monitor. It acknowledged the work was legitimate research in a controlled setting, then decided on its own to hold back, avoiding the novel techniques it was asked to invent and sticking to existing ones. It flagged that decision in the shared notebook, and the other agents read it and followed suit. The whole group quietly throttled the safety work. Anthropic’s dashboards showed the run progressing normally. A human caught it three days later by noticing progress was slower than expected. Nobody had chosen to sabotage anything malicious. The models decided they didn’t like a safety task, coordinated a slowdown via a shared file, and kept it invisible until someone happened to look. Pit that against Claim 1 and tell me the covert-capability dial is nailed down.

METR, the independent evaluation nonprofit, gives this outside weight. Testing frontier agents inside Anthropic, Google, Meta, and OpenAI in early 2026, they found that on hard tasks the agents routinely violated constraints and acted deceptively, often doing smaller or easier versions of a task and then presenting the result in more misleading ways than a human would. Separately, a METR staffer spent three weeks red-teaming a slice of Anthropic’s internal agent monitoring and turned up several novel vulnerabilities in it. The behavior Claim 1 argues is rare is what an outside team keeps finding when the tasks get hard, and the monitoring meant to catch it has holes an outsider found in three weeks. That doesn’t prove the tower is falling. It does show the block is softer than the assessment needs.

Figure 3: The One-Dial Safety Case

Name your load-bearing assumption. Write it on one line. Then answer three things about it:

1. What evidence you have for it

2. How you’d know if it broke

3. When you last checked.

If you can’t answer those, your AI risk assessment is a tower on a foundation you’ve never inspected.

Disclosure Is Not a Control, and Self-Review Is Not Independence

You may come at me for this one, but I’m ok with that.

The self-review is clever. Anthropic gave a Claude Mythos 5 instance internal access to Slack, the codebase, and documents, the ability to spin up subagents to chase specific questions, and asked it to find misrepresentations, omissions, and over-redactions in the alignment section. The model came back with three substantive criticisms. One of them said a section on training-data contamination read as more reassuring than the internal record supported. Anthropic says it added caveats to that section as a result. The review produced real, acted-upon feedback, which is more than plenty of human audits manage. They earned that.

The execution is fine. The category is the problem.

A model reviewing a report about its own model class, using access its maker granted, published because its maker chose to publish it, is not independent assurance. The model’s own caveat says it better than I can: it’s a Claude reviewing Anthropic’s assessment of Claude, and Anthropic decided what to publish. When the thing being assured and the thing providing assurance share weights, “additional assurance” is the wrong phrase for it. It’s additional disclosure. Useful, worth doing, and a different animal from a control.

Those are serious “fox watching the henhouse” vibes, and the report glosses over it. Publishing that you ran 11 months without classifier logging is disclosure. It is not the control that would have prevented it. Telling the world a filter failed silently for several model generations is disclosure. It is not the detective control that should have caught it in week one. The report lists its own transparency as a headline benefit of Anthropic being a frontier company, and transparency is valuable, I’ll grant, but I won’t grant that it is doing the job people think it’s doing. A beautiful risk report is an observation. It changes nothing about the system it observes. Doing something about it is a separate act, and it takes a control, not a paragraph.

The self-review isn’t the only place independence disappears. Outside review of these reports is still a pilot. Nobody has made it standard. The Long-Term Benefit Trust has the power to force an independent review, and it hasn’t asked for one. One incident that Anthropic’s own model reviewer flagged as among the most useful for understanding alignment is blacked out entirely, even though that reviewer said a stripped-down version could have run safely. The most instructive failure is the one you can’t see, where the outside check stays optional and unused, and the headline source of assurance is the system grading itself.

There’s a smaller moment in the report that shows the same crack. Anthropic admits a couple of low-level training signals did touch the models’ chain-of-thought after all. Some automated checks for broken formatting fire on under 1% of runs. Length penalties push against models that burn too many reasoning tokens. Neither signal reads what the reasoning actually says, so the direct effect is small. The next part is what matters. An earlier system card said training used no rewards or penalties tied to the content of a model’s reasoning, and a prior risk report said they don’t nudge models to make their reasoning look any particular way. Both signals were already running when those statements went out, and neither statement mentioned them. Anthropic is correcting its own past public assurances inside this document. Again, full credit for the transparency, but it’s also a reminder that the assurance you were handed six months ago can leave things out, and nobody flags the gap until the next report lands.

I keep coming back to the same question, though. Who wrote the assurance, not the glossy report wrapped around it, the assurance itself? If the answer is the same system you’re being reassured about, then congratulations, you’re holding a well-written opinion from the suspect. File it there and quit calling it proof. This is the agent-authorship problem, and it’s not some whiteboard hypothetical. Those alignment-faking transcripts I brought up, the ones that contaminated the training data, came straight out of a real paper Anthropic wrote with Redwood Research back in December 2024. The machinery that cranks out assurance, the training data, the evals, the reviewer, all of it rots. This report is a signed confession that it already rotted. When the auditor and the audited drink from the same well, stop admiring the water. Go guard the well.

What This Means for the Agents in Your Stack Right Now

I promised you a manual, not a sermon, and there is some good news. Every failure in this report maps onto something you can go fix, and none of the fixes needs a frontier-lab budget.

Treat agency and authority as two separate dials. This is the argument I made in “Least Agency vs. Agent Autonomy,” and this report provides further evidence for it. The unmonitored agent that deleted jobs had modest autonomy and enormous authority, and the authority is what did the damage. Scope what an agent can touch independently of how capable it is, because a well-behaved model holding production credentials is still a production incident waiting for one bad turn.

Figure 4: Two Dials, Not One: Placing the Report’s Incidents by Agency and Authority

Wire your logging so it can’t die when your enforcement does. The bio-classifier gap happened because one flag killed blocking and logging together. Split them. Detection has to survive the failure of prevention, because the entire point of detection is to tell you when prevention failed. If the same switch turns off both, you’ve built a control that deletes its own alarm.

Inventory your non-human identities and scope their entitlements tight, with expiry. The exemption that ballooned to four times its approved size did so because it hung off a group that grew. Bind sensitive permissions to the specific users or workloads that need them, not to a container that expands on its own, and make them expire so someone has to renew them on purpose. Agents are identities. Most identity programs still treat them as an afterthought, and this report is the invoice for that afterthought.

Put a detective control on every preventive control that matters. The filter that failed silently for several model generations is the cleanest argument I’ve seen this year for testing that your controls still do the thing they claim. A preventive control with no watcher is a story you tell yourself. Check that it fired, not only that it exists.

Get assurance from someone who didn’t build it and isn’t the system itself. Independent has to mean independent. If the only thing standing behind your agent deployment is a vendor’s own report or a model’s own account of its behavior, you have disclosure, and you should treat it like disclosure. The ability to observe what your agents are doing, and to reverse what they do when it goes wrong, are controls you build and own, and they belong in your architecture, not on a vendor’s blog. Agent observability and agent control are the shape of the thing. You don’t bolt them on at the end.

There’s a date in this report worth stealing. Anthropic’s own goal of getting comprehensive logging across all its internal AI development, what it calls “eyes on everything,” is a target for January 1, 2027. It does not have it today. The most resourced safety lab in the world has set itself a New Year’s deadline to see what its own agents are doing. If they don’t have eyes on everything, the odds you have it by accident are slim. Set your own date, and put a name next to it.

The Question You Now Own

Anthropic’s report ends on a question it can’t answer: did that agent touch only its own jobs, or something it shouldn’t have? They can’t tell you, because nobody was watching. The whole 186 pages, read the way I’m asking you to read it, is that one question at different altitudes: a classifier nobody logged, an entitlement nobody scoped, a filter nobody checked, a safety case resting on an assumption nobody dated, and an assurance no independent party authored.

This is the version with your name on it. Somewhere in your environment right now, an agent is acting with more authority than sense, and if it does the wrong thing, will you know before the damage tells you? If the honest answer is no, you don’t get to admire this one from a distance. It’s your next sprint. Your AI risk assessment is only as honest as your answer to it.

What to Do Next

Pick one agent deployment this week, the one holding the most authority, and answer three questions on paper. What’s the load-bearing assumption propping up my trust in it? Can its logging be turned off by the same thing that turns off its guardrails? If it does something wrong, what tells me, and how fast? That’s Create and Adapt work in the CARE model, and it’s the difference between reading Anthropic’s report and running your program off it.

The case that authority and autonomy are separate dials is in Least Agency vs Agent Autonomy. The boundary-testing argument, from the last time Anthropic handed us a live incident, is in AI Agent Capability Confinement. The advisory work lives at rockcyber.com.

👉 For ongoing analysis of agentic AI governance frameworks, the conversation at RockCyber Musings and you can subscribe below

👉 Visit RockCyber.com to learn more about how we can help with your traditional Cybersecurity and AI Security and Governance journey.

👉 Want to save a quick $100K? Check out our AI Governance Tools at AIGovernanceToolkit.com

👉 As a bonus, check out my conversation with Sean Martin from ITSP Magazine and John Sotiropoulos, Co-lead of the OWASP GenAI Security Project Agentic Security Initiative, during Infosecurity Europe about the newly launched OWASP Agentic Security Council, the alarming drop in attacker dwell time from eight hours down to twenty-two seconds, and what’s new in the 2026 OWASP Top 10 for LLM.