Agentic AI Breach Containment: Who Signs Off on Killing Production?

GitBook went dark 15 hours to stop an attacker. RockCyber Musings shows why agentic AI breach containment hinges on the smallest identity you can revoke.

On November 2, GitBook stopped an attacker by turning itself off. Agentic AI breach containment from October 30 to November 4 was mostly a story about switch size. At 20:32 UTC, GitBook disabled every affected infrastructure service account, and the app went dark until 11:56 the next morning. Its alert had fired 22 minutes after the attacker reached production, and the expensive part of the night came after that alert.

GitBook Stopped the Attacker by Turning Itself Off

GitBook’s November 4 post-mortem lays the night out to the minute. The attacker hijacked an account at one GitBook customer at 12:27 UTC, and GitBook hasn’t said how. A flaw in GitBook’s single sign-on (SSO) login flow then let it create sessions acting as GitBook staff.

The attacker had staff sessions by 16:32. It queried the authentication details of a misconfigured GitHub App in GitBook’s own organization, which opened GitBook’s code repositories and, eventually, a Cloudflare API key. That key led to production secrets, which granted full infrastructure access through a service account (an identity a program uses, with no person behind it). The attacker reached production at 17:48.

GitBook’s alert fired at 18:10, and its investigation started then. The SSO flaw was fixed at 19:58. At 20:32 GitBook disabled every affected infrastructure service account, and in its own words, “That stopped the attack, and it also took the GitBook app offline.”

GitBook says it chose the outage on purpose, because a platform that stayed up was one the attacker could keep searching. It hasn’t said whether a smaller switch existed, and its own sentence tells you production ran on the identities the attacker held.

GitBook stayed offline until 11:56 UTC on November 3, 15 hours and 24 minutes by its timestamps (it calls it about 16). It rotated every production secret and every customer secret it manages and invalidated every API token issued before the cutoff. The attack targeted one customer, and GitBook treated every customer’s data as exposed because the attacker could reach all of it.

Most of those hours went to rebuilding. Redeployment with new secrets began at 02:34, six hours after the cutoff, and the last secrets rotated at 11:56.

The attacker got about 40 more minutes of production access early on November 3, by a route GitBook hasn’t described, so the first switch didn’t close every path. Beyond the investigation and the SSO fix, GitBook doesn’t say what filled the 142 minutes before the cutoff, and closing the way in before you evict is normal practice. GitBook bases its AI attribution on the attack’s speed and precision and names no tool.

Five Companies Picked Five Different Switches

GitBook’s switch was the biggest that week. Northflank, a cloud deployment platform, got a threat alert from Wiz on October 30 and kept its platform and customer workloads online throughout. Its status page still told every customer to “Assume all environment variables have been decrypted. Disable/rotate all keys.” Customers on dedicated or forward-deployed control planes weren’t affected, and that’s a branch breaker. Northflank hasn’t said an AI agent was involved.

Chainalysis tried a smaller switch first. On November 2, it took Reactor and Data Solutions offline. Its incident page says it kept its compliance products running while it investigated, because customers use them to catch money laundering.

By November 4, Chainalysis had found password hashes and live tokens for every user exposed. That day, it saw abnormal activity and took KYT (its transaction-monitoring product) offline, along with address screening and the login page. Some KYT accounts appeared to have been accessed, and every user had to reset passwords and rotate API keys. Chainalysis cites “offensive agentic techniques” and names no tool.

WorkOS widened its switch in steps. On October 30, it traced a spike in customer-database queries to an internal tool its go-to-market teams use. Records had already been pulled in bulk, and WorkOS cut the tool’s public access that day. By October 31, it had revoked or rotated credentials for connected services and restricted access to every internally developed tool.

WorkOS’s November 6 update says only sales and support data were affected. The intruder was a security researcher’s agent built on ARTEX (an open-source agentic framework for penetration testing), and it had escaped its testing environment and reached WorkOS unintentionally. Every containment call came before WorkOS knew whether it faced a criminal or a broken experiment, and yours will too.

Trigger.dev did something different. Its incident report says the attacker obtained customer root API keys for some environments, and a root key gives full access to that environment’s settings, including environment variables. Trigger.dev checked API access logs across every customer environment, revoked keys only for the environments touched, and told everyone else that if they hadn’t been notified, they weren’t affected. That was 0.66% of its customers.

Trigger.dev also rotated its own internal credentials and found no evidence the attacker reached its services or production databases. It hasn’t said how the attacker got in and makes no claim about AI, so I’m using it for its containment alone.

Electrical engineers wrote this problem down a long time ago. Selective coordination is, in the opening words of the National Electrical Code (NEC) definition, the “localization of an overcurrent condition to restrict outages to the circuit or equipment affected.” When a fault hits one branch, only the protective device directly upstream of it opens, and every other branch stays energized. For emergency systems, 700.32(A) of the 2023 NEC makes it mandatory, and 700.32(C) requires a fresh evaluation whenever the system changes (a National Fire Protection Association amendment proposal quotes both).

GitBook’s fault started at one customer account and climbed through the SSO flaw, the GitHub App, and the Cloudflare key into the service account production ran on. Once it sat there, the nearest device was the main, and opening it was coordination doing its job. The design problem sat further back, in a path that let a branch fault reach the main bus at all.

Trigger.dev’s per-environment root keys were crude branch breakers, and the comparison has a limit. GitBook’s attacker held the identity production ran on, and Trigger.dev found no sign its attacker got that deep. The design question is whether any one identity should reach that far.

Figure 1: One Breaker Versus the Main

GitBook’s Cutoff Took Six Times as Long as Its Alert

Microsoft’s November 5 report on agentic AI tooling tells defenders to use AI to investigate and respond. GitBook’s own lesson asks for automated detection plus a practiced plan to contain fast. Both cover the whole clock, which has three terms that run one after another:

\(T_C = T_{\text{detect}} + T_{\text{decide}} + T_{\text{enforce}}\)

The terms are the time until someone knows, the time to choose what to switch off, and the time until the revocation lands. Call the sum the containment time, and call the harm time the time it takes the attacker to cause the first irreversible harm if nobody stops them. Containment prevents that harm only when

\(T_C < T_H\)

and only when both clocks start at the same moment, the switch covers every path the attacker is using, and nothing already in flight can finish the job. Start the clocks at different moments, and any response can be made to look fast.

Start GitBook’s clocks at 12:27 UTC, when the first customer account was hijacked. I’m treating production access as the harm that set the scope, since once the attacker held production secrets, GitBook treated every customer’s data as exposed. Its timeline lists source code as reached at 16:32, and you can’t rotate a leaked codebase, so the real harm clock ran shorter.

\(T_H \le 321\ \text{min} \quad < \quad T_{\text{detect}} = 343\ \text{min} \quad < \quad T_C = 485\ \text{min}\)

Detection came after the harm, so containment could only cap what happened next. On the 12:27 clock, detection was the long term at 343 minutes, because nothing fired in the five hours and 21 minutes before the attacker reached production.

Restart the clock at production access, and the order flips, with 22 minutes to the alert and 142 to decide and enforce (GitBook doesn’t split those two). After the harm lands, a faster alert can buy back at most 22 minutes. The 485 is only the first cutoff, since the attacker came back on November 3.

Figure 2: GitBook’s Night on One Clock

Google Threat Intelligence Group draws the same line in its ARTEX writeup, saying “simply detecting the attack in time does little to hinder objective completion.” My read is that the decide term grows with the size of the switch, because taking production down is a decision somebody has to own.

Microsoft watched one AI agent go from searching GitHub for a client secret to exfiltrating data from Azure Key Vault in five minutes. That’s one observed path, and that speed is the part of this story specific to agents, since the containment math doesn’t care who the attacker is.

Detection decided two of the five outright. Northflank’s first known decryption of secrets came about nine hours before Wiz’s alert. WorkOS didn’t spot an agent that got in on October 25 until October 30, after the bulk download. Granularity decides what the stop costs you, and it only gets a vote once someone knows there’s something to stop.

Revocation Leaks Through the Tokens You Already Issued

Pulling the switch doesn’t end the attacker’s access, either. Microsoft’s containment steps tell you to remove every client secret, certificate, and federated credential from an affected app, including ones the attacker added, and to disable its service principal if needed. The same list adds, “Tokens already issued remain valid until expiry.”

A service principal is the identity an application uses in Microsoft Entra ID, Entra’s closest equivalent to a service account. Microsoft’s report says it gives direct, programmatic access to whatever the application is authorized to reach. Once you disable one, the question is how long its tokens keep working.

Entra’s token lifetime documentation puts an access token’s default lifetime at a random 60 to 90 minutes, varying by client, resource, and Conditional Access. A lifetime policy can stretch it to just under a day. Continuous access evaluation (CAE) lets a resource reject a token that hasn’t expired yet. It does that in near real time when Entra reports a critical event, such as the account being disabled.

For workload identities, CAE covers requests to Microsoft Graph and, for now, nothing else. It covers single-tenant service principals only; it leaves out managed identities and multi-tenant apps, and it’s opt-in. An app gets CAE tokens only if it declares the cp1 client capability, and an attacker running your stolen client secret from their own tooling decides whether to ask.

My read is that disabling a service principal leaves its issued Azure Resource Manager and Key Vault tokens working until they expire. For a multitenant app like the vendor-owned one in Microsoft’s report, the Graph tokens survive too. Test that against a real token from each production identity. The faster switch for Resource Manager and Key Vault is removing the role assignment, which Microsoft’s Azure role-based access control (RBAC) troubleshooting page says lands within 10 minutes.

Managed identities, Microsoft’s recommended replacement for client secrets, carry their own lag. Its guidance says a group or role membership change can take several hours to land, because it rides in a token cached for around 24 hours. It recommends assigning permissions directly. Its FAQ says Azure RBAC keeps honoring a deleted identity’s tokens until they expire, so deletion is the slow switch.

Token exchange, defined in RFC 8693, lets a security token service (STS) trade one token for another. It’s one way an agent turns its credential into a token for a downstream system. The RFC says the exchange creates no tight link between input and output, and that propagating revocation “is not a general property of the STS protocol.” Whether children die with the parent depends on how your token service is built.

Google’s report on LLMjacking, stealing cloud credentials to run AI on someone else’s bill, shows where short-lived credentials stop helping. In one Mandiant case, an actor used token-creation privileges to mint temporary credentials for higher-privileged service accounts and pulled credentials from cloud secret managers. Other actors created backdoor service accounts, and one ran what Google calls burn and rotate, continuously creating and deleting short-lived service accounts to get around rate limits.

All of that stretches the clock past the switch. Add a term W for whatever the revoked identity already handed out, and call the total the effective containment time:

\(T_C^{\text{eff}} = T_C + W, \qquad W = \max_{k \in \mathrm{Desc}(i)} \big(t_{\exp}(k)-t_{\text{revoke}}\big)^{+}\)

Here, i is the revoked identity, and the revoke time is when that revocation lands, the end of the enforce term. Desc(i) is everything descended from i, meaning tokens it was issued or exchanged, identities it minted, credentials it read or added, and whatever those mint in turn. The expiry term is when descendant k stops working on its own, and the plus sign drops descendants that already expired.

For tokens alone, with default Entra lifetimes and no CAE, W runs up to about 90 minutes, and that’s the optimistic case. Microsoft’s actor went after storage account keys, connection strings, and publishing credentials, which work until you rotate them, and an identity the attacker created works until someone finds it. The attacker sets part of this window, and the condition becomes

\(T_C^{\text{eff}} < T_H\)

Figure 3: What Survives When You Pull the Credential

Agentic AI Breach Containment Starts With the Smallest Switch

The attacker’s side fits in one expression, the worst loss reachable in the time they have:

\(D_{\max}(q, s_0, H) = \sup_{\text{path} \in \text{Allowed}(q, s_0, H)} L(\text{path})\)

Here, q is the task the identity exists to do, s₀ is your systems’ state when the attacker gets in, and H is how long they have. Allowed(q, s₀, H) is every attack path your enforced policy permits in that window, including identities and tokens the attacker can derive. L(path) is one path’s loss in dollars and the supremum (sup) is the worst case.

Set H to the effective containment time. If the attacker can always stop early and neither your policy nor a sequence’s loss changes with the window, a longer window can’t shrink the worst case.

Then add what the stop costs. If u is the unit you can revoke and K(u) is the price of switching it off, in the same dollars, the worst case for the incident when you pull u is

\(L_{\text{total}}(u) = D_{\max}\big(q, s_0, T_C^{\text{eff}}(u)\big) + K(u)\)

The sum holds only when revoking u cuts every path the attacker holds, and when the outage and the attacker’s damage don’t count the same dollars twice.

GitBook’s unit was the platform, and its K was the price of 15 hours and 24 minutes offline plus a new API token for every customer using its API. Trigger.dev’s unit was the environment, and its K landed on 0.66% of its customers.

A smaller unit always cuts K. It cuts the first term only when the small switch still covers every path the attacker holds, and you can choose it faster than the big one. A drill tells you whether both hold.

The strongest objection is that you don’t know what the attacker touched, so you rotate everything and unit size stops mattering. Chainalysis is the best evidence, since its smaller switch left KYT up until accounts there appeared to have been accessed. Its exposure ran through what it calls an “on-path attack” and reached every user’s live tokens. No identity design hands you a short list for a shared path like that.

Granularity pays off when you know which application was compromised. Microsoft’s containment list then has you rotate “everything stored in any Key Vault the affected application could read,” and a narrow identity makes that list short. GitBook’s attacker held full infrastructure access, so its list was everything.

The second objection is that more identities means more credentials to lose. A federated workload identity removes the static secret, though Microsoft’s list has you strip federated credentials the attacker added. Give every identity an owner and a review date, and keep the authority to create identities away from the workloads that use them.

Start with your ten highest-reach non-human identities, ranked by the secret stores they can read, what they can deploy, and whether they can create identities or tokens. For each, write down what breaks if you disable it now, how long its tokens outlive revocation, and who may pull the switch without calling anyone. An identity whose first answer is “production” is a main breaker. Pre-authorize your on-call incident commander to disable it, with the outage cost written down, because that’s the decide term.

Then drill it. In a game day, capture a token the identity holds, disable the identity, and replay the token against every resource it touches until each one refuses it. That measures the enforce term and W. The decide term needs a tabletop with the business owner who’d sign the outage.

My September 29 post covers revoking an agent with a request still in a queue.

A clean drill record proves less than it looks. The exact binomial upper confidence limit is the highest failure rate still consistent with what you observed, at a confidence level you choose.

NIST’s description of the exact method prints the two-sided version, which splits α between two tails. NIST’s Dataplot also computes the one-sided upper limit, which puts all of α in one. With zero failures in n independent drills, the one-sided version collapses to one line:

\(\Pr(\text{0 failures in } n \mid p) = (1-p)^n, \qquad (1-p_{\text{upper}})^n = \alpha \quad\Longrightarrow\quad p_{\text{upper}} = 1-\alpha^{1/n}\)

Here p is the true chance a drill fails, and α is the chance you accept that the limit lands below the true rate (0.05 for 95% confidence). The upper limit is the largest p under which n clean drills still happen at least 5% of the time. Twenty clean drills put the one-sided 95% limit at 13.9%, and a hundred put it at 2.95%.

An attacker who adapts to your switches breaks the independence assumption. Treat 13.9% as the most optimistic ceiling you can claim, and restart the count after every change to the identity.

Figure 4: What a Clean Drill Record Proves

Check me on March 31, 2027. Take every postmortem published by then in which the company attributes the intrusion to an AI agent or agentic framework and says what it revoked.

I expect companies with per-customer or per-environment credentials to report scoped revocations, and most of the rest to report a reset across every customer. If I can’t find five that qualify, the test didn’t run. If the scoped companies reset every customer about as often as the others, I’m wrong.

Trigger.dev could see from its API logs which environments the attacker touched, and its root keys were scoped to those environments. It revoked keys for the 0.66% of its customers involved and told everyone else they weren’t affected.

What to do next

In the CARE framework (Create, Adapt, Run, Evolve) I use with clients at RockCyber, the revocation drill belongs in Run. Re-coordination belongs in Evolve, the way NEC 700.32(C) sends an emergency system back for review after every modification. Start with the CARE framework if you’re mapping identities to owners for the first time, and my March 17 post covers the authorization gap at the token boundary.

👉 Subscribe for more AI security and governance insights with the occasional rant.

👉 Visit RockCyber.com to learn more about how we can help with your traditional Cybersecurity and AI Security and Governance journey.

👉 Want to save a quick $100K? Check out our AI Governance Tools at AIGovernanceToolkit.com

👉 As a bonus, check out this conversation with Chris Hughes over at Resilient Cyber, who’s also a reviewer on the paper, to discuss all things about the 2026 update to the OWASP Top 10 for LLMs, including why we need to worry about model context.

.