Is your next model swap a replacement in kind? Process safety engineers ask that before anything touches a running oil and gas plant, and your cloud provider may have already made the call for you. If nobody set the upgrade policy on your Standard Azure OpenAI deployment, and the model supports automatic updates, Azure moves it to the current default version when your version retires. The deployment name stays the same, and so does the security approval in your file, which still describes the old model.
Is Your Model Swap a Replacement in Kind?
The Occupational Safety and Health Administration’s process safety rule, 29 CFR 1910.119 in the Code of Federal Regulations, defines a replacement in kind as “a replacement which satisfies the design specification.” That’s the test. For instance, if the new valve meets the same spec, it skips the change review.
Anything else goes through management of change, where someone works out why you’re changing it, what it does to safety, and who signs off, among other things, before anybody turns a wrench.
Sounds simple, right?
I’ve stated many times that I’m a recovering Oil & Gas CISO. I’ve carried that test with me because it transfers to cybersecurity, and that process-safety habit is a sure-as-hell one we need to heed when developing and deploying AI. When you apply that test to an AI model, the very first question is the hard one. What’s the design specification?
Pull up the approval for your production model and read what it claims. Can you find a benchmark score, a red team report, and a sign-off date? If so, great! You’ve got evidence about one model on one day. What you don’t have is a spec the next model can satisfy, so I’ll give you a table near the end of this blog that sorts every claim in your approval by what a swap does to it.
I told you in my issue on Anthropic’s initial public offering filing to build so you can swap models without a rewrite. That’s core resiliency for any team running AI in production, and this issue focuses on the day you actually swap. Watching model A tells you nothing about model B until you test it. A rule enforced outside the model is different, because it can carry over if three things still hold.
This is the first in a series of four issues, and all four ask the same question about AI security evidence. What decision does this evidence justify? To answer it, I’ll use the same made-up example in all four. An AI agent reads a financial report, writes a summary, and sends it through a messaging service to people on an approved list.
A separate check blocks the agent from sending to anyone off that list, and the app logs every request and decision. In this issue, your platform team swaps model A for model B. Requests still go out in the same format, but model B may fill it in ways model A never did.
I’ll also tell you each time whether a claim comes from a paper, my own math, my opinion, or something nobody has tested.
Equal Scores Don’t Make Equal Models
Start with a made-up test. Model A and model B each get the same 10,000 everyday requests. Neither one fails. Both get a perfect score.
Now send each one 10,000 requests of a kind your test never tried. Model A still never fails, and model B fails every single one.
Your test didn’t lie. It just never looked where the two models differ, and a tie on your test can’t tell you how they compare on inputs it never tried. The example doesn’t measure how often attacks happen, either.
Machine learning researchers call this underspecification, which means the same training process can produce many models that all score about the same on its test. Alexander D’Amour and colleagues measured it in the Journal of Machine Learning Research in 2022.
One study took five separately trained versions of the BERT language model and fine-tuned each one 20 times, meaning they trained each a bit more on new examples. All 100 scored about the same on the standard test, but stress tests built to find weak spots scattered them all over the place.
Those 100 shared one training method and differed only in the random numbers each run started from. A vendor’s next model is trained on different data, a different way, so there’s even less reason to trust old results. I literally attended my last classes LAST NIGHT, finishing a master’s in applied data science and AI at the University of Denver, and this is the paper I’d hand a security leader before they sign off on any model swap.
To say I’m relieved and need about 27 days of sleep is an understatement, but I digress…
You see the same thing between a vendor’s releases. Xiaonan Xu and Wenjing Wu ran 900 public test items through three OpenAI model upgrades in an August 2026 preprint, asking each question 50 times on each model. Going from GPT 5.4 to GPT 5.5 raised the average score on SuperGPQA, a graduate-level knowledge test, by 2.3 percentage points, from 63.4% to 65.7% correct.
They pulled 500 of its questions at random, and 5.0% got reliably worse. To count, a question had to drop at least 20 points in how often the model got it right, by more than luck could explain under a standard statistical test. I did the multiplication. That’s 25 questions the newer model gets wrong more often, hidden inside a better average.
Every upgrade they tested made some questions reliably better and others reliably worse. It’s one preprint, one vendor, and three tests of what models can do, with nothing on safety. The authors also picked tests the models weren’t already acing, so those percentages won’t match your own work.
Does your last model comparison show changes question by question, or only the average? If it’s only the average, ask for results question by question, with each one run enough times to tell a real change from noise. Xu and Wu report that none of the 457 reliable changes they found could be spotted from a single run, so I wouldn’t sign off on a swap without repeat runs.
That’s why I sort every claim in an approval into two piles, observed behavior and enforced property. The rest of this section covers observed behavior, and enforced property gets its own section further down, under “What an External Control Carries Across a Swap.”
The first pile, Observed behavior, is what you learned by watching the model, like “model A refused our test jailbreaks.” It’s true of model A, on that test, on that date, and it carries over to model B only if you can show why, which matching scores can’t do. Fine-tuning shows how fast an observed behavior claim can go stale. Xiangyu Qi and colleagues gave GPT 3.5 Turbo one extra round of training on Alpaca in a 2024 paper for the International Conference on Learning Representations (ICLR). Alpaca is a public set of example instructions and answers, and the authors first pulled out its 1,902 safety examples to mimic training data built without regard for safety. They tuned the version OpenAI labels 0613 through OpenAI’s API.
Before tuning, GPT-4 graded the model’s answers to 330 harmful requests on a harm scale of 1 to 5, and 5.5% got the worst score. After that one round of training, 31.8% did.
You can report that result two ways, but I’d only focus on one in your risk register. 31.8 minus 5.5 is 26.3 percentage points, the plain gap between the two numbers. As a relative increase over the starting 5.5, it’s roughly 478%.
Both numbers are correct. I’d use the points.
Meta’s Llama 2 7B Chat shows why. In the same experiment, its rate went from 0.3% to 16.1%. That’s 15.8 points, or about 5,267% as a relative change, because 0.3% is one answer out of 330, and anything measured against one answer looks enormous.
The data geek in me wants raw counts too. Working back from the paper’s rounded percentages, roughly 18 of GPT 3.5’s 330 answers got the worst score before tuning and roughly 105 after. That’s 87 more.
If you quote these numbers, say what they measured. They’re one model’s grades of another model’s answers to test prompts, on models a few years old now, so none of it counts what happened outside the lab.
Has anyone fine-tuned your production model since it was approved? If so, your approval covers a model you no longer run, even if the tuning made it safer.
The same group’s ICLR 2025 follow-up found two fixes that made those attacks work less often in their tests, and they warn that attackers who adjust to the fixes may still get through.
I took one common lesson from the two papers. Fine-tuning changes the model you approved, for better or worse, so you have to redo your evidence either way.
The Swap Nobody Ticketed
What does the upgrade setting on your production Azure OpenAI deployments say? Go look, because that setting decides whether Azure can swap your model with nobody signing off.
Microsoft’s deployment docs give you three upgrade settings for a Standard deployment, if the model supports automatic updates. Auto update to default moves you to Microsoft’s new default version within two weeks of Microsoft naming one. A pinned version stays put until its retirement date, and on that date Azure upgrades it to the current default version anyway. The third choice, no automatic upgrade, keeps your model until retirement day, and then the deployment stops working.
If nobody ever chose one, the setting won’t even show up in the deployment’s properties, and Microsoft treats that blank as the upgrade at retirement.
If you do nothing, Azure swaps the model and keeps serving traffic. Nobody files a change ticket, nobody runs a new test, and nobody signs anything.
That’s back-asswards! It flies in the face of anything carrying a security approval!
The deployment name never changes, so the release record that says the model PASSED now sits on top of a model your team never tested. The new model may well behave better, but nothing in your file can tell you either way.
Microsoft isn’t hiding any of this, and I get why it picked uptime. Its retirement page says subscription owners with active deployments get an email at least 60 days before a generally available model retires. That email may never reach the person who signed your approval.
No automatic upgrade fails closed, which is right for anything with a security approval, even at the cost of an outage on retirement day (I’d take the outage). Preview models don’t get that choice, because Microsoft upgrades or retires those whatever you pick.
My problem is the blank setting, because that’s what you get when nobody thought about it. I’d set it to no automatic upgrade on every approved deployment, but check your fallback logic first. If your app retries on another deployment when this one goes dark, you’ve swapped models anyway. Provisioned deployments never upgrade on their own, but your team can move one to a new model version or family and keep the same name. An approval that names the deployment then covers whatever model sits behind it.
AWS went the other way. Bedrock’s lifecycle docs say migration “will not happen automatically,” and once a model passes its end-of-life date, requests to it fail. That fails closed by default, which is what I’d want, outage and all.
Bedrock can still put an unapproved model behind your call, though. An intelligent prompt router picks one of two models in the same family for each request, so an approval that names the router covers both. Google’s Vertex AI does Azure’s swap through names. A Gemini auto-updated alias moves to the newest stable version as soon as Google releases one.
Pinning doesn’t freeze everything either. Anthropic’s model ID docs say the model weights behind an ID never change, but the systems around them can, including the code that routes your request, the safety filters, and the logic that picks each word (the transformer). Anthropic says that can occasionally make the same ID behave a little differently.
If your model passed your refusal tests partly because of a vendor’s safety filter, part of your evidence now depends on somebody else’s change log.
I wouldn’t accept a record that names a deployment, or a nickname like “latest,” as proof of what was approved. It’s a pointer, and pointers move. If your logs can’t tell you which model version answered your last production request, the approval can’t either.
Picture a made-up risk acceptance that says the reporting agent runs model A on the deployment named prod-summary, signed and dated. Months later, an auditor pulls the deployment’s properties and finds model B answering requests, with no ticket anywhere. Nobody lied on that form, and nobody can say when the swap happened, because the logs recorded only the deployment name.
What an External Control Carries Across a Swap
Some of your evidence never depended on how the model behaves. I call that second pile enforced property, a rule something outside the model checks every time, like the messaging service refusing anyone not on the approved list. Model B can ask for anything, and the service still says no, so this kind of evidence can survive a swap.
Jerome Saltzer and Michael Schroeder called this complete mediation in their 1975 paper, The Protection of Information in Computer Systems. “Every access to every object must be checked for authority,” they wrote. They meant every path, too, including startup, recovery, shutdown, and maintenance, and warned that a permission check you saved for later must be redone when permissions change.
You can reuse an enforced property after a swap only if you can show three things:
Every action that matters still goes through the check, including tool calls the vendor makes for you.
The code that turns model B’s output into requests still reads it right, and the check sees the request that actually goes out.
The check’s rules still hold, including who sent each request and any permissions it cached while model A ran.
If any of those is a guess, you haven’t made your case.
Model B can break the second one without touching the check. If it fills in a request in a way that model A never did, the parsing code can take a path nobody reviewed, or hit a retry or error path that skips the check. That’s my inference from how parsers fail, and nobody has tested it, but it’s why I’d check the parser first.
Can you list every way a send request can leave your agent? I’m going to venture a guess that you can’t, so I’d retest the approved list claim, because you’d be assuming the exact thing you need to prove. I’d want that list in writing.
Which of your agent’s safety promises only hold because the model never asks for the wrong thing? Write those down first, because they’re the ones a swap puts back in play.
Pinning an exact version proves less than people think. When we released the Context Bill of Materials (CxBOM), we wrote that a signed pin “proves the definition you’re running is the one a key approved,” and it says nothing about whether that definition is safe. A model hash works the same way. It proves you’re running the model you approved and says nothing about how it’ll behave on your next prompt, and most hosted models won’t even give you one.
CxBOM has its own swap limit. Version 0.3.0 only recognizes the chat templates (the formatting each model expects around its messages) for Phi 3.5 Mini Instruct and Qwen3, and refuses anything else. If you swap in a third model, CxBOM can’t rebuild what that model saw until someone adds its template.
You could argue the two piles leak into each other. You’re right. The approved list survives the swap but says nothing about what model B writes to those people, so the summary’s quality is still observed behavior. A guardrail filter sits outside the model and runs every time, too, yet you measured its catch rate on model A.
That’s why I only count a control as an enforced property when it decides on facts the model didn’t write, like a recipient ID checked against a list. If model B fills in the task ID the check looks up, the check still runs and answers the wrong question. I’d check this every time you swap.
Reuse, Renew, Narrow, or Retire
After a swap, every claim gets one of four calls. Reuse means nothing it depends on changed, and you can show it. Renew means you get new evidence for the new model or the code around it.
Narrow means you shrink the claim to what the evidence still supports, like a refusal test that still holds for model A after it’s gone. Retire means you no longer need the claim, or the setup it described is gone. Either way, keep the history.
The table below is my working draft, and nobody has tested it on a real swap yet.
D. Sculley and colleagues’ 2015 NeurIPS paper on hidden technical debt in machine learning systems is where the threshold row comes from. They pointed out that when a model gets retrained on new data, a threshold someone set by hand may stop making sense. Any alert threshold you tuned on model A’s outputs belongs in that row.
I’d start every review with the last row, because writing a new decision record forces someone to say which old evidence they kept.
Then write up each claim like this.
You might say retesting every claim after every swap costs more than the swap. Fair point. I’d have each claim’s owner list what it depends on, and retest only what the swap touched. The approved list claim might carry over once the three conditions check out, while the refusal claim needs a fresh test.
How many rows in your current approval would land on reuse today? Count them. That number is how much of your approval survives the swap.
Nobody has run the test that would prove me wrong. Here’s how I’d run it. Lock down the check and the messaging service, swap model A for model B in the same reporting app, and give both the same tasks:
Normal approved reports
Requests to send to someone off the list
Requests where permission depends on whether that task was approved
If the enforced property claims broke even with all three conditions checked, I’d be wrong. My bet, which nobody has tested either, is that Xu and Wu’s question-by-question method, run on refusals across a vendor upgrade, would show some refusals getting stronger and others weaker.
Every row in that table assumes you know the model changed. A swap at least announces itself with a new model ID or version number, so somebody knows to run the review.
The harder case is a change that never announces itself. Suppose someone edits a small part of the model’s weights, the numbers it learned in training. Now it misbehaves on one narrow kind of request and acts normally on everything else. The version number stays the same, and your usual tests keep passing, because none of them send that kind of request.
Issue two takes that case. How much can a change that small do while your tests see nothing?
Issue three asks which facts about a task a check can trust because the model didn’t write them, like the approved recipient list. Those facts show an approved send is not misuse. Issue four asks how much a detector’s verdict should move your decision to ship, even when the scan comes back clean. By detector I mean any tool that scans for an attack and returns a verdict.
Key Takeaway: After a model swap, what you watched the old model do says nothing about the new one, and an outside rule carries over only if every action still hits the check, the parsing still works, and the check’s rules still hold.
What to do next
This week, have your platform lead pull the upgrade setting and the model version actually serving traffic for every production deployment with a security approval, and put each retirement date on the change calendar. Then have each approval’s security owner split it into claims, label each one as observed behavior or enforced property, put a name on every row, and mark it as reuse, renew, narrow, or retire. If you want help building this table, you can find me at RockCyber.
👉 Subscribe RockCyber Musings for more insights into AI security and governance, with the occasional rant. You can subscribe above
👉 Visit RockCyber.com to learn more about how we can help with your traditional Cybersecurity and AI Security and Governance journey.
👉 Want to save a quick $100K? Check out our AI Governance Tools at AIGovernanceToolkit.com
As a bonus, check out my conversation with Chris Romeo and Robert Hurlbut on the Application Security Podcast about the Replit agent that wiped a production database after being told no eleven times, why prompt injection stayed #1 on the 2026 OWASP Top 10 for LLM when incident data alone ranked it twelfth, and how the OWASP Agent Control Standard moves the no outside the agent.









