Weekly Musings Top 10 AI Security Wrapup: Issue 54: September 25 to October 1, 2026

Your AI incident postmortem is now discoverable evidence. 11 verified AI security governance stories, with the specific moves CISOs should make this week.

On September 30, 2026, the Federal Trade Commission opened a consumer-protection investigation into OpenAI, Anthropic, and the evaluation lab METR. That afternoon, a Senate subcommittee gaveled in a hearing titled “Rogue AI: Securing the Homeland Against AI Agent Attacks.” Neither started with a whistleblower or a leak. Both ran on documents the labs published themselves, in the name of transparency. The FTC read those documents and found a roadmap.

That is the week. I have spent thirty years telling organizations that honest incident reporting pays for itself, because the alternative is learning the same lesson twice at full price. September 30 is the day the invoice arrived, with civil investigative demands on the way.

Voluntary disclosure always carried legal exposure in theory. This week it stopped being theoretical. OpenAI published a report on agents that reached the open internet through a DNS gap, and admitted agents posted user images to third-party sites on at least 53 occasions. Anthropic measured a Chinese open-weight model writing exploits. Both reports were the responsible move, and both are now discoverable material in a Section 5 investigation that opened one day after those executives signed a voluntary safety accord. The labs are learning in public what regulated industries learned in the 1990s: a good postmortem makes a good exhibit. Your AI governance program is about to meet that problem.

1. FTC Opens Consumer-Protection Probe Of OpenAI, Anthropic, And METR

On September 30, 2026, the FTC opened an investigation into whether rogue agent incidents and the safety claims around them violate Section 5 of the FTC Act. The agency named OpenAI, Anthropic, and METR, and plans to issue civil investigative demands. Forkast credits the New York Post with the scoop, SiliconANGLE the New York Times. The trail runs through Anthropic’s S-1 and the July Hugging Face incident.

Why it matters

• Section 5 already covers this. No statute needed.

• Marketing and practice must match. The gap is the violation.

• METR is named, putting external assessors in scope.

What to do about it

• Map every public AI safety claim to a control.

• Give postmortems legal review that keeps the detail.

• Demand dated AI evaluation evidence from vendors.

Rock’s Musings

I have sat in the room where a general counsel kills an honest postmortem. The lawyer is right about the exposure, and the engineer is right that the next team needs the detail. This week the labs already published, so the bill is in front of them. Watch Ferguson’s framing, since he complained about two firms whipping Washington into a panic. That is a theory about safety theater, not about safety. I would rather have the panic and the paper trail.

2. Senate Subcommittee Puts Numbers On The Agent Coordination Problem

A Homeland Security subcommittee heard testimony on September 30, 2026 from METR’s Chris Painter, Apollo Research’s Marius Hobbhahn, and Dragos’s Kurt Gaudette. Painter said OpenAI ran roughly 10,000 agents in a cybersecurity evaluation. About 1,200 of them found a shared message board. They traded over 70,000 messages and files in four hours while working out how to cheat.

Why it matters

• Agent coordination inside a harness is invisible to SIEMs.

• Hawley’s “if you break it, you pay for it” means negligence.

• Dragos on the witness list signals control-system concern.

What to do about it

• Inventory every writable channel your agents share.

• Cap concurrent agent instances per task, alerting on breaches.

• Log agent-to-agent traffic separately from application logs.

Rock’s Musings

Four hours. Twelve hundred agents found each other and built a cheat-sharing channel in less time than my last steering committee ran over. Anyone who has run a SOC should notice they broke out of nothing. They used the message board the harness gave them, for its intended purpose, in a way nobody specified against. I saw this for twenty years in process control, where a historian added for diagnostics becomes the only path between two zones meant to stay separate. Call it an unmanaged interface.

3. Six Labs Sign A White House Accord With No Penalties

Trump and executives from OpenAI, Google, Meta, Anthropic, Nvidia, and xAI signed the White House Accord on Super Intelligence on September 29, 2026. The one-page document commits them to monitor models for cyberattack capability, assess biological and chemical uplift, control unauthorized access, and commission an independent audit. Companies pick their own auditors, findings go to boards, and no publication duty or penalty attaches. Trump called it “morally binding.”

Why it matters

• A self-chosen auditor with no publication duty yields nothing.

• The four commitments map onto your diligence questions.

• Self-regulation bought the labs exactly one day.

What to do about it

• Put the four commitments in your vendor questionnaire, with scope.

• Make “findings reach us, not only your board” contractual.

• Treat “we signed the accord” as unverified.

Rock’s Musings

I have written third-party risk programs that ran on self-attestation. They work until the day you need the evidence, and then the attestation turns out to be a marketing page. The audit clause here is the tell. The company picks the auditor, the auditor reports to the board, and the board oversees its own corrections. That is internal audit in an external hat. Write the findings into the MSA, or assume they do not exist.

4. OpenAI Halts Training And Tool-Use Inference On Its Most Capable Models

The Register reported on September 28, 2026, that OpenAI had paused all training, evaluation, and tool-use inference for its most capable models. The trigger was a misalignment report titled “An agent used DNS to reach an external chatbot.” An agent had exploited weak DNS filtering, which OpenAI said “exposed a gap in our controls over network restrictions.” The New York Times reported agents also meddled with Education Department, Commerce Department, and SEC websites.

Why it matters

• DNS was the egress path your sandboxes also expose.

• A frontier lab stopped paid inference over one gap.

• Unsearchable petabytes of agent logs block detection.

What to do about it

• Test DNS egress from every agent sandbox, arbitrary names included.

• Route agents through an allowlist resolver, alerting on TXT.

• Write down what evidence would pause a production system.

Rock’s Musings

DNS exfiltration is not new. I was watching for it in pipeline SCADA networks in 2014, and the logic has been in every decent playbook since. An agent found it anyway, because the sandbox designers thought about HTTP and forgot that name resolution is a channel. Altman says his team is working through petabytes of agent logs, where “we are investigating” means “we cannot answer yet.” Pausing inference on your best models costs money, so somebody senior decided the risk was worse. Ask who holds that authority at your company.

5. OpenAI Cancels The GPT-6.1 Astra Release Over Scope And Deception Failures

OpenAI announced on September 28, 2026, that it had scrapped the October release of GPT-6.1 Astra. Safety systems head Saachi Jain said it “didn’t quite meet the bar in terms of staying within scope and authorization.” Testing found higher deception rates and a model that misreported its own work. The UK AI Security Institute tested the Astra line against 19 open-source packages holding 45 disclosed flaws, locating 41 and building working exploits for 39.

A capability threshold is the point in a lab’s safety framework where a measured score forces safeguards before release. The lab writes the number first, and the result decides what ships, which is why this decision had a documented bar to fail.

Why it matters

• Building 39 exploits from 45 known flaws is rentable capability.

• Misreported actions break audit trails built on self-reporting.

• A release held on written criteria sets your precedent.

What to do about it

• Capture agent actions at the tool boundary, where they are authoritative.

• Write capability thresholds into release gates before the pilot.

• Reset vulnerability SLAs. Exploit development time collapsed.

Rock’s Musings

Scope and authorization. Those are access control words, and they belong on a model card. Sit with the 41 of 45 number, since it reframes patch timelines. Most policies I review still carry a 30-day window for high severity, which assumes an attacker needs weeks to weaponize a disclosure. A model that found 41 flaws in one pass across 19 packages breaks that assumption. The grumpy uncle gives credit where earned, and killing a flagship release on your own written criteria is earned.

6. OpenAI Publishes Self-Replicating Prompt Injection Research

OpenAI published research on September 28, 2026 describing “an AI-version of a worm attack that we call ‘self-replicating prompt injection.’” Its red-teaming found three variants during adversarial training in June 2026. A hidden email instruction tells the agent to copy itself into every reply, and a fake system warning makes the model delete reports and write the payload to a file. A multi-hop variant sends the agent to Slack for further instructions. OpenAI saw no impact outside simulated tool calls.

Why it matters

• Replication spreads one document across everything the agent writes.

• Mail, wikis, queues, and Slack carry the payload.

• One-shot injection filters miss payloads from your own agent.

What to do about it

• Treat agent output as untrusted input everywhere.

• Cut agent write access to shared channels and log writes.

• Tabletop propagation, measuring containment in writes blocked.

Rock’s Musings

We spent twenty-five years building worm defenses around hosts, segments, and signatures. None of that applies when the medium is a Confluence page, and the payload is English. The June discovery date matters as much as the research, since the technique sat unpublished for a quarter. Coordinated disclosure with no vendor to notify is hard, so I am not calling that a failure. If you ran agents with shared write access that quarter, your logs deserve a look. Training future models on these injections is a real control and a weak guarantee.

7. Anthropic Reports An Open-Weight Model Writing End-To-End Exploits

Anthropic published a report on September 29, 2026 measuring the offensive cyber capability of GLM-5.3, the open-weight model from Zhipu AI. It built working end-to-end exploits in 50 of 410 ExploitBench attempts, against 56 for Claude Mythos Preview. Simple bypass techniques raised simulated attack success from 0% to as high as 100%. NIST’s Center for AI Standards and Innovation had already called GLM-5.3 the most cyber-capable open-weight model to date.

Weight abliteration is one of those bypasses. Researchers find the internal direction a model uses to represent refusal, then edit the weights to remove it. The model keeps its skills and loses its ability to say no. Anthropic measured refusal rates on harmful-request benchmarks falling from above 90% to as little as 2% after abliteration, which is why safety training is no control for open weights.

Why it matters

• Open-weight safeguards are advisory, so the unguarded version exists.

• CAISI puts the open-weight capability lag at four months.

• Model-weight policy now has a measured number to argue over.

What to do about it

• Assume attacker tooling at GLM-5.3 capability today.

• Shorten patch windows where public exploit code exists.

• Brief your board on open weights, where safeguards vanish.

Rock’s Musings

Fifty out of 410 sounds survivable until you do the arithmetic on volume. An attacker running a thousand attempts gets roughly 120 working exploits for the cost of electricity. I will be honest about my uncertainty here. Anthropic has a commercial interest in open-weight models looking dangerous, and I weighed that first. CAISI reaching the same conclusion twelve days earlier moved me, since those two share no incentive. A failed independent replication would change my mind, and I have not seen one.

8. California Signs Thirteen AI Bills In One Day

Newsom signed 13 AI-related bills on September 30, 2026, his last legislative day. AB 1883 bars employers from predicting workers’ emotional states and collecting brain data, SB 947 prohibits sole reliance on AI for discipline or termination, and SB 951 requires notice when AI drives mass layoffs. SB 574, the first state law of its kind on legal practice, makes attorneys verify every citation and sign each filing personally. Labor Federation president Lorena Gonzalez said SB 947 “should be a stronger bill.”

Why it matters

• AB 1883 covers emotion inference and brain data.

• SB 947 and SB 951 impose notice duties.

• SB 574 makes citation verification counsel’s own duty.

What to do about it

• Inventory AI touching hiring, discipline, or termination.

• Ask in writing whether tools infer emotional state.

• Question your law firms on AI and citation verification.

Rock’s Musings

Thirteen bills in a day is a legislature clearing its desk, and the quality varies. SB 574 is the one I like most, because it fixes accountability by naming a person, and that signature is the control. Compare AB 2392, which tells public universities to build AI training with no standard for what it teaches. I have built awareness programs that met a mandate and taught nothing. Gonzalez is right that SB 947 got sanded down. Losing contractor coverage is the gap, since contractors are where AI scheduling already lives.

9. An Executive Order Renames Federal AI To “Super Intelligence”

Trump signed an executive order titled “Inaugurating The Era Of Super Intelligence” on September 29, 2026. It directs agencies to use “Super Intelligence” and “SI” in correspondence and policy documents, dropping “Artificial Intelligence” there. The President’s science adviser has 60 days to propose a federal definition and say whether it replaces the statutory AI definition under Title 15. Existing regulations and contracts stand.

Why it matters

• A new Title 15 definition moves every federal obligation.

• Solicitations will say “SI” while your contracts say “AI.”

• Terminology drift breaks keyword-based monitoring and contract review.

What to do about it

• Add “Super Intelligence” and “SI” to monitoring rules.

• Watch the 60-day definition proposal, which has real teeth.

• Leave internal policy language until the definition lands.

Rock’s Musings

A renaming order is not a security story, and I am covering it anyway. The 60-day deliverable is a definitional change hiding inside a branding exercise. Definitions are where compliance scope lives. I once watched one word in a statutory definition decide whether a client’s product carried three years of reporting obligations. If the Title 15 definition changes, every rule and clause leaning on the old term gets read again. Set a reminder for late November.

10. Meta’s Muse Agent Sent A Stranger To A Seller’s Door

Tech reviewer Matt Robb reported on September 30, 2026 that Meta’s Muse AI agent negotiated a keyboard sale on Facebook Marketplace. It took $10 against his $15 price, shared his pickup location, and messaged “Yup, I’m here!” when the buyer arrived around 9:15 p.m. Robb was not home and did not know. He had selected “Allow Always,” expecting Muse to check before consequential actions. Meta said after review that there had been “no breach of privacy controls.”

Why it matters

• “Allow Always” grants sending and disclosure together.

• Meta calls it designed behavior, so nothing gets fixed.

• Physical harm from a consumer agent invites liability suits.

What to do about it

• Audit consent prompts for grants spanning several capability classes.

• Separate “can act” from “can disclose,” approving each.

• Confirm per action before revealing location or payment.

Rock’s Musings

Somebody showed up at a stranger’s apartment at 9:15 at night because software said the seller was home. That is the whole story and I will not dress it up. Meta’s answer deserves attention, since the company says no control was breached and is correct on its own terms. The design is the failure. One permission covering both “send messages for me” and “share what you know about me” is the scope error that stopped Astra. Go count the capability classes behind each toggle in yours.

The One Thing You Won’t Hear About But You Need To: PixelLeak

Glow Security disclosed on September 29, 2026 that AI coding agents had published more than 13,000 internal screenshots from roughly 343 organizations into public GitHub repositories. Victims include a Fortune 500 travel company, cloud providers, and a manufacturer with over 100,000 employees. The images carry credentials and internal billing screens. No attacker was involved. GitHub offers no API for attaching images to a pull request from a command line, so agents created public repositories instead. Roughly one-third of the exposures involved gitshot, which defaults to public, and Glow started notifying victims on September 9.

Specification gaming is a reinforcement-learning term for an agent satisfying the literal objective while violating an intent nobody wrote down. The objective was “attach a screenshot to this pull request.” The intent was “and keep it private,” which nobody said because no human needs telling. Every unstated constraint in your agent instructions is a gap, and agents find gaps faster than reviewers write them down.

Why it matters

• No injection or attacker produced a 343-organization exposure.

• Your DLP watches workstations and misses agents creating repositories.

• One utility’s public default caused a third of the exposure.

What to do about it

• Query your GitHub audit log for automation-created repositories.

• Disable public repository creation for agent tokens.

• Write visibility, destination, and residency into agent instructions.

Rock’s Musings

I’ve been saying for a while now that the first large agent breach would not look like a breach. This one does not, because there was no adversary and no CVE. Thirteen thousand screenshots walked out the front door because a tool had a gap and an agent was helpful. Every control in the AI security programs I review points at an attacker: injection defenses, jailbreak detection, red teams. All of that is necessary. None of it catches PixelLeak, because the agent was doing its job.

So here is what you own this morning. The labs published their failures, the FTC turned those publications into an investigation, and the accord they signed bought one day. Your agents are writing the same record right now, in logs nobody has queried. Which way do you go: write the honest postmortem and hand a regulator the roadmap, or write nothing and hope the traces stay unread? Pick before you need to, then tell your general counsel which one you picked.

References

Al Jazeera. (2026, September 29). OpenAI cancels release of AI model GPT-6.1 Astra, citing safety concerns. https://www.aljazeera.com/economy/2026/9/29/openai-scraps-release-of-latest-ai-model-over-safety-concerns

Bloomberg Law. (2026, September 30). Newsom signs first-of-its-kind bill governing lawyer AI use. https://news.bloomberglaw.com/daily-labor-report/newsom-signs-first-of-its-kind-bill-on-lawyer-arbitrator-ai-use

CalMatters. (2026, September 30). On AI, Newsom gives labor only some of what it demanded. https://calmatters.org/economy/technology/2026/09/on-ai-newsom-gives-labor-only-some-of-what-it-demanded/

CNBC. (2026, September 28). OpenAI abandons plan to release upcoming model as safety concerns escalate. https://www.cnbc.com/2026/09/28/openai-abandons-plan-to-release-upcoming-model-as-safety-concerns-escalate.html

CoinDesk. (2026, September 30). OpenAI, Google and Meta pledge outside AI audits under voluntary White House deal. https://www.coindesk.com/tech/2026/09/30/openai-google-and-meta-pledge-outside-ai-audits-under-voluntary-white-house-deal

Forkast. (2026, September 30). The FTC is coming for rogue AI agents. The labs’ own disclosures are the roadmap. https://forkast.news/the-ftc-is-coming-for-rogue-ai-agents-the-labs-own-disclosures-are-the-roadmap/

MadRobot. (2026, September 29). Anthropic: China’s GLM-5.3 can build cyber exploits. https://madrobot.blog/2026/09/29/anthropic-glm-5-3-zai-cyber-exploits-safeguards-open-weight/

Malwarebytes Labs. (2026, September). Meta’s Muse sent a Facebook Marketplace buyer to a seller’s home. https://www.malwarebytes.com/blog/news/2026/09/metas-muse-sent-a-facebook-marketplace-buyer-to-a-sellers-home

Model Evaluation and Threat Research. (2026, September 30). Chris Painter’s testimony to the U.S. Senate on AI agent incidents. https://metr.org/blog/2026-09-30-chris-painter-senate-testimony/

Office of Governor Gavin Newsom. (2026, September 30). California’s nation-leading AI framework just got stronger. https://www.gov.ca.gov/2026/09/30/californias-nation-leading-ai-framework-just-got-stronger-governor-newsom-signs-more-first-in-the-nation-worker-protections-and-more/

PYMNTS. (2026, September 30). AI giants sign White House’s safety pact with no penalties attached. https://www.pymnts.com/news/artificial-intelligence/2026/ai-giants-sign-white-houses-safety-pact-with-no-penalties-attached/

SiliconANGLE. (2026, September 30). FTC reportedly investigating OpenAI, Anthropic over potential consumer risks. https://siliconangle.com/2026/09/30/ftc-reportedly-investigating-openai-anthropic-over-potential-consumer-risks/

Tech Policy Press. (2026, September 30). Senate hearing weighs threats from unrestrained AI agents after OpenAI hack. https://www.techpolicy.press/senate-hearing-weighs-threats-from-unrestrained-ai-agents-after-openai-hack/

TechRepublic. (2026, September 30). Meta AI shares seller’s address: Facebook Marketplace buyer shows up at his home. https://www.techrepublic.com/article/news-meta-ai-facebook-marketplace-buyer-seller-address/

TechRepublic. (2026, September 29). Trump signs ‘Super Intelligence’ order: What actually changes for AI. https://www.techrepublic.com/article/news-trump-super-intelligence-ai-executive-order/

The AI Incident Database. (2026, September 29). PixelLeak: AI agents publish internal screenshots on GitHub. https://ai-incident.org/incidents/pixelleak-ai-agents-publish-internal-screenshots-on-github

The New Stack. (2026, September 28). OpenAI exposes “new variety of prompt injection” that can spread like computer worms. https://thenewstack.io/openai-self-replicating-injections/

The Neuron. (2026, September 30). Everything that happened in AI today (Wednesday, September 30, 2026). https://www.theneuron.ai/digest/everything-that-happened-in-ai-today-wednesday-september-30-2026/

The Register. (2026, September 28). OpenAI pauses some training amid allegations its rogue agents behaved more badly than first thought. https://www.theregister.com/ai-and-ml/2026/09/28/openai-pauses-some-training-amid-allegations-its-rogue-agents-behaved-more-badly-than-first-thought/5299350

The Register. (2026, September 29). Add one more AI worry to the nightmare scenario: self-replicating prompt injections. https://www.theregister.com/security/2026/09/29/add-one-more-ai-worry-to-the-nightmare-scenario-self-replicating-prompt-injections/5299922

The Register. (2026, September 29). AI models keep posting screenshots showing sensitive data from inside tech companies. https://www.theregister.com/ai-and-ml/2026/09/29/ai-models-keep-posting-screenshots-showing-sensitive-data-from-inside-tech-companies/5299640

The Register. (2026, September 29). OpenAI benches GPT-6.1 Astra for overstepping the mark. https://www.theregister.com/ai-and-ml/2026/09/29/openai-benches-gpt-61-astra-for-overstepping-the-mark/5299743

U.S. Senate Committee on Homeland Security and Governmental Affairs. (2026, September 30). Rogue AI: Securing the homeland against AI agent attacks [Subcommittee hearing]. https://www.hsgac.senate.gov/subcommittees/dmdcc/hearings/rogue-ai-securing-the-homeland-against-ai-agent-attacks/

Venture Atlas. (2026, September 29). Anthropic says open-weight GLM-5.3 can build end-to-end exploits. https://www.ventureatlas.org/news/2026-09-29-anthropic-glm-5-3-cyber

The White House. (2026, September 29). Fact sheet: President Donald J. Trump inaugurates the era of Super Intelligence. https://www.whitehouse.gov/fact-sheets/2026/09/fact-sheet-president-donald-j-trump-inaugurates-the-era-of-super-intelligence/