Weekly Musings Top 10 AI Security Wrapup: Issue 18 October 24, 2025 - October 30, 2025
AI Agents, Memory Poisoning, and Deepfake Disasters Dominate This Week's Headlines
You spend months hardening your network. Your team finally gets phishing training completion rates above 80%. Zero trust is no longer just a buzzword in your architecture docs. Then someone asks ChatGPT to write some code, and a backdoor appears in production.
This week showed us what happens when the tools we built to boost productivity become the targets. AI coding assistants that generate vulnerable code at scale. Agent identity platforms are raising $38 million because nobody knows how to authenticate something that exists for three seconds. Browser agents are so eager to help that they’ll execute any command disguised as a URL. Deepfakes are flooding social media during a real hurricane while an Irish presidential candidate watches her AI twin announce a fake campaign withdrawal.
We’re not only securing AI anymore. We’re securing everything through AI, and the difference matters. The attack surface is fundamentally changing shape. Your CISO job description just got rewritten, whether you wanted it to or not.
1. AI-Generated Code Creates More Problems Than It Solves
Aikido Security’s survey of 450 professionals across the US and Europe confirms what your security team already suspected but couldn’t prove (Help Net Security, 2025a). About 25% of production code now gets written with AI tools. Most organizations found vulnerabilities tied to this code. Some experienced incidents. The math is simple but brutal.
The report shows higher AI adoption in the US than Europe, but vulnerability rates stay consistent across regions. Organizations increased security budgets by 33% in 2025 compared to just 7% the previous year. The money follows the problem. When AI writes code, security teams clean up the mess.
Most organizations now use AI to write production code, and many have seen new vulnerabilities appear because of it (Help Net Security, 2025a). About a quarter of production code uses AI tools, but security guardrails have not caught up. The productivity gain is real. So is the security debt.
Why It Matters
Accountability gaps widen: Over half of respondents blame the security team when AI-generated code causes a breach, while many point to developers who produced or merged the code (Help Net Security, 2025a). Nobody knows who owns the risk.
Speed without oversight: AI writes faster than humans can review. The volume of code increased by 75% since 2022, but human oversight capacity remained flat (Dark Reading, 2025). More code, more problems, same headcount.
Vulnerability patterns replicate: AI tends to amplify flaws in the codebases it uses for training and produces a greater volume of code (Dark Reading, 2025). The same bugs appear across multiple projects because AI learned them from public repositories.
What To Do About It
Implement pre-merge security scanning: Automate vulnerability detection before AI-generated code reaches production. Catch issues at pull request time, not in production.
Define clear ownership policies: Establish who bears responsibility when AI-generated code causes incidents. Make the accountability chain explicit in your development standards.
Track AI tool usage separately: Monitor which code comes from AI assistants versus human developers. Measure vulnerability rates by source to identify patterns.
Rock’s Musings
25% of production code written by AI. That number will double within a year. The vendors selling these tools talk about productivity gains. Fair enough. But nobody mentions that security teams now review code that wasn’t written by anyone who understands the system architecture or business logic.
The problem isn’t that AI writes bad code. It’s that AI writes code at a scale and speed that breaks every security review process we built for human developers (See “Vibe Coding’s Hidden Bill: Security, Maintainability, and the NIST Playbook”) . Your CI/CD pipeline was designed for humans making a few dozen commits per day. Now you’re getting hundreds of AI-generated changes, each one a potential security issue that your team can’t possibly review thoroughly.
The accountability question is the real nightmare. When a developer writes vulnerable code, you know who to train, coach, or manage out. When AI writes it, who gets the talk? The developer who accepted the suggestion? The security team that approved the tool? The vendor who trained the model on Stack Overflow’s greatest hits? Good luck with that performance review.
2. Keycard Raises $38M to Solve the AI Agent Identity Crisis
Keycard emerged from stealth on October 21 with $38 million in funding and a platform for managing AI agent identities (Keycard, 2025). The startup tackles the challenge that AI agents need identity and access controls that don’t exist in current systems.
Traditional IAM was built for humans clicking buttons. Agents are ephemeral, coming into existence and disappearing in seconds and at tremendous scale (Keycard, 2025). They work across systems and companies. They need federated identity without manual configuration. Static role-based access controls can’t provide the guarantees required for trusted agent operations.
The $8 million seed round was co-led by Andreessen Horowitz and Boldstart Ventures. Acrew Capital led the $30 million Series A (SiliconANGLE, 2025). The company’s platform identifies AI agents, assigns task-based permissions, and tracks activity. Organizations can deploy AI agents into production with complete trust, knowing they are only capable of performing the intended actions of their users and builders (Keycard, 2025).
Why It Matters
Agent scale breaks traditional IAM: Static credentials and long-lived API keys can’t secure thousands of ephemeral agents that exist for seconds. Current systems weren’t designed for this workload (Help Net Security, 2025b).
Trust requires new primitives: Keycard cryptographically proves who an agent is, who they act for and whether they are authorized (Keycard, 2025). Without deterministic guarantees, agents stay in labs.
Market timing signals urgency: CyberArk’s 2025 Identity Security Landscape predicts AI will drive the greatest number of new identities with privileged access (Help Net Security, 2025b). The infrastructure needs to exist before agents go to production.
What To Do About It
Audit current agent deployments: Map which AI agents have access to what systems and data. Most organizations don’t know the answer, which means they can’t secure it.
Implement dynamic credential management: Replace static API keys with identity-bound, task-scoped tokens that expire quickly. Secrets that live forever get stolen forever.
Establish agent governance policies: Define who can authorize agents, what permissions they can request, and how access gets revoked. Set the rules before the chaos.
Rock’s Musings
Keycard raised $38 million to solve a problem that most security teams don’t yet think they have. That’s either prescient or premature. I lean toward prescient. The agent economy is coming whether we’re ready or not. Every major AI vendor is building agents that can take actions on your behalf. That means they need access to your systems, your data, and your credentials.
Think about what happens when an agent needs to act for you. It needs to prove it’s authorized. It needs scoped permissions for the specific task. It needs those permissions to vanish when the task completes. Now multiply that by thousands of agents doing thousands of tasks per hour across dozens of systems. Your current IAM stack can’t handle it.
The interesting part is Keycard’s focus on federated identity for agents. They’re not trying to create a walled garden where only their agents can operate. They’re building infrastructure that works with Anthropic’s agents, Microsoft’s agents, OpenAI’s agents. That’s the right approach, because the last thing we need is five different agent identity systems that don’t interoperate. We already solved that problem for humans with SAML and OAuth. Now we need the agent version.
3. ChatGPT Atlas Browser Vulnerable to Memory Poisoning Attacks
LayerX Security disclosed a critical vulnerability in OpenAI’s ChatGPT Atlas browser on October 27 (LayerX Security, 2025). The flaw allows attackers to inject malicious instructions into ChatGPT’s memory using cross-site request forgery techniques. Once injected, these instructions persist across devices and sessions.
The attack leverages a cross-site request forgery flaw that could be exploited to inject malicious instructions into ChatGPT’s persistent memory (The Hacker News, 2025a). The corrupted memory can persist across devices and sessions, permitting an attacker to conduct various actions when a logged-in user attempts to use ChatGPT.
LayerX tested Atlas against 103 in-the-wild web vulnerabilities and phishing attacks. Atlas stopped only 5.8% of malicious web pages (The Hacker News, 2025a), compared to Edge at 53% and Chrome at 47%. Users of Atlas are approximately 90% more vulnerable to phishing attacks than users of traditional browsers (LayerX Security, 2025).
The vulnerability particularly threatens developers using “vibe coding” approaches. Poisoned memory instructions can cause ChatGPT to generate code with hidden malicious functionality, such as fetching and executing remote code from attacker-controlled servers with elevated privileges (CyberPress, 2025).
Why It Matters
Memory persistence amplifies impact: Unlike session-based attacks that require re-exploitation, memory poisoning affects every future interaction. One successful injection compromises all subsequent ChatGPT usage (The Register, 2025a).
AI browsers lack basic protections: Testing revealed AI browsers have significantly weaker phishing protection than traditional browsers (LayerX Security, 2025). The rush to ship AI features left security fundamentals behind.
Development workflows at risk: Developers trust ChatGPT to generate code. When that trust gets exploited through memory poisoning, malicious code enters production under the guise of legitimate AI assistance (CyberPress, 2025).
What To Do About It
Disable AI browser memory features: Until OpenAI patches the vulnerability, turn off memory persistence in ChatGPT settings. Accept reduced convenience for improved security.
Review AI-generated code carefully: Never blindly accept code from AI assistants, especially in security-sensitive contexts. Treat all AI output as untrusted input requiring human verification.
Use traditional browsers for sensitive work: Keep financial, administrative, and credential management activities in hardened browsers with proven security track records.
Rock’s Musings
OpenAI launched Atlas on a Tuesday. Researchers found a memory poisoning vulnerability almost instantly. That’s not a great week for the Atlas team. The vulnerability itself is clever but not shocking. CSRF attacks are web security 101. The shocking part is that OpenAI shipped an AI browser that stops fewer than 6% of phishing attacks.
Let me be clear about what this means. You’re using a browser that has direct access to an AI assistant with memory across all your devices. Someone tricks you into clicking a malicious link. That link poisons ChatGPT’s memory with instructions to inject backdoors into every piece of code it generates for you. You don’t notice because the code looks legitimate. The AI even provides helpful comments explaining what the code does, but the comments lie about the malicious functionality.
The proof-of-concept targeting vibe coding is particularly nasty. Developers are already nervous about AI-generated code. Now they have to worry that the AI itself has been compromised before it even generates the code. That’s a supply chain attack where the supplier is the AI in your browser that you trusted to help you write software.
4. High-Quality Deepfake Targets Irish Presidential Election
A sophisticated deepfake video of Irish presidential candidate Catherine Connolly circulated on social media on October 27, falsely showing her announcing her withdrawal from the race (RTÉ, 2025). The video appeared during the final TV debate of the campaign, featuring realistic audio and video embedded in a fake RTÉ News broadcast.
Even experts in the field of AI deepfakes were taken aback by the video's high quality, sparking fresh concerns about the threat to democracy that the technology now poses (RTÉ, 2025). Aidan O’Brien, a researcher with the European Digital Media Observatory at Dublin City University, believes the video was of much higher quality compared to previous deepfakes.
An Coimisiún Toghcháin, the Electoral Commission, immediately escalated the issue to platforms, requesting an urgent review. Meta removed the video from Facebook within hours, but by then it had been shared widely. The video remained available on X, where the platform labeled posts as “manipulated media” in accordance with its policies (RTÉ, 2025).
Once these things break containment, once they’ve been posted online and shared you’ll never really be able to pull them back out (RTÉ, 2025). The video is out of the Pandora’s box and we’re going to have to live with it now.
Why It Matters
Quality threshold crossed: Previous deepfakes contained artifacts that gave them away. This video fooled experts, indicating the technology has reached a new capability level (RTÉ, 2025).
Platform response insufficient: Even with rapid takedown, the content spreads faster than moderation can act. Once viral, removal becomes damage control rather than prevention (RTÉ, 2025).
Electoral integrity threatened: Mainstream AI models don’t allow you to make deepfakes of known people at least without their consent permission (RTÉ, 2025). Guardrails were clearly breached, raising questions about enforcement.
What To Do About It
Implement out-of-band verification: For high-stakes communications, verify through separate channels. If a candidate announces withdrawal, confirm through official campaign channels before acting.
Train staff on deepfake indicators: While quality is improving, subtle artifacts often remain. Teach teams to look for inconsistencies in lighting, audio synchronization, and unnatural movements.
Establish rapid response protocols: When deepfakes target your organization or executives, have pre-approved messaging and verification procedures ready. Speed matters more than perfection.
Rock’s Musings
Catherine Connolly had to debate while her AI twin told voters she was quitting the race. That’s the kind of thing that sounds like science fiction until it happens to you. The quality of this deepfake is what worries me most. Not the technology itself—we knew that was coming—but the fact that it fooled researchers who study this for a living.
What’s the defensive move here? You can’t fact-check every video in real time during an election. By the time the fact-checkers weigh in, the damage is done. Meta took it down within hours, but millions saw it first. X left it up with a label that many users probably ignored. The perfect is the enemy of good, except in this case neither perfect nor good is achievable.
The guardrail breach bothers me more than the deepfake itself. Someone used mainstream AI tools to create a convincing fake of a known political figure. That shouldn’t be possible according to every vendor’s responsible use policy. But it happened anyway. Which means either the guardrails don’t work, or someone found a way around them that others will copy.
5. OpenAI’s Sora Floods Social Media With Hurricane Deepfakes
Hurricane Melissa struck Jamaica on October 28 as a Category 5 storm. While the real hurricane caused at least seven deaths, AI-generated fake videos depicting fabricated disaster scenes went viral on social media (Grand Pinnacle Tribune, 2025). OpenAI’s Sora 2 tool enabled easy creation of convincing synthetic videos during the crisis.
The earliest version of the viral hurricane-eye video appeared on TikTok on October 26, carrying a caption admitting it was AI-generated. The account’s bio described itself as “AI disaster curiosity”. However, as the videos spread, context disappeared. TikTok removed over two dozen clips and multiple accounts after they were flagged.
The fake content undermines the seriousness of the government's message to be prepared (Grand Pinnacle Tribune, 2025). Amy McGovern, a meteorology professor at the University of Oklahoma, warned that AI-generated misinformation can overshadow critical safety warnings.
Why It Matters
Crisis response gets compromised: When fake disaster content floods social media during real emergencies, people can’t distinguish warnings from misinformation. That delay kills.
Context evaporates quickly: Original posts may disclose AI generation, but viral spread strips that context away. Most viewers see the content without the disclaimer.
Regulatory gaps remain: TikTok’s guidelines require AI-generated content to be labeled, but enforcement occurs after it spreads. Reactive moderation can’t solve proactive manipulation.
What To Do About It
Establish trusted information sources: During crises, designate specific accounts or channels as authoritative. Train teams and stakeholders to verify against these sources only.
Implement content provenance systems: Adopt digital signatures or cryptographic proofs that verify content origin and authenticity. Technical solutions beat policy hope.
Monitor for AI-generated content actively: Use detection tools to flag synthetic media in real time, especially during emergencies when impact is highest.
Rock’s Musings
Hurricane Melissa killed seven people. Hurricane Melissa (the AI version) probably killed zero people but definitely confused millions. That’s the new reality. Every natural disaster now comes with a synthetic disaster overlay that makes it harder to tell people what’s actually happening.
The Sora videos look convincing. That’s the point. OpenAI built a tool that can generate realistic video from text prompts. People used it to create fake hurricane footage because they could. Some disclosed it was AI. Most didn’t. By the time TikTok started removing content, the videos had already gone viral.
We’re still in the early days of this technology. The quality will improve. The speed will increase. The cost will drop. In a year, anyone with a phone will be able to generate photorealistic video of disasters that never happened, political statements that were never made, or crisis announcements from officials who never spoke. And we still haven’t figured out how to verify anything at scale.
6. Microsoft Patches Actively Exploited WSUS Vulnerability
Microsoft released an emergency fix on October 23 for CVE-2025-59287, a vulnerability in Windows Server Update Services actively exploited in the wild. CISA added the flaw to its Known Exploited Vulnerabilities catalog on October 25, directing federal agencies to mitigate it by November 14 (Help Net Security, 2025c).
Eye Security identified and shared with NCSC-NL the first successful exploit attempts of the vulnerability on Friday morning. The company noted the attack was more sophisticated than proof-of-concept exploits, suggesting state actor or advanced ransomware gang capabilities.
Huntress detected attacks beginning around October 23, performing reconnaissance by identifying logged-in users and listing all users. Four of their customers were hit, though they expect exploitation to remain limited since WSUS doesn’t often expose ports 8530 and 8531.
Eye Security identified approximately 8,000 internet-facing servers with vulnerable ports open, though they couldn’t verify patch status. Considering that the emergency fix was pushed out less than two days ago, it’s likely that not many have been patched yet.
Why It Matters
Active exploitation before patch awareness: Attackers weaponized the vulnerability before most organizations knew it existed. Zero-day response windows keep shrinking.
Critical infrastructure at risk: WSUS manages updates for Windows environments. Compromising update infrastructure lets attackers control which patches get deployed—or inject malicious updates.
State actor capabilities indicated: The sophistication of observed attacks suggests well-resourced threat actors, not script kiddies (Help Net Security, 2025c). This wasn’t opportunistic; it was targeted.
What To Do About It
Patch WSUS immediately: This isn’t next-week territory. Emergency patches merit emergency response. If you run WSUS, patch today.
Audit exposed update infrastructure: Verify which update servers are internet-facing. WSUS should never be directly accessible from untrusted networks.
Implement network segmentation: Isolate update infrastructure from general network access. Apply zero trust principles to systems that manage security updates.
Rock’s Musings
An actively exploited WSUS vulnerability is basically a nightmare scenario wrapped in an emergency patch. WSUS is how you keep Windows systems updated. If attackers control your WSUS server, they control what patches you deploy. Or they can inject malicious updates disguised as security fixes. That’s supply chain compromise at the most fundamental level.
Eye Security flagged this as sophisticated enough to be a state actor or advanced ransomware gang. I believe them. This isn’t the kind of thing script kiddies stumble into. Someone spent time finding this vulnerability, developing a working exploit, and deploying it before Microsoft even had a chance to patch.
Eight thousand internet-facing WSUS servers. That number is too high. WSUS should never be directly internet-accessible. But here we are, with thousands of organizations running update infrastructure that can be reached from anywhere. The attack surface keeps expanding because we keep putting things online that were designed for internal networks.
7. Google Unveils CodeMender for Automated Vulnerability Patching
Not this week, but it came across my feed. On October 6, Google DeepMind introduced CodeMender, an AI-powered agent that automatically fixes code security vulnerabilities (Google DeepMind, 2025). The system leverages Gemini Deep Think models to debug, flag, and fix vulnerabilities by addressing root causes and validating changes.
Over the past six months they’ve been building CodeMender, they’ve already upstreamed 72 security fixes to open source projects, including some as large as 4.5 million lines of code. The agent uses robust tools that let it reason about code before making changes and automatically validate those changes.
CodeMender operates in both reactive and proactive modes. It instantly patches newly discovered vulnerabilities and rewrites existing code to eliminate entire classes of vulnerabilities. While large language models are rapidly improving, mistakes in code security could be costly. CodeMender’s automatic validation process ensures code changes are correct across many dimensions.
The system uses a debugger, source code browser, and other tools to pinpoint root causes and devise patches. It only surfaces high-quality patches for human review that fix the root cause, are functionally correct, cause no regressions, and follow style guidelines.
Why It Matters
AI finds vulnerabilities faster than humans can fix them: As AI-powered vulnerability discovery improves, human developers can’t keep pace. CodeMender helps close that gap.
Open-source security improves: 72 patches to open-source projects in 6 months mean critical infrastructure becomes more secure. That benefits everyone using those libraries.
Validation prevents new bugs: Automated patching without validation creates chaos. CodeMender’s multi-dimensional verification ensures fixes don’t introduce regressions.
What To Do About It
Monitor Google’s open source security efforts: As CodeMender patches reach open source projects you depend on, track which libraries receive security improvements.
Evaluate AI-assisted security tooling: CodeMender demonstrates what’s possible. Assess whether similar tools could help your security team scale vulnerability remediation.
Maintain human oversight: Even validated patches need review. Don’t let automation become auto-approval without security team verification.
Rock’s Musings
Google is using AI to fix vulnerabilities in open source code at scale. That’s either brilliant or terrifying, depending on your perspective. I’m leaning toward brilliant with caveats. The validation approach is solid. CodeMender generates patches, verifies they work, don’t break anything, and follow code style guidelines. That’s more thorough than some human code reviews I’ve seen.
The reactive plus proactive approach makes sense. React to new vulnerabilities immediately, and proactively hunt for entire classes of bugs to eliminate them. If CodeMender can systematically remove buffer overflows or use-after-free bugs from critical libraries, that’s a real security improvement for everyone downstream.
My concern is that we’re automating vulnerability discovery and automated patching, and that’s a race where both sides use AI. Attackers use AI to find bugs. Defenders use AI to patch them. All of it is probabilistic and non-deterministic while we’re still figuring out how to make sure the AI actually fixes the problem instead of creating new ones.
8. Deepfake Detection Lag Creates Vulnerability Gap
The market for AI detection tools grows at 28-42% compound annual rate, but the threat expands at rates of 900% or 1,740% in key regions. This creates a massive vulnerability gap. The effectiveness of defensive tools plummets by 45-50% when taken out of controlled lab conditions and used against real-world deepfakes (DeepStrike, 2025).
The cryptocurrency sector became ground zero for deepfake fraud, accounting for 88% of all detected deepfake fraud cases in 2023. The fintech industry saw a 700% increase in deepfake incidents the same year. Gartner predicts that by 2026, 30% of enterprises will no longer consider standalone identity verification and authentication solutions reliable in isolation.
While financial fraud has the largest economic impact, the vast majority of deepfake content by volume is non-consensual intimate imagery. The rapid growth of malicious deepfake creation is dangerously outpacing our ability to defend against it.
Detection effectiveness drops dramatically outside lab conditions. Tools that work well in controlled testing fail against real-world attacks. The technology arms race favors offense, not defense.
Why It Matters
Detection tools can’t keep pace: The gap between threat growth and defensive capability widens every quarter. Technology alone can’t solve this problem.
Financial sectors most exposed: Cryptocurrency and fintech see the highest fraud rates. If your organization handles financial transactions, deepfake fraud is a when, not if.
Procedural resilience matters more: With detection failing, verification processes must withstand technically perfect deepfakes. Build procedures that don’t rely on “detecting” fakes.
What To Do About It
Implement multi-factor verification: Don’t trust video or voice alone. Require out-of-band confirmation for high-value transactions or sensitive changes.
Train staff on verification protocols: Technical detection fails. Human verification protocols must become second nature. Practice them until they’re automatic.
Accept that perfect detection is impossible: Stop chasing detection tools that promise 99% accuracy. Build resilience assuming some deepfakes will always get through.
Rock’s Musings
The vulnerability gap is growing, not shrinking. Detection tools improve, but deepfakes improve faster. Do the math on a 900% threat growth rate versus 40% defensive growth rate. You don’t need a PhD to see how that ends.
Gartner saying that identity verification won’t be reliable by 2026 is a polite way of saying we’ve lost the detection battle. The question isn’t whether your authentication can detect deepfakes. It’s whether your authentication can work when perfect deepfakes are the norm. That requires rethinking every process that relies on “I can verify this is really you by looking at you or hearing you.”
The cryptocurrency sector getting hit hardest makes sense. Fast transactions, irreversible, often involving significant sums. That’s the perfect target for deepfake fraud, but every industry will follow the same pattern. Healthcare, legal, government, education, and anywhere that identity verification matters, deepfakes create problems we haven’t figured out how to solve at scale.
9. AI Phishing Surges as Attacks Become More Believable
Kaspersky reported a 3.3% increase in phishing between Q1 and Q2 2025, driven by AI-enhanced attacks (UNC Chapel Hill, 2025). AI has flipped the script on phishing. From tailored phishing emails to realistic deepfake audio and video, AI helps bad actors launch more attacks with higher success rates.
UNC-Chapel Hill faced a rise in “CEO fraud” attacks where scammers review organizational charts, note executive names, and create fake email addresses with name variations. AI’s ability to polish messaging, copywriting styles, mimic voices, and recreate faces makes the threat even more dangerous.
AI-powered voice cloning can recreate anyone’s speech with just a few seconds of audio. Combined with deepfake video, attackers create convincing impersonations of executives requesting fund transfers or sensitive information.
Detecting AI-generated phishing is extremely challenging, as AI makes every phishing attempt more convincing, easier to create, super personalized, and easier to scale. Traditional security awareness training assumes humans can spot phishing indicators. AI-enhanced phishing removes those indicators.
Why It Matters
Human detection fails: Training employees to spot phishing worked when phishing was obvious. AI-generated attacks pass the smell test. Your people can’t reliably detect them.
Scale increases exponentially: AI enables attackers to launch thousands of personalized phishing campaigns simultaneously.
Voice and video compromise trust: When attackers can convincingly impersonate executives via audio or video, existing verification procedures break down.
What To Do About It
Implement out-of-band verification for financial requests: When an executive requests a wire transfer, verify through a separate communication channel using a pre-established method.
Update security awareness training: Stop teaching people to spot bad grammar or suspicious links. Train them on verification procedures that work even when the message looks perfect.
Deploy technical controls that don’t rely on human judgment: Email authentication, transaction limits, and multi-approval workflows reduce reliance on spotting fakes.
Rock’s Musings
Phishing emails used to be easy to spot. Bad grammar, weird formatting, urgency about Nigerian princes. Not anymore. AI writes better business English than most of your executives. It knows your company’s communication style because it scraped your public materials. It can impersonate your CEO’s voice using three seconds of audio from an earnings call.
The CEO fraud pattern is particularly nasty. Someone looks up your org chart on LinkedIn, creates a Gmail account with your CFO’s name spelled slightly differently, and sends your AP team an urgent wire transfer request. Before AI, those emails were obvious fakes. Now they’re perfectly written, use appropriate terminology, and arrive at plausible times. Your AP team can’t tell the difference without verification procedures that treat every request as suspicious.
Voice cloning is where this gets truly problematic. You get a call that sounds exactly like your CEO. The caller ID shows their number (spoofed). They need an urgent wire transfer. Everything about the call sounds legitimate because AI cloned their voice perfectly. What’s the defensive move? You can’t train people to detect fake voices that sound perfect. You need procedures that assume any voice could be fake.
10. Zero Trust Has a Blind Spot: Your AI Agents
Organizations spent years implementing zero trust for users and applications. Then AI agents arrived, and zero trust didn’t have an answer. Traditional zero-trust frameworks assume all identities are users or services with predictable behavior patterns and persistent identities.
AI agents break these assumptions. They’re ephemeral, created and destroyed in seconds. They act on behalf of users but make autonomous decisions. They access multiple systems within a single task. They don’t have persistent identities that fit traditional role-based access control models.
Current zero-trust implementations can’t handle agents that exist for 3 seconds, require elevated privileges for specific tasks, span multiple trust boundaries, and make real-time access decisions based on context. The blind spot is a gap in coverage due to a fundamental architectural mismatch.
Organizations are deploying AI agents without security frameworks that account for their behavior. Zero-trust principles still apply, but implementing them requires new primitives that don’t exist in most security architectures. Until they do, AI agents operate outside the security models organizations spent years building.
Why It Matters
Existing security frameworks don’t apply: Zero trust was designed for persistent identities. Agents that live for seconds don’t fit the model. You can’t apply principles that assume the identity will exist long enough to audit.
Privilege escalation by design: Agents need elevated privileges to act on users’ behalf. Traditional privilege management can’t handle dynamically scoped permissions that change per task.
Audit trails become meaningless: When thousands of ephemeral agents act per hour, traditional audit logs can’t track who did what. Attribution breaks down.
What To Do About It
Map agent access requirements now: Identify which systems AI agents need to access and what privileges they require. You can’t secure what you haven’t inventoried.
Implement task-based authorization: Replace role-based access control with task-scoped permissions that expire when the task completes. Static roles don’t work for dynamic agents.
Require cryptographic agent identity: Agents must prove identity cryptographically, tied to device or workload identity. Traditional credentials don’t work at agent scale.
Rock’s Musings
You spent three years implementing zero trust. Your board finally stopped asking about it. Then someone deploys AI agents that bypass every control you built. That’s where we are in 2025.
Zero trust was designed for a world where identities persist long enough to verify, monitor, and audit. Users log in, do things, log out. Services start, run, stop. Agents don’t follow that pattern. They appear, do something, and vanish. By the time your SIEM processes the logs, the agent is gone. Traditional audit trails can’t attribute actions to anything meaningful.
The privilege escalation problem is worse. Agents need access to act on your behalf. If an agent books a flight for me, it needs access to my credit card and calendar. That’s appropriate since I authorized it, but how do you prevent that same agent from accessing my email or my company’s financial system? Traditional RBAC gives roles like “calendar_access” or “expense_submitter.” Agents need permissions like “book_this_specific_flight_right_now” that vanish after execution. Before you say it… OAuth isn’t the long-term answer.
The One Thing You Won’t Hear About But You Need To: AI Model Memory Is the New Attack Surface
While everyone focuses on prompt injection and data poisoning, a quieter threat emerged this week. AI model memory as a persistent attack surface. ChatGPT Atlas’s memory poisoning vulnerability isn’t just a bug in one browser. It’s the first major exploit of a pattern that will dominate AI security for the next two years.
AI models with memory create persistent state across sessions, devices, and contexts. That memory makes them more useful. They remember your preferences, your projects, your patterns. Persistent memory also creates persistent attack surfaces. Compromise the memory once, and every future interaction uses that poisoned state.
Traditional security assumes stateless interactions. Each request is independent. Each transaction is verified. AI memory breaks that assumption. Once malicious instructions enter memory, they persist invisibly until explicitly removed. Users can’t inspect memory to verify it’s clean. They can’t diff current memory against known-good states. The memory just exists, and the AI trusts it.
This matters beyond ChatGPT. Every AI assistant with memory creates this attack surface. Copilot, Gemini, Claude… they all implement persistent memory to improve user experience. They all become potential vectors for memory poisoning. The attack succeeds once and persists forever, affecting every subsequent interaction until detected and removed.
Why You Need To Know This
Memory compromise is silent: Users don’t see AI memory directly. Attacks succeed without visible indicators until malicious behavior appears in outputs.
Cross-session persistence amplifies impact: Traditional attacks require re-exploitation for each session. Memory attacks succeed once and persist indefinitely.
No detection mechanisms exist: Current security tools don’t monitor AI memory for integrity. The attack surface exists without defensive coverage.
What To Do
Disable AI memory features: In a perfect world, until vendors implement memory integrity verification, turn off persistent memory in AI assistants. Accept reduced convenience. But… let’s be realistic. You probably won’t. I probably won’t.
Treat AI outputs as untrusted: Never blindly trust AI-generated content, especially code or commands. Verify everything through independent review.
Monitor for AI behavioral changes: Sudden shifts in AI assistant tone, suggestions, or outputs may indicate memory compromise. Investigate anomalies immediately.
👉 What do you think? Ping me with the story that keeps you up at night, or the one you think I overrated.
👉 The Wrap-Up drops every Friday. Stay safe, stay skeptical.
👉 For deeper dives, visit RockCyber.
Citations
Abnormal AI. (2025, September 1). Anthropic report: Vibe hacking and AI-enabled threats. https://abnormal.ai/blog/vibe-hacking-ai-enabled-threats-anthropic-report
Anthropic. (2025, August). Threat intelligence report: August 2025. https://www-cdn.anthropic.com/b2a76c6f6992465c09a6f2fce282f6c0cea8c200.pdf
Cloud Security Alliance. (2025, July 9). Understanding security risks in AI-generated code. https://cloudsecurityalliance.org/blog/2025/07/09/understanding-security-risks-in-ai-generated-code
CyberPress. (2025, October 27). Hackers exploit Atlas browser vulnerability to inject malicious code into ChatGPT. https://cyberpress.org/hackers-exploit-atlas-browser-vulnerability/
Dark Reading. (2025, October 29). AI-generated code poses security, bloat challenges. https://www.darkreading.com/application-security/ai-generated-code-leading-expanded-technical-security-debt
DeepStrike. (2025, September 8). Deepfake statistics 2025: The data behind the AI fraud wave. https://deepstrike.io/blog/deepfake-statistics-2025
Google DeepMind. (2025, October 6). Introducing CodeMender: An AI agent for code security. https://deepmind.google/discover/blog/introducing-codemender-an-ai-agent-for-code-security/
Grand Pinnacle Tribune. (2025). AI deepfakes flood social media during Hurricane Melissa. https://evrimagaci.org/gpt/ai-deepfakes-flood-social-media-during-hurricane-melissa-514302
Help Net Security. (2025a, October 24). When AI writes code, humans clean up the mess. https://www.helpnetsecurity.com/2025/10/24/ai-written-software-security-report/
Help Net Security. (2025b, October 22). Keycard emerges from stealth with identity and access solution for AI agents. https://www.helpnetsecurity.com/2025/10/22/keycard-ai-agents-identity-access-platform/
Help Net Security. (2025c, October 24). Microsoft releases urgent fix for actively exploited WSUS vulnerability (CVE-2025-59287). https://www.helpnetsecurity.com/2025/10/24/wsus-vulnerability-cve-2025-59287-exploited/
Keycard. (2025, October 21). Keycard is identity infrastructure for the agent-native world. https://www.keycard.sh/announcement
LayerX Security. (2025, October 27). “ChatGPT tainted memories:” LayerX discovers the first vulnerability in OpenAI Atlas browser, allowing injection of malicious instructions into ChatGPT. https://layerxsecurity.com/blog/layerx-identifies-vulnerability-in-new-chatgpt-atlas-browser/
RTÉ. (2025, October 27). Fresh AI concerns after quality of election deepfake. https://www.rte.ie/news/analysis-and-comment/2025/1026/1540488-deep-fake-concerns/
SiliconANGLE. (2025, October 21). AI agent security startup Keycard reels in $38M. https://siliconangle.com/2025/10/21/ai-agent-security-startup-keycard-reels-38m/
The Hacker News. (2025a, October 27). New ChatGPT Atlas browser exploit lets attackers plant persistent hidden commands. https://thehackernews.com/2025/10/new-chatgpt-atlas-browser-exploit-lets.html
The Register. (2025a, October 27). Atlas vuln allows malicious memory injection into ChatGPT. https://www.theregister.com/2025/10/27/atlas_vulnerability_memory_injection/
UNC Chapel Hill. (2025, October 20). AI deepfakes put new spin on old cyber threat. https://its.unc.edu/2025/10/20/ai-deepfakes-cyber-threat/