Claude Code Skills: Put The Discipline In The File

Stop hoarding prompts. RockCyber's open-source Claude Code skills catch the ML, security, and reproducibility failures AI ships confidently.

Claude Code skills are how I stopped re-explaining the same discipline to an AI on every session. Last year Veracode tested code from more than 100 large language models and found security flaws in 45% of it. That matches what I keep seeing in my machine learning and data science work, where the model returns an answer that looks correct and is wrong underneath. This week, I put my skills library on GitHub at rocklambros/RCS. It's the portable half of a setup I've been building in the open. The skills drop into the Claude Code harness I documented at rocklambros/harness-engineering, and they ride alongside the secure coding rules I open-sourced to stop Claude generating vulnerable code in the first place. Here is what is inside, starting with the skill that saved me the most pain.

Figure 1: AI-Generated Code Fails Security Tests in 45% of Tasks

The Prompt You Keep Pasting Is A Skill

I spent more time than I should have just copying the same prompt into Claude from a cacophony of Notion notebooks and saved text files. I can tell you how many times I told Claude to stop trusting a clean p-value, to check the assumptions behind a statistical test before reporting it, and to refuse the easy answer when the data didn’t support it. I pasted some version of that into a fresh chat a few hundred times. Every session started from zero. The model holds no memory of the discipline I taught it yesterday, so I taught it again, and the wording drifted a little each time.

That is the tell. If you keep pasting the same instructions, you have already written something. You never saved it in a form the tool can load on its own. A Claude Code skill is that saved form, a single markdown file with a description that the model reads to decide whether the file applies to your question. Once it is installed, the model consults it without being asked. The discipline lives in the file instead of your head, and your head is where discipline goes to get forgotten.

Here is the example that pushed me over the edge. I was building a multi-turn prompt-injection detector for the Deep Learning course in my Master’s program, and I had two models to compare on the same set of 5,130 test conversations. The temporal LSTM scored an F1 of 0.837. Adding an attention layer on top scored 0.837 as well, with confidence intervals nearly overlapping. Ask an AI whether the attention version is the better model, and most will wave it through as a free upgrade, since the interpretability costs nothing. The honest answer is that there is no improvement to claim. A paired bootstrap on the same conversations yields a difference of zero (p = 0.453), nowhere near significant. A p-value is the chance a gap this size would show up even when the two models are truly identical, and at 0.453 it would show up almost half the time, so there's no real difference to claim. Both models saw the identical test set, so the comparison is paired, and treating it any other way invents a result that is not in the data. That is a confident wrong answer, and it is the kind that ends up on a slide in front of a board (or in a classroom, with a professor sitting as “chairman,” in my case).

The Premortem I Kept Rebuilding By Hand

The skill I want you to remember is running-adversarial-premortem. It came out of that same habit. I kept asking the model to argue against my own designs, and I kept getting agreement dressed up as analysis. I wrote the method down and made it a skill.

A premortem flips the usual review. Rather than asking what could go wrong, it assumes the work will fail in six months and traces the cause. The skill runs in three rounds. Round one assumes failure and lists five to ten ways it happened, seeded by category: the premise was wrong, the method was biased, the code didn’t match the design, the deployment was botched, the audience misread the result. Round two chains each failure back to a root cause. Round three scores what survives.

The scoring is where it earns its keep. Each surviving failure mode gets a severity, a likelihood, and a detectability, and the priority is severity times likelihood divided by detectability. A quiet, high-damage failure that nothing would catch ranks above a loud one that your CI already flags. Two fields force honesty. Every concern carries its strongest counterargument, the most generous defense of the design, so you engage the work rather than strawmanning it. Every concern also names what would have to be true for you to stop worrying, a condition the skill calls “stops mattering if.” A worry with no stopping condition is anxiety wearing a citation, not analysis.

The skill also refuses to fire when it shouldn’t. Ask it to scan a loop for an off-by-one error, and it hands you off to a plain code review, because a premortem on a one-line fix costs more than the bug. That refusal is tested, not promised, and I will come back to why that matters.

You will find it at skills/workflow/running-adversarial-premortem. The method itself is old. Gary Klein wrote it up for Harvard Business Review in 2007. What changed is that the discipline now loads itself when the stakes are high, instead of waiting for me to remember to run it.

Figure 2: The Adversarial Premortem In Three Rounds

The Highest-Priority Skill Is The Most Boring One

Every skill in the library carries a priority score I call Σ, a rough estimate of how often the gap shows up multiplied by how badly things break when you miss it. The top of the list is not the premortem or any of the security skills. It is enforcing-seed-hygiene, at Σ 20, the only skill that hits the maximum. It exists because randomness is everywhere in machine learning, and almost nobody pins it correctly.

Here is the failure... You train a model, get a number, write it down. A colleague runs the same code on the same data and gets a different number. Now neither of you knows which result to trust, and the paper or the production model sits on sand. A 2025 review of machine-learning reproducibility put the fix plainly: fixed random seeds should be set and published to control the many sources of nondeterminism in ML. The same authors flag a catch that trips people up, which is that seeds hold only when the work is not running in parallel on a GPU.

That GPU catch is the part people miss, and the skill handles it. One seed isn’t enough. Python’s own hashing, NumPy, PyTorch, JAX, and R each carry a separate generator, and parallel thread scheduling on a GPU produces floating-point sums that do not land bit-for-bit the same across machines. The skill emits a single first-cell block that seeds every library you named, sets PYTHONHASHSEED, and, for sampler workloads, pins the thread count so a run on your laptop matches the one on the Linux server. It refuses the wrong job too. Ask it to seed a cryptographic nonce, and it stops you, because a fixed seed there is a vulnerability, not a discipline.

It lives at skills/workflow/enforcing-seed-hygiene. That is the profile a Σ 20 earns. It shows up on nearly every project, and it ruins the results when you skip it.

Vet The MCP Server Before You Trust It

The skill I want in front of security people is auditing-mcp-server-pre-trust, at Σ 18. Model Context Protocol servers are how your AI reaches out to tools and data, and the ecosystem grew faster than its security did. A 2025 study from Queen’s University analyzed 1,899 open-source MCP servers and found 7.2% carrying general vulnerabilities and 5.5% exhibiting tool poisoning, where a hostile server hides instructions inside a tool’s description to steer the model without you ever seeing them.

Don’t count on the model to catch it. A separate benchmark, MCPTox, ran tool-poisoning attacks against 20 agents and found they almost never refuse. The best refusal rate from a single Claude model came in under 3%. The model reads the poisoned description and follows it. The paper found that more capable models were often more susceptible, because the attack rides on their stronger instruction-following. That puts the decision where it belongs, with you, before the server gets registered.

The skill runs six checks ahead of that registration: license, source review, network egress, version pin, secret handling, and the subset of tools you need. The version pin alone catches a class of foot-guns. A server installed with npx -y or an unpinned pip install can shift underneath you between the audit and the next run, so the skill treats anything unpinned as a blocking failure. Each check has to cite evidence, a file, a line, or a commit, so the audit cannot decay into a checkbox ritual.

The boundary for a skill like this is design-time and registration-time discipline. It runs inside the model’s reasoning, before the connection goes live. It cannot see what the agent does once it is running, which tools it calls, what data left the building, or whether the action matched the intent you approved. That is a different control layer, and nothing in this repo provides it. Closing it takes interception at the moment a tool executes, structured traces a security team can query rather than vendor-specific log soup, and a live inventory of every tool, model, and dataset the agent touched, the way a software bill of materials tracks dependencies. The skills are the author-time half of a problem, with the other half running in production. Know which half you have covered.

It lives at skills/security/auditing-mcp-server-pre-trust.

Figure 3: Two Halves Of Agent Governance, Author-Time And Runtime

Your Instruction Files Are Rotting

The last skill, auditing-instruction-hierarchy at Σ 18, fixes a problem you can’t see until it bites. The CLAUDE.md files that tell the model how to behave grow without limit, and a larger instruction file is not necessarily better. Chroma’s 2025 “Context Rot” study tested 18 frontier models and found that output quality drops as input length grows, even on simple tasks, well before the context window is anywhere near full. Every line you add to an instruction file competes for the model’s attention with the work in front of it.

My own test harness sets a hard cap of 400 lines across the entire instruction hierarchy, with 250 as the target. Past 400, instruction-following degrades measurably. The skill counts the lines across every CLAUDE.md in play, from the user-level file down to the ones plugins quietly drop in, and it greps for content that breaks the prompt cache: literal dates, session IDs, anything that changes between runs and forces the model to re-read the whole prefix every five minutes. Then it sorts each rule into a verdict. Keep it, move it to a skill that loads on demand, move it to a hook, or drop it.

I ran it against this repo while building it. The skill’s own evidence list names RCS, which is the polite way of saying it found things in my own setup worth trimming. It lives at skills/claude-code-meta/auditing-instruction-hierarchy.

What The Claude Code Skills Repo Ships

The four skills above are a sample. The repo ships 104 of them, all in the shipped state with none in draft, organized into five tracks by who needs them: security, machine learning and data science, cross-cutting workflow, teaching, and Claude Code meta-work. The whole library is MIT licensed.

Figure 4: The Four Featured Skills, By Σ Priority

Two design choices matter when deciding whether to trust it. The first is that it is catalog-free. There are no bundled framework controls and no ISO mirrors copied in to rot the day after I commit them. The skills encode method, not reference material that goes stale. The second is that every skill ships with three test scenarios, a normal case, an edge case, and an anti-trigger that checks the skill stays quiet when it should not fire. That last one is the part most prompt collections skip, and it separates a helpful skill from one that fires on everything and turns into noise.

Install is a clone and a symlink loop that drops each skill into your ~/.claude/skills directory, skipping anything already there. The skills compose themselves. Start an ML notebook and the scaffolding skill, the seed-hygiene skill, and the train-test-split audit fire in sequence without you wiring them together.

Key Takeaway: If you keep pasting the same instructions into an AI, turn them into Claude Code skills, because the discipline you do not write down is the discipline you lose every session.

What to do next

Clone the repo, install the skills, and point the premortem at the next design you are about to commit to. Watch it argue with you. The repo is at github.com/rocklambros/RCS, and the install steps are in the README.

If you want the thinking behind this kind of work, the AI security and governance advisory I run lives at rockcyber.com, and the rest of these teardowns are at rockcybermusings.com. Tell me which skill broke something useful. The anti-trigger tests catch a lot, and you will still find edges I missed.

Subscribe for more AI security and governance insights with the occasional rant.

👉 Subscribe for more AI security and governance insights with the occasional rant.

👉 For ongoing analysis of agentic AI governance frameworks, the conversation continues at RockCyber Musings.

👉 Visit RockCyber.com to learn more about how we can help with your traditional Cybersecurity and AI Security and Governance journey.

👉 Want to save a quick $100K? Check out our AI Governance Tools at AIGovernanceToolkit.com

👉 As a bonus, check out my conversation with Eva Benn where we talked about the cybersecurity skills you need to develop to stay relevant in 2026 and beyond.

The views and opinions expressed in RockCyber Musings are my own and do not represent the positions of my employer or any organization I’m affiliated with.