I Gave My AI Thirty Skills. Turns Out That Wasn't Smart.

Like I do every month, I sat down to tidy up the instruction files I've written for my AI. I pulled up the folder and counted: almost thirty of them, big and small, and I couldn't remember why I'd written more than half.

These files have a technical name — Skills — but really they're just extra instructions you write for it. I wrote most of them because it was convenient at the time, telling myself "more can't hurt." I never went back and checked if that was actually true.

This time I checked, because I ran into two pieces of third-party research, and my face went a little red.

So what exactly is a Skill? I use two AI coding assistants day to day, Claude Code and Codex. They're capable, but they have no idea which folder my files live in, what fixed workflow my accounts follow, or which pitfall I already tripped over and how to route around it. A Skill is just a document that spells out the stuff the AI can't guess on its own — the local paths, the real API protocols, the deterministic steps. Write it once, and it doesn't have to relearn it next time.

They exist because once AI becomes part of your daily workflow, the repetition piles up: explaining the same folder structure over and over, re-teaching the same deploy command every single time. Writing a Skill locks that explanation in for good. The first few times, it genuinely saved a lot of back-and-forth.

Then it became a habit. Every new situation, I'd think "should I write a Skill for this too?" And the Skills folder started looking like most people's phone home screen — installed for convenience, rarely uninstalled, always growing.

What actually made my face burn was two separate evaluations. The first, called SkillsBench, told a good story: a carefully curated set of Skills lifted the average pass rate from 33.9% to 50.5% — nearly a 50% jump. So Skills genuinely help. That part isn't hype.

The second, SWE-Skills-Bench, was less flattering. Out of 49 real-world Skills tested, 39 produced no measurable improvement at all. The average gain across the whole set was just 1.2% — meaning most of what people install is basically dead weight. The sharper sting: three Skills with version mismatches didn't just fail to help, they actively hurt — scores dropped by up to 10%, and token cost per call spiked as high as 451%. A bill four and a half times larger, in exchange for a dumber assistant. Zoom out further and it holds: running retrieval across 34,000 real-world Skills shows the same diminishing returns — without curation and pruning, more Skills just means more noise.

The numbers don't lie: a bigger Skill library isn't a safety net. Stale or mismatched ones actively drag the thing down.

After reading those two reports, I went back through my own almost-thirty Skills. The ones with correct local paths and stable, reproducible behavior, I kept. The ones that said nothing useful, or hadn't run in months, I disabled — not deleted yet, in case some project-specific one was still quietly in use and I'd have no way to get it back. This is the same habit a lot of people fall into with ChatGPT plugins or a string of MCP servers hooked up to Cursor: install because it's easy, tell yourself more can't hurt, and never go back to check the bill or the scorecard.

Next time I do this cleanup, what I'm actually watching for is whether that same list quietly grows back to thirty.