TL;DR Install it with
/plugin install skill-creator@anthropic-plugin-directoryand ask "create a skill that does X and test it with and without it". When it asks to write to.claude/, pick Yes, and allow Claude to edit files in this project's .claude folder for this session. You get the skill, its test cases and a benchmark of pass rate, time and tokens.
skill-creator is the skill Anthropic publishes for creating skills. It doesn't stop at writing the SKILL.md: it runs every test case twice, with the skill and without it, each in a subagent with a clean context, grades every output against a set of checks, and tells you how much the result improves and what that costs. Without that comparison you don't know whether your skill does anything Claude wouldn't have done on its own.
Install it: the docs command fails
The Claude Code docs give this command. On a clean install of v2.1.330 it answers this:
❯ claude plugin install skill-creator@claude-plugins-official
Installing plugin "skill-creator@claude-plugins-official"...× Failed to install plugin "skill-creator@claude-plugins-official": Plugin "skill-creator" not found in marketplace "claude-plugins-official". Your local copy may be out of date — try `claude plugin marketplace update claude-plugins-official`.
The update it suggests doesn't help either: Marketplace 'claude-plugins-official' not found. Available marketplaces: anthropic-plugin-directory. The marketplace Claude Code ships with is called anthropic-plugin-directory, and with that name it installs first time:
❯ claude plugin install skill-creator@anthropic-plugin-directory
Installing plugin "skill-creator@anthropic-plugin-directory"...√ Successfully installed plugin: skill-creator@anthropic-plugin-directory (scope: user)
Inside a session it's the same with /plugin install. And if you sign in to Claude Code with your claude.ai account, you may already have it: skill-creator is one of the Anthropic skills that sync from your account. Look for it in /skills, under claude.ai sync, and call it as /anthropic-skills:skill-creator.
Ask it for the skill
1. Say what you want, not how to write it
Use skill-creator to create a project skill called commit-msg that writes a Conventional Commits message from the staged diff. Test it on 2 cases, with the skill and without it.
Whatever you leave out, it asks: what the skill should do, when it should trigger, what the output looks like, and whether it deserves test cases. It recommends them for outputs you can check (code, transforms, fixed steps) and skips them for style or tone.
2. Let it save the skill in .claude/
.claude/ is a protected path: in default and acceptEdits modes every write there prompts you, and according to the docs not even an Edit(.claude/**) rule in your settings changes that. Answer Yes, and allow Claude to edit files in this project's .claude folder for this session. In claude -p there's nobody to ask: in this test, skill-creator left the skill in its working folder and returned the cp to install it by hand.
3. Wait for the benchmark
It launches every run at once, drafts the checks while they run, and stores each run's tokens and time.
What a real run returned
On Opus 5.5 it took 3 minutes and 40 seconds. It left the skill, evals/evals.json with both cases, and one folder per iteration (trimmed, without the second case):
commit-msg-workspace/
├── skill/commit-msg/SKILL.md
├── grade.py
└── iteration-1/
├── benchmark.json
├── benchmark.md
└── eval-0/
├── eval_metadata.json
├── with_skill/run-1/{outputs/commit_msg.txt, grading.json, timing.json}
└── without_skill/run-1/{outputs/commit_msg.txt, grading.json, timing.json}
The summary in benchmark.md:
| Metric | With Skill | Without Skill | Delta |
|--------|------------|---------------|-------|
| Pass Rate | 100% ± 0% | 81% ± 27% | +0.19 |
| Time | 17.1s ± 0.4s | 12.2s ± 1.1s | +4.9s |
| Tokens | 19556 ± 388 | 17257 ± 454 | +2298 |
The whole gain came from one case: the prompt "write me a commit message" without mentioning Conventional Commits. With the skill, the first line read feat: add tax rate and item quantity support to total(); without it, Add quantity and optional tax to total(), which failed the format, type and style checks. In the other case the prompt already asked for Conventional Commits and both passed everything. benchmark.md says so itself: with one run per configuration, it's a first signal, not proof.
That's what you get out of it: how much the skill adds and at what price, about 2,300 tokens and 5 seconds more per use.
Make it trigger when it should
A skill that doesn't trigger is useless, and its description decides that. skill-creator tunes it in a separate loop:
| Step | What it does |
|---|---|
| Queries | About 20: half should trigger it, the other half are near misses that shouldn't |
| Split | 60% to tune on, 40% held out to validate |
| Measure | Each query runs 3 times for a reliable rate |
| Iterations | Up to 5; it keeps the best description by held-out score |
It runs claude -p under the hood, so it only works where you have Claude Code. The description it wrote here was already pushy on purpose ("even if they don't say "Conventional Commits""), the second of Anthropic's five rules for skills.
skill-creator or claude plugin eval
| skill-creator | claude plugin eval |
|
|---|---|---|
| For | One skill, while you write it in a conversation | A whole plugin, against not loading it |
| Cases | evals/evals.json inside the skill |
Its own format; they aren't interchangeable |
| Output | benchmark.json and an HTML viewer to review |
An exit code to gate CI |
The second one has its own tip on plugin evals. And if you don't want to write even the first draft, record the task in Cowork, then hand the skill to skill-creator to measure it. If you solved the task in Claude Code a moment ago, capture it on the spot before you close the session.
Official docs: Run evals with skill-creator · Protected paths
Requirements
- Claude Code with plugins. Its benchmark and description scripts use Python.
- Tested with Claude Code v2.1.330 and Opus 5.5 on November 21, 2026.