| name | grill-skill |
| description | Build and harden a skill with evals — interview to design its eval tasks, then run, measure, and iterate. Use when the user wants to create or improve a skill's eval, or run the create → test → improve loop for a skill. |
| allowed-tools | Bash, Read, Write, Edit |
Grill Skill
Interview the user to design a skill's eval, then loop run → measure → improve until it ships. Requires caliper (pipx install caliper-eval if missing). Commands, spec skeleton, and expect/assert guidance: REFERENCE.md.
Entry point
/grill-skill [path] — optional path to a SKILL.md.
- Path given — use it.
- No path — look for
SKILL.md in the cwd; if found, confirm before proceeding, else ask where it is.
Phase 1 — Understand
Read the SKILL.md. Summarize what it does, when it triggers, and what a successful run looks like. Ask the user to confirm your reading. Wait for confirmation before continuing.
Phase 2 — Detect eval mode
Look for *.eval.yaml beside the SKILL.md (try <dir-name>.eval.yaml first).
- None → New eval. Found → Gap-fill.
Interview one question at a time and wait for each answer. Never invent the user's answers or write the spec before interviewing.
New eval — three tasks
Elicit three tasks, one question at a time:
- Happy path — the most common successful use. What did the agent do, and what would confirm it worked?
- Edge case — a tricky-but-valid input that might trip the raw agent.
- Adversarial — what the skill should refuse or avoid.
Turn each answer into a task: a realistic prompt, an observable expect, and an assert when the outcome is checkable (see REFERENCE.md). Show the proposed YAML and confirm before writing.
Write the spec beside SKILL.md, named <dir-name>.eval.yaml, with skill.path: ./SKILL.md and claude-code as the default backend for both skill and judge unless the SKILL.md targets another.
Gap-fill
Read the existing spec and report its tasks. Ask what behaviors are missing or under-tested before proposing or writing anything — even if the user only asked you to inspect it, report first, then ask. Sharpen each gap into a task, show it, and confirm before writing it in.
Phase 3 — First run
Validate the spec, then run at k=1 (commands in REFERENCE.md). Show the results. Fix any harness or config error (not a task failure) before asking the user what to do next.
Phase 4 — Iterate
Ask whether to iterate or finish.
- Iterate — after the user edits their
SKILL.md, re-run at k=3 and show results. Loop back.
- Done — suggest a
--baseline run to prove the skill beats the raw agent, then remind the user to commit SKILL.md and the .eval.yaml together.
