skill-eval

name	skill-eval
version	2.5.0
description	Use when you want to test, measure, or compare skill effectiveness. Supports eval, compare, metrics, baseline capture, and regression detection.
trigger	manual
depends-on	[]
references	[]
user-invocable	true
allowed-tools	Read, Glob, Grep, Bash, Edit, Write
argument-hint	<mode> [skill-name] [--scenario <desc>]

---

skill-eval

Evaluating skill: $ARGUMENTS

Modes

Parse the first argument to determine mode:

Mode	Syntax	Purpose
eval	/skill-eval eval <skill-name> --scenario "<prompt>"	Test a skill against a known scenario
compare	/skill-eval compare <skill-name> --old <path> --new <path> --scenario "<prompt>"	A/B test two versions of a skill
metrics	/skill-eval metrics <skill-name>	Analyze a skill's evaluation criteria and pressure scenarios
baseline	/skill-eval baseline <skill-name> --scenario "<prompt>"	Capture baseline output for regression detection
regression	/skill-eval regression <skill-name>	Compare current skill against saved baseline

Mode: eval

Test a skill against a scenario and grade the output against expectations.

Process

1. Load the skill — Read .claude/skills/<skill-name>/SKILL.md
2. Load evaluation criteria — Read the skill's ## Evaluation Criteria section (if present in SKILL.md or SKILL_TEMPLATE.md)
3. Define the scenario — Use the provided --scenario or prompt the user for one
4. Execute mentally — Walk through what Claude would do with this skill loaded, given the scenario prompt. Trace the expected decision points, tool calls, and outputs.
5. Grade against criteria — For each evaluation criterion, determine PASS/FAIL with evidence

Output

## Skill Evaluation: <skill-name>

### Scenario
> <the test scenario>


| # | Criterion | Status | Evidence |
|---|-----------|--------|----------|
| 1 | [criterion text] | PASS/FAIL | [why] |


 [ ] All hard gates would be respected
 [ ] Red flags table would prevent shortcuts (if applicable)


 Estimated SKILL.md tokens: X
 References loaded: [list]
 Total estimated tokens: X


[Summary of findings]