| name | agentpack-skill-review |
| description | Review an existing agent skill and generate a balanced trigger/non-trigger eval set with a reproducible iteration workspace. Use when improving, benchmarking, or checking whether a SKILL.md triggers correctly. |
| license | AGPL-3.0-only |
AgentPack Skill Review
Use when the user invokes $agentpack-skill-review <skill path or name> or asks to review, improve, benchmark, or generate evals for a skill.
Workflow
- Run
agentpack skill-review --skill "$ARGUMENTS" --json and read the generated review.md, findings.json, and evals.json.
- Inspect the target
SKILL.md directly. Treat the deterministic findings as evidence-backed checks, not a complete quality judgment.
- Review the candidate eval set. Keep an even split of realistic should-trigger and near-miss should-not-trigger queries; edit or add cases when the generated wording is artificial.
- Run the evals against both the current skill and a baseline without the skill when the host can run isolated agent tasks. Store transcripts, outputs, timing, and token metadata under the workspace's next
iteration-N/ directory.
- Grade objective assertions programmatically where possible and record human feedback separately. Compare trigger precision/recall, output quality, latency, and token cost.
- Propose targeted changes to
SKILL.md and rerun the same evals as a new iteration. Do not claim improvement from a single example or from generated candidates that the user has not reviewed.
- Report the workspace paths, findings, eval-set balance, baseline comparison, and validation status. If a browser viewer is available, use the host's standard eval viewer rather than writing a custom one.
The CLI command only writes local artifacts. It does not call a hosted model API or edit the skill automatically.
