| name | deckforge-skill-evaluator |
| description | Evaluate and improve DeckForge Agent Skills by comparing baseline, current, and candidate conditions on deterministic outcome tests, trigger precision, blind review, context cost, runtime, and cross-agent consistency. Use for skill development and release gates; do not use for end-user deck creation. |
| version | 1.0.0 |
| user-invocable | false |
| license | MIT |
deckforge-skill-evaluator
Development evaluator. Grade outcomes, not whether the agent mentioned an internal step.
- Identify the intended outcome.
- Select three to five high-quality tasks from
evals/core-eval-cases.json.
- Create baseline, current, and candidate conditions.
- Validate automated graders against reference results.
- Run install, trigger, build, behavior, accessibility, visual, and export assertions.
- Run across supported agents where applicable.
- Blind human reviewers to condition labels.
- Measure quality delta, variance, context cost, runtime, and false claims.
- Analyze failures and revise.
- Block release when critical tasks regress.
Read references/blind-review-protocol.md and assets/quality-rubric.json, plus the
shared contract at ../deckforge/references/skill-evaluation-contract.md. Write results
per ../../schemas/skill-eval-result.schema.json and ../../schemas/trigger-eval-result.schema.json.
