| name | eval |
| description | Eval-Driven Development (EDD) for AI workflows. pass@k metrics, capability evals, regression evals. Triggers: eval, edd, pass@k, capability, regression, benchmark. |
| allowed-tools | Read, Bash, Write, Edit, Grep, Glob |
| kernel | {"kind":"methodology","version":1,"side_effects":"none","confirmation":"none"} |
AgentDB read-start has run. Check for existing eval definitions in _meta/research/.
Understand what behavior you're evaluating before writing evals.
Skill-specific: skills/eval/reference/eval-research.md
<core_principles>
- DEFINE BEFORE CODE: Evals written first force clear thinking about success criteria.
- CODE GRADERS > MODEL GRADERS: Deterministic checks beat probabilistic judgments.
- STRUCTURAL SEPARATION FOR HIGH-STAKES: When stakes are real (security, payments, eval-of-evals, agent quality scoring), use the blind-evaluator agent — never self-score. Self-scoring inflates results ~36% structurally; procedural separation ("I won't peek") does not fix it.
- TRACK PASS@K: pass@1 (first attempt), pass@3 (within 3 attempts). Target pass@3 > 90%.
- REGRESSION BEFORE SHIP: Every change must pass existing evals before merge.
- FAST EVALS GET RUN: Slow evals get skipped. Keep evaluation fast.
</core_principles>
1. DEFINE: Write eval criteria before implementation. (gate: criteria exist in writing before any code)
2. IMPLEMENT: Code to pass defined evals.
3. EVALUATE: Run evals, record pass@k. (gate: pass@3 > 90% for capability; pass^3 = 100% for regression)
4. REPORT: Document results in eval report format. See reference for template.
pass@k: "At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts (typical target: > 90%)
