Menu
Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use *before* starting work in a domain you don't know well, to surface the "unknown unknowns" — the things you don't even know to ask about — and learn just enough to prompt and decide well. Implements the "blind spot pass" pattern from Anthropic's Fable "finding your unknowns" field guide. Triggers when you say "I'm new to this", "I don't know what to ask", "teach me before we start", "blind spot pass", "unknown unknowns", "find my unknowns", or the Korean "내가 뭘 모르는지 알려줘" / "이 분야 처음인데" / "먼저 가르쳐줘" / "블라인드 스팟", or when you hand off a task while admitting you're a non-expert in that field (color grading, video editing, legal, tax, finance, design, an unfamiliar codebase, etc.). Do NOT use for simple factual questions, domains you already know, or directly-actionable work — answer those directly or route to planner / architect.
Turn a one-line description of a repetitive task into a reusable, self-guarding slash command. loop-forge diagnoses the task into one of 5 loop shapes (Batch / Pipeline / Refine / Watch / Explore), interviews for the blanks, and auto-injects two safety devices the user didn't know they needed — an independent verifier (maker ≠ checker) and a hardstop (a budget/count/cooldown ceiling) — then previews the result and stamps it as a `/command` they can run forever. Use when the user says "/loop", "/loop-forge", "/make-it-loop", "automate this", "make this repeatable", "turn this into a command", "do this for all 100 items", "do X every time Y happens", "generate several and pick the best", or otherwise wants to capture a recurring task as a reusable slash command instead of re-typing the prompt by hand. Works in any language: it interviews the user and writes the stamped command in the user's own language. Non-goal — it does not schedule unattended runs (launchd/cron) or publish externally on its own; it stamps t
Use when detecting and running project build systems automatically. Supports npm/yarn/pnpm/pip/poetry/gradle/maven/cargo/go/make. Triggers on build, test run, project setup, package manager detection.
Use when starting Claude Code projects, writing CLAUDE.md/spec.md, dispatching subagents, or requesting Agent Teams parallel development. Covers Spec-Driven Development, Context Engineering, and post-dev workflow.
Instinct-based learning system that observes sessions via hooks, creates atomic instincts with confidence scoring, and evolves them into skills/commands/agents.
Use when searching, installing, or improving AI skills and prompts via prompts.chat or skills.sh. Triggers on skill search, prompt lookup, install skill, improve prompt, prompts.chat.
| name | eval-harness |
| description | Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles |
| allowed-tools | Read, Write, Edit, Bash, Grep, Glob |
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
Eval-Driven Development treats evals as the "unit tests of AI development":
Test if Claude can do something it couldn't before:
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
Ensure changes don't break existing functionality:
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
Deterministic checks using code:
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
Use Claude to evaluate open-ended outputs:
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?
Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
Flag for manual review:
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
"At least one success in k attempts"
"All k trials succeed"
## EVAL DEFINITION: feature-xyz
### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely
### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact
### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals
Write code to pass the defined evals.
# Run capability evals
[Run each capability eval, record PASS/FAIL]
# Run regression evals
npm test -- --testPathPattern="existing"
# Generate report
EVAL REPORT: feature-xyz
========================
Capability Evals:
create-user: PASS (pass@1)
validate-email: PASS (pass@2)
hash-password: PASS (pass@1)
Overall: 3/3 passed
Regression Evals:
login-flow: PASS
session-mgmt: PASS
logout-flow: PASS
Overall: 3/3 passed
Metrics:
pass@1: 67% (2/3)
pass@3: 100% (3/3)
Status: READY FOR REVIEW
/eval define feature-name
Creates eval definition file at .claude/evals/feature-name.md
/eval check feature-name
Runs current evals and reports status
/eval report feature-name
Generates full eval report
Store evals in project:
.claude/
evals/
feature-xyz.md # Eval definition
feature-xyz.log # Eval run history
baseline.json # Regression baselines
## EVAL: add-authentication
### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session
Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible
### Phase 2: Implement (varies)
[Write code]
### Phase 3: Evaluate
Run: /eval check add-authentication
### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT