菜单
Eval-Driven Development — define capability and regression evals before building; code graders use verify commands, model graders use explicit rubrics; log pass@k. Use before develop-tdd on new features, or when measuring agent capability over runs.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | run-evals |
| model | sonnet |
| description | Eval-Driven Development — define capability and regression evals before building; code graders use verify commands, model graders use explicit rubrics; log pass@k. Use before develop-tdd on new features, or when measuring agent capability over runs. |
HARD GATE — Define evals before implementation. Code graders = runnable
verify:commands; model graders = explicit rubric with pass/fail criteria.
Name the capability under test (one sentence).
Write specs/EVALS-<feature>.md with:
Assign grader type per eval: code (shell verify) or model (rubric).
Assign strictness tier per eval (graduated promotion — e45s37):
| Tier | Meaning | Promotion rule |
|---|---|---|
EXPERIMENTAL | New eval, may flake | Not gating |
USUALLY_PASSES | Stable in dev; ≥2/3 recent runs pass | Blocks BUILD only when combined with ALWAYS_PASSES suite |
ALWAYS_PASSES | Zero tolerance; required for release | Any single failure blocks BUILD and merge |
Promote: EXPERIMENTAL → USUALLY_PASSES after 3 consecutive passes; USUALLY_PASSES → ALWAYS_PASSES after 5 consecutive passes with zero flakes documented in specs/state.yaml.
Run evals; log results table with pass@k (e.g. 3/3 runs) and tier per eval.
Block BUILD phase until all ALWAYS_PASSES evals pass at agreed k. USUALLY_PASSES failures warn; EXPERIMENTAL failures log only.
specs/verifications/eNNsYY-eval-report.md — see REFERENCE.md for template. Eval reports are stored alongside verification evidence in specs/verifications/, keyed by story ID for traceability.
→ verify: find specs/verifications -name "*-eval-report.md" | wc -l | awk '{if($1>0) print "OK: "$1" eval reports"; else print "MISSING"}'
Add a tier: column to each eval row:
| Tier | Gate behaviour |
|---|---|
EXPERIMENTAL | Log only — does not block |
USUALLY_PASSES | Warn on failure; blocks only when paired with failing ALWAYS_PASSES |
ALWAYS_PASSES | Hard block on any failure |
# EVALS: <feature>
## Capability
| ID | Eval | Grader | Tier | verify / rubric |
|----|------|--------|------|-----------------|
| C1 | ... | code | ALWAYS_PASSES | `verify: npm test -- <file>` |
| C2 | ... | model | USUALLY_PASSES | Rubric: [ ] criterion A [ ] criterion B |
## Regression
| ID | Eval | Grader | verify / rubric |
|----|------|--------|-----------------|
| R1 | Full suite passes | code | `verify: npm test` |
## Results
| Run | C1 | C2 | R1 | pass@k |
|-----|----|----|-----|--------|
| 1 | PASS | PASS | PASS | 3/3 |
Run capability evals k times (default k=3). Ship when all k pass or document known flake in specs/state.yaml handoff.open_decisions.
Self-review checklist for the coding agent to run before dispatching a reviewer. Checks CONVENTIONS.md compliance, Boy Scout Rule, test coverage, types, and SOLID. Produces a pass/fail checklist. Use before request-review, before committing, or when user asks for a code quality check.
Create new bigpowers skills with proper structure, progressive disclosure, and bundled resources. Use when user wants to create, write, or build a new skill for the bigpowers lifecycle.
Refine a rough idea into a clear, detailed specification through dialogue. Does not produce code. Use when user has a vague idea, wants to think through a feature before planning, or needs to turn "I want X" into a concrete spec.
Benchmark-gated skill evolution — consume bigpowers-benchmark report, propose plan-work change, edit skill via craft-skill, re-run benchmark, record ADR. Use when a skill underperforms on benchmark or stocktake finds systemic gap.
Interactive assumption-surfacing Q&A that stress-tests a plan through relentless questioning until every decision is resolved. Use when user wants to challenge a plan, validate decisions from conversation/context, or mentions "grill me". For doc-grounded variant, use grill-with-docs.
Block dangerous git commands (push, force push, reset --hard, clean, branch -D, checkout/restore .) and enforce Conventional Commits & Branch Protection before an AI agent runs them. Installs hook scripts for Claude Code, Cursor, Cursor CLI, and Gemini CLI; documents Google Antigravity Terminal deny lists. Use when the user wants git safety hooks, to block git push or destructive git in agents, or to mirror the same policy across AI coding tools.