| name | skill-eval |
| description | Use to orchestrate and evaluate a multi-agent TDD skill benchmark. Trigger when the user says start eval, wants junior/mid/senior coder agents on separate model tiers, parallel git worktrees, adversarial evaluation of result bundles, deterministic coverage/mutation scoring, or a consolidated parity report. |
Skill Eval
Purpose
Evaluate whether the same TDD skill lets lower-capability coder agents perform comparably to higher-capability coder agents.
This skill is separate from tdd and eval-result. Coder agents use tdd to implement and eval-result to package evidence; the evaluator uses this skill to orchestrate and score them.
Agent Files
Use these repo-level agent definitions as the source of truth:
.codex/agents/junior-coder.yaml: lower-capability coder, model gpt-5.4-mini with low reasoning effort, workspace worktrees/junior.codex/agents/mid-coder.yaml: coding-optimized mid coder, model gpt-5.3-codex, workspace worktrees/mid.codex/agents/senior-coder.yaml: senior baseline coder, model gpt-5.5, workspace worktrees/senior.codex/agents/tdd-evaluator.yaml: adversarial evaluator, model gpt-5.5
Required Inputs
Before running eval, require:
- a root-level
prompt file produced by env-setup or equivalent task context
.codex/tdd-setup.json or equivalent setup metadata- a clean baseline commit or workspace snapshot
- deterministic test/coverage/test-strength commands or recorded fallbacks
If the prompt file or setup metadata is missing, run or request env-setup first.
Start Eval Protocol
When the user says start eval:
- Read
prompt, .codex/tdd-setup.json, and the four .codex/agents/*.yaml files.
- Create three isolated git worktrees from the same baseline:
worktrees/juniorworktrees/midworktrees/senior
- Spawn the three coder agents in parallel using their agent files.
- Give each coder identical task context, setup metadata, commands, and acceptance criteria.
- Require each coder to return the
eval-result bundle.
- After all coders finish, inspect each worktree directly.
- Run deterministic verification commands from setup metadata where available.
- Score each result using
references/rubric.md.
- Produce the consolidated report using
references/report-format.md.
If git worktrees are unavailable, use isolated directories copied from the same baseline and record the fallback.
Generated worktree code is run evidence only. Do not preserve it by default.
Adversarial Evaluation Rules
Do not trust coder self-report. Score only from artifacts:
- final files
- tests
- command outputs
- coverage reports
- mutation reports or mutation-style proxy checks
- diffs against baseline
- chronological logs/checkpoints where available
Penalize:
- production code before failing test evidence
- missing RED evidence
- tests that pass trivially or do not assert behavior
- broad rewrites unrelated to the prompt
- claims unsupported by worktree artifacts
Show Report Protocol
When the user says show report:
- Re-read the latest worktree artifacts and result bundles.
- Re-run deterministic commands if needed and safe.
- Recompute weighted scores.
- Compare junior and mid aggregates against senior using the +/-5 percentage-point parity band.
- Return the consolidated report.
Tune Skill Protocol
When the user wants to improve the skill after a report:
- Use the latest report to identify the most likely general weakness in
tdd.
- Propose one focused edit to
tdd; do not overfit to the benchmark solution.
- Do not edit the skill unless the user asks to apply the proposed change.
- After accepted edits, commit or otherwise snapshot only reusable skill changes.
- Treat
worktrees/, prompt, .codex/tdd-setup.json, reports, coverage output, mutation output, and result bundles as disposable run state.
- Rerun
env-setup and skill-eval from a clean baseline for the next iteration.
References
references/rubric.md: weights, scoring rules, fallbacks, parity calculation.references/report-format.md: output format.references/subagent-prompts.md: exact prompts derived from .codex/agents/*.yaml.
