| name | prompt-optimizer |
| description | A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results. |
Prompt Optimizer
Systematically improve Mycelium instructions through measurement. Adapted from n-trax.
Commands
baseline -- Capture current performance
- Run
/eval-runner run-all
- Record to
.claude/optimization/baseline.json: timestamp, CLAUDE.md hash, overall and per-category metrics (pass_rate, avg_iterations, avg_time)
test <variant> -- Test a variant
- Read variant from
.claude/optimization/variants/<variant>.md
- Apply the CLAUDE.md changes described
- Run
/eval-runner run-all
- Store results in
.claude/optimization/results/<variant>.json
- Compare against baseline
- Do NOT auto-revert -- let user decide
report -- Compare all variants
Generate comparison table: variant, pass rate, delta vs baseline, avg iterations, decision (keep/revert).
exemplar <eval-name> -- Capture winning trajectory
After a clean eval win (1 iteration, fast), save the approach to .claude/optimization/exemplars/.
Workflow
- Capture baseline
- Hypothesize an instruction improvement
- Document in variants/ directory
- Test the variant
- Compare via report
- Keep or revert based on data
- Capture exemplars from clean wins