Menu
A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Baseado na classificação ocupacional SOC
Use to evaluate the current state of a diamond. Checks theory gates, confidence levels, and recommends next action.
Use when building anything USER-FACING (or with persuasion/retention/cancellation/consent/pricing flows, or that touches vulnerable people) to surface design-level harm the security/privacy/compliance gates miss: dark/deceptive patterns and foreseeable misuse. Assumes the product works as designed and asks who it could harm and whether it is Happier-negative. NUDGE, not a block.
Lint canvas files for staleness, missing fields, inconsistent evidence types, and orphaned references. Run periodically or before major transitions.
Accessibility audit against WCAG 2.1 AA. Checks semantic HTML, ARIA, keyboard navigation, color contrast, screen reader compatibility.
Design the smallest viable test to validate or invalidate a critical assumption. Based on Torres's assumption testing framework, organized by Gilad's AFTER model (Assessment → Fact-Finding → Tests → Experiments → Release Results).
Use before any research activity or significant decision. Reviews cognitive biases relevant to the current stage.
| name | prompt-optimizer |
| description | A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results. |
| metadata | {"instruction_budget":"17","framework_dependency":"mycelium","framework_dependency_note":"This skill is designed to run within the Mycelium framework (https://github.com/haabe/mycelium). Standalone use will skip the canvas state, theory gates, and harness behavior the skill assumes. Install: /plugin install mycelium@haabe-mycelium."} |
Systematically improve Mycelium instructions through measurement. Adapted from n-trax.
baseline -- Capture current performance/mycelium:eval-runner run-split optimization — record as optimization scores/mycelium:eval-runner run-split holdout — record as holdout scores.claude/optimization/baseline.json: timestamp, CLAUDE.md hash, optimization metrics, holdout metrics, overall and per-category metricstest <variant> -- Test a variant.claude/optimization/variants/<variant>.md/mycelium:eval-runner run-split optimization — this is the hill-climbing signal/mycelium:eval-runner run-split holdout — this validates generalization.claude/optimization/results/<variant>.jsonreport -- Compare all variantsGenerate comparison table with split-aware columns:
| Variant | Opt Pass Rate | Holdout Pass Rate | Delta Opt | Delta Holdout | Overfit? | Decision |
Flag Overfit? = YES when optimization delta is positive but holdout delta is negative.
exemplar <eval-name> -- Capture winning trajectoryAfter a clean eval win (1 iteration, fast), save the approach to .claude/optimization/exemplars/.