Menu
Use when implementing any experiment, analysis, or computational task — enforces the HYPOTHESIZE-BASELINE-EXPERIMENT-VERIFY-INTERPRET cycle and reproducibility requirements
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use when designing the evaluation protocol for a method development project — locks primary metrics, datasets, seeds, statistical tests, and baseline list into an immutable contract
Use when G2 (plan freeze) and G3 (execution readiness) gates have passed — handles all implementation, baseline reproduction, experimentation, and iteration for both Type M and Type D projects; this is Phase 4
Use when research direction is validated (G1 passed) and the project needs method design, analysis framework, evaluation protocol, or story line — this is Phase 3 and must complete before any code or experiments
Reusable protocol for multi-agent discussions that iterate until convergence. Used by results-integration (Phase 5) and paper-writing (Phase 6) to ensure discussions actually resolve issues rather than just noting them.
Use when and ONLY when the user explicitly requests paper writing and G4 (write-ready) gate has passed — handles LaTeX structure, senior-level writing, reference verification, and iterative refinement
Use when Phase 1 (research-direction-exploration) is complete and G1 gate has passed — subjects the research problem to adversarial defense-style questioning, classifies project intent, re-evaluates venue target, and produces a final feasibility ruling before any method design begins
| name | reproducibility-driven-research |
| description | Use when implementing any experiment, analysis, or computational task — enforces the HYPOTHESIZE-BASELINE-EXPERIMENT-VERIFY-INTERPRET cycle and reproducibility requirements |
NO EXPERIMENT WITHOUT PREDEFINED SUCCESS CRITERIA AND BASELINE FIRST
This skill is active for every computational task — experiments, analyses, data processing, model training. No exceptions. No "quick checks."
Like RED-GREEN-REFACTOR for software, research follows HYPOTHESIZE-BASELINE-EXPERIMENT-VERIFY-INTERPRET. Every cycle produces one atomic, reproducible unit of evidence.
Before running anything, write down:
Write it down. If you cannot state the prediction, you do not understand the experiment.
Run the baseline or known result first.
A method that "beats" an unreproduced baseline proves nothing.
Execute the experiment with full controls:
evaluation-protocol.yaml.Statistical verification — not eyeballing:
If results are within noise of the baseline, that is a null result — record it as such.
Whether the result supports or refutes the hypothesis:
Both positive and negative results are data. Record both with equal care.
These are non-negotiable for every computational task:
| Excuse | Reality |
|---|---|
| "Just a quick test, no need to log" | Quick tests become final results. Log everything. |
| "I'll set seeds later" | Later = never. Set them now. |
| "The baseline is well-known" | Reproduce it anyway. Your setup may differ. |
| "One run is enough to see the trend" | One run is anecdote. Multiple seeds are evidence. |
| "I'll clean up the code later" | Dirty code = unreproducible code. Clean as you go. |
| "Manual preprocessing is faster" | Faster now, unreproducible forever. Script it. |
| "The environment doesn't matter" | It does. Library version differences cause silent result changes. |
| "Git commits slow me down" | Losing results slows you down more. Commit after every meaningful run. |
Unreproducible results are not results.
They are anecdotes with extra steps.
Follow the cycle. Log everything. Reproduce baselines. Fix seeds. Script all steps. This is the minimum standard for computational research.