paper-replication

name	paper-replication
description	Replicate LaTeX-first research papers with a persistent per-paper harness instead of a giant one-shot prompt. Use when Codex needs to reproduce figures, tables, experiments, or reports from paper sources, paper-provided data, and paper-derived specs; especially when the work is long-running, stateful, or vulnerable to context compaction.

Paper Replication

Use this skill to turn a paper-replication request into a repo-local harness with persistent state, validators, and reproducible entrypoints.

Default policy:

• source_mode=latex-first
• author_code_policy=forbid_by_default
• stack_policy=paper-driven
• compute_mode=auto
• target_progression=single-active-target

First move

Bootstrap or reopen the harness before doing substantive work.

Create a new case-study scaffold:

python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py bootstrap \
  --project-dir ./case_studies/my-paper \
  --paper-title "My Paper" \
  --paper-source /absolute/path/to/paper-source-tree

Resume an existing scaffold:

python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py status \
  --project-dir ./case_studies/my-paper

Refresh the generated harness files in an existing scaffold:

python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py sync-harness \
  --project-dir ./case_studies/my-paper

The harness is the system of record. Keep operational state in:

• paper_manifest.json
• spec/reproduction_matrix.csv
• todo.md
• spec/*.md

Do not rely on chat history to remember target order, assumptions, or completion status.

Core workflow

1. Bootstrap or reopen the case-study harness.
2. Run inspect-paper to inventory the TeX tree, figure assets, appendices, and data references.
3. Fill the spec files and enumerate every target in spec/reproduction_matrix.csv, including acceptance_mode when a target should be judged by numeric, distributional, or qualitative equivalence instead of exact visual matching.
4. Keep exactly one ACTIVE target until all rows are terminal.
5. Use the run wrapper to generate artifacts and record provenance before any target can be claimed.
6. Register baseline targets only after a real paper-method implementation exists under project code/config/spec-trace files.
7. Mark a target MATCHED only after the generated artifact exists, provenance exists, comparison evidence exists, and report/main.tex embeds it.
8. Use validate-spec, validate-progress, and validate-report to gate local consistency.
9. Treat the paper as done only when validate-completion passes for the full target set.

Core commands:

python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py inspect-paper --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py track-run --project-dir <case-study> --label "<label>" --shell-command "<command>" --expected-artifact <artifact>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py register-target-artifact --project-dir <case-study> --target-id <target> --run-id <run-id> --method-label <method> --code-path <code> --config-path <config> --paper-trace-path <spec-trace> --implementation-kind paper-method --method-component <component> --implementation-summary "<summary>" --seed <seed> --baseline-faithful
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py record-comparison --project-dir <case-study> --target-id <target> --kind figure --acceptance-mode numeric-equivalence --note "<summary>" --metric final_value_error=0.02
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py validate-spec --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py validate-progress --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py validate-report --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py validate-completion --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py sync-harness --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py build-paper-pdf --project-dir <case-study>
python3 ~/.codex/skills/paper-replication/scripts/paper_replication.py render-paper-pages --project-dir <case-study>

Anti-cheating enforcement

• Treat paper figures, tables, extracted pages, and source-tree assets as reference-only.
• Never copy paper-provided outputs into reproduced artifact paths.
• Use wrapper-generated run records and target provenance; do not satisfy MATCHED with loose files.
• Require comparison evidence before any figure/table can be marked MATCHED.
• Use exact visual matching only for targets that actually require it. Convergence curves and other stochastic summaries should usually be matched on numeric or structural equivalence.
• Treat baseline replication as paper-faithful by default. Keep improved, surrogate, or alternate methods separate and clearly labeled.
• Reject baseline claims that are only artifact generators, paper-pattern matchers, or other non-method stand-ins.
• Do not treat a clean scaffold as completion; every planned paper target must reach a terminal matched state before validate-completion can pass.

Operating rules

• Externalize decisions. Update repo files, not just chat.
• Replace prose constraints with executable checks whenever possible.
• Treat missing paper details as explicit hypotheses with evidence, not silent guesses.
• Keep author code blocked unless the manifest policy is changed intentionally.
• Keep paper assets under artifacts/paper_figures/ and reproduced artifacts under artifacts/figures/ or artifacts/tables/.
• Delegate cluster execution to $cluster-slurm; do not embed SLURM orchestration here.
• Treat one SSH/DNS transport miss as transient. Retry the project-local scripts/cluster_reproduce.py path before writing cluster execution off as blocked.

If cluster execution is needed, read references/cluster-integration.md. If you need the file/state contract, read references/harness-contract.md. If you need to choose per-target matching rules, read references/acceptance-modes.md. If you need the policy switches, read references/policy-modes.md. For the minimal operator flow, read references/quickstart.md.