| name | researchstack-paper-reproduction |
| description | Paper reproduction skill for computer systems, networking, and AI research. Use when the user provides a paper PDF,
appendix, or repository and wants Codex to reconstruct the method, extract claims and experiments, identify missing
details, plan a faithful reproduction, and judge whether the paper is reproducible, partially reproducible, blocked,
or contradicted.
|
Paper Reproduction
Read ../references/workflow.md, ../references/venues.md, and ../references/review-rubric.md.
Use the bundled templates:
Treat reproduction as a structured audit, not a vague attempt to "see if it runs."
Core outputs:
- Reproduction brief.
- Claim extraction.
- Method reconstruction.
- Experiment reconstruction matrix.
- Ambiguity log.
- Reproduction verdict.
Work in this order:
- Extract the paper's top claims and intended contribution.
- Identify the exact experiments, datasets, workloads, metrics, and baselines that support those claims.
- Reconstruct the minimal environment and dependency assumptions needed to rerun the evidence.
- List missing or ambiguous details before pretending the paper is reproducible.
- Decide which experiments are mandatory for a faithful reproduction and which are nice-to-have.
- Distinguish four outcomes clearly:
- reproduced
- partially reproduced
- blocked by ambiguity or missing assets
- contradicted by rerun evidence
Be explicit about failure sources:
- paper underspecification,
- code-release mismatch,
- environment mismatch,
- hidden tuning or preprocessing,
- unstable results,
- invalid assumptions by the reproducer.
Do not equate "result mismatch" with "paper is wrong" unless the evidence is strong enough to support that conclusion.
When a repository is available, connect this skill to:
experiment-design for the reproduced experiment matrix,experiment-ops for run tracking,artifact-audit for provenance and credibility,peer-review for reviewer-style reproducibility critique.
