| name | micro-reproducibility |
| description | Use when hardening a MICRO paper's results for re-derivation — pinning simulator commits and configs, recording workload trace provenance and SimPoint recipes, versioning power/area models, capturing RTL toolchain state, and writing the run manifests that later survive MICRO's post-acceptance artifact evaluation. |
MICRO Reproducibility
In microarchitecture research the "experiment" is usually a software artifact
pretending to be hardware. That makes results perfectly reproducible in
principle — and silently unreproducible in practice, because a simulator commit, a
config diff, or a trace-generation flag changed between the headline run and the
camera-ready. This skill installs the record-keeping that prevents that.
What must be pinned, by evidence type
| Evidence | Pin these or the number is unrecoverable |
|---|
| Cycle-level simulation | Simulator name + exact commit hash; full config files (not "8-wide OoO" prose); patch set applied; host compiler for the simulator build |
| Workload traces | Source binary + compiler flags; trace tool + version; SimPoint interval size, K, and seed; warmup instructions per region |
| Power / area estimates | McPAT/CACTI/Accelergy version; technology node; the XML/cfg input actually fed in; any manual scaling factors |
| RTL results | HDL commit; synthesis tool + version; target library/PDK; constraints file; which corners were run |
| FPGA / silicon | Board/chip stepping; bitstream hash or firmware; frequency; measurement method for power (rail vs counter model) |
| DRAM behavior | Ramulator/DRAMsim3 version + timing config (device datasheet parameters) |
The rule of thumb: a result is reproducible when a stranger can regenerate the
exact bar in Fig. 3 from the manifest alone, without asking you anything.
The run manifest
Emit one manifest per experiment batch, generated by the launch script — never
written by hand after the fact:
simulator: {name: champsim, commit: 3fa9c21, patches: [prefetch-fix.diff]}
config: {file: configs/8w_ooo_baseline.json, sha256: 9be1...}
mechanism: {file: configs/ours_32kb.json, knobs: {table_entries: 2048, tag_bits: 12}}
workloads: {suite: spec2017_rate1, traces: traces/v3/, simpoints: k30_int200M_seed7,
warmup_M: 200, detailed_M: 1000}
power: {tool: mcpat, version: "1.3", node_nm: 22, input: mcpat/ours.xml}
host: {machine: lab-node-14, compiler: gcc-13.2}
outputs: {raw: stats/, figures: fig3_headline.pdf, script: plot_headline.py}
Commit the manifest with the raw stats files. Figures are regenerated from stats by
a script named in the manifest — a figure that cannot be regenerated does not go in
the paper.
Disclosure lines the paper itself owes
The methodology section (see micro-writing-style, ~1 page) must state, in text:
- Simulator and version, with your modifications summarized honestly ("we added a
cycle-accurate model of X; the interconnect remains the default model").
- The full baseline configuration in a table (pipeline width, ROB, caches with
latencies, predictor, prefetchers, DRAM timing).
- Sampling recipe: how regions were chosen, warmup, and measured instruction counts.
- Power/area model, node, and the sentence "these are model estimates" — the
community's tools (CACTI, McPAT — both MICRO papers themselves) are respected but
known to be approximations, and pretending otherwise reads as naivety.
Determinism and drift checks
- Two identical launches must produce bit-identical stats; if the simulator has any
nondeterminism (thread interleaving in parallel sims), pin it or report run count
and spread.
- Re-run the baseline after every simulator patch — a patch that changes baseline
IPC by 0.5% invalidates every relative number already collected.
- Keep a
results-ledger.md: date, manifest path, headline geomean, what changed.
Rebuttal season (June) will ask "what config produced Fig. 7?" — from three
months earlier; the ledger answers in minutes, memory does not.
Validating the instrument itself
Reproducibility of a wrong number is worthless; record the evidence that the
simulator models what the paper says it models:
- Cross-check the baseline against published behavior (see
micro-experiments): store the comparison table — suite, metric, published
range, your value — in the repo, not just in someone's memory.
- Unit-test the mechanism model. Hand-construct access sequences with known
correct outcomes (a burst that must train the table, a pattern that must not)
and assert the simulator's counters match. These micro-tests are the first
thing an AE evaluator can run and the best evidence the model is not a stats
bug.
- Record known fidelity gaps in a
LIMITS.md: default interconnect model,
simplified TLB, unmodeled prefetcher interactions. The paper's claims must
stay inside these limits; writing them down keeps deadline-week edits honest.
Anonymity interaction (submission phase)
MICRO 2026 requires artifact links to be fully anonymized or removed. Before the
April upload, sweep the repo you intend to link: git author fields, hostnames in
manifests (lab-node-14 is fine; patt-lab-utexas-14 is not), cluster paths,
funding strings in READMEs. The manifest discipline above makes this sweep
mechanical rather than archaeological.
Storage economics of keeping everything
Raw cycle-level stats for a full paper (dozens of configs × dozens of workloads ×
sweeps) usually fit in single-digit gigabytes — trivially archivable, and the
cheapest insurance in research. Policy: keep all raw stats files, forever,
compressed; regenerate figures on demand. Delete only simulator checkpoints and
intermediate trace files that the recipes can rebuild. When a reviewer asks in
June about a bar from a January run, or AE asks in August, or a follow-up paper
asks in two years, the manifest + stats pair answers; the alternative is
re-simulating weeks of compute under deadline pressure.
Output format
[Pinning audit] per evidence type: pinned / partially pinned / loose (list)
[Manifest coverage] experiments with manifests: N/M
[Figure regen] all paper figures scripted from raw stats: yes / no (list unscripted)
[Methodology text] simulator+mods / baseline table / sampling / model-estimate
disclosure: present-absent each
[Determinism] identical-rerun check passed: yes / no / untested
[Ledger] results-ledger current through: <date>
[Anonymity sweep] repo clean for anonymized linking: yes / issues listed
