icse-supplementary

name	icse-supplementary
description	Use when deciding what goes into the 10-page body versus the replication package of an ICSE research-track submission, covering the no-unlimited-appendix page model, anonymized supplementary links and HotCRP uploads, package organization for reviewer navigation, and content-placement rules that protect the review.

ICSE Supplementary Material

ICSE's page model is stricter than most authors coming from ML venues expect, and it inverts their packing instincts. The 2027 research-track format (read 2026-07-08): main text ≤ 10 pages inclusive of all figures, tables, and appendices, plus up to 2 pages of references only. There is no unlimited in-PDF appendix. Everything else travels as supplementary material — an anonymized link in the paper or an upload via HotCRP — and must be assumed optional reading.

The placement rule

One question decides placement: must a reviewer see it to score the paper?

Content	Placement	Why
Anything a criterion score depends on: RQ answers, key tables, protocol essentials, the threat analysis	The 10 pages	Optional material cannot carry mandatory weight
Full prompt templates, extended proofs, extra result tables, per-subject breakdowns	Package, summarized in-body	Body states the finding + pointer; package holds the mass
Tool source, benchmark, scripts, raw outputs	Replication package	This is the verifiability substrate (icse-reproducibility)
Interview guides, codebooks, instruments	Package, named in the availability statement	Reviewers audit method through them when they choose to
Demo video, screenshots gallery	Package, only if genuinely clarifying	Never as a substitute for a precise in-body description

The failure mode is load-bearing supplement: "due to space, the evaluation methodology is described in the supplementary material" invites the reviewer to score what the body shows — an unevidenced claim. Compress into the body instead: a protocol table instead of two prose pages, a summary row instead of the full matrix.

Summarize-and-point discipline

Every relocation leaves a residue in the body — the conclusion plus a precise pointer:

Patches for all 214 bugs, with per-bug generation logs, are in the package (results/patches/); Table 5 reports the aggregate. The full 41-item codebook is in codebook.pdf; Table 2 shows the 6 categories with ≥10% prevalence.

Precise pointers (directory, file, section of the package README) also pay forward: they become the artifact-evaluation roadmap after acceptance.

Package layout reviewers can navigate in five minutes

replication-package/
├── README.md            # map: claim/table/figure -> command -> output
├── REQUIREMENTS.md      # env: versions, container digest, hardware, runtime
├── LICENSE              # open license even at review time (anonymized holder)
├── code/                # tool source, pinned dependencies
├── benchmark/           # subjects or fetch-scripts with SHAs + mining date
├── scripts/             # one entry point per paper table/figure:
│   ├── run_rq1.sh       #   names match the RQs, not internal jargon
│   └── make_table3.py
├── results/             # raw outputs the paper's numbers derive from
└── appendix.pdf         # extended tables/proofs that did not fit the body

The README's first section is a claims map: each headline claim, the command that regenerates its evidence, expected runtime, and expected output. A reviewer who checks one row and succeeds extends trust to the rest; one who hits a broken path on the first try generalizes that, too.

Anonymity of the archive

The supplement must hold the same double-anonymous line as the paper: git archive exports (no history), scrubbed notebook metadata, no /home/username/ paths in configs or logs, no institutional hostnames, no author names in LICENSE/setup.py/CITATION files, hosting through an anonymizing service rather than a personal account. Scrub, then re-run the entry-point scripts — anonymization that breaks paths converts your evidence into a liability.

What must stay out of the package

Screening for exclusions is as important as packing inclusions: third-party code or datasets whose licenses forbid redistribution (link with pinned fetch-scripts instead); human-subjects raw data beyond what consent covers — transcripts, recordings, un-aggregated survey rows; credentials, API keys, and internal hostnames hiding in config files and CI logs; and proprietary industrial code shown to you under NDA, even in fragments quoted in comments. A replication package is a publication: it gets the same legal and ethical review as the paper, and a leak in the supplement is harder to retract than a sentence.

Size and dependency realism

No supplement size cap was verifiable for 2027 (待核实 on the live HotCRP form), so engineer for reviewer patience instead: keep the download small (fetch-scripts for large corpora, with checksums), keep the smoke test under minutes on a laptop, and never require credentials, cluster schedulers, or paid API keys for at least a reduced-scale verification path. Provide cached outputs beside every expensive step so the analysis chain can be verified without the compute.

Reverify each cycle

The 10+2 model, the HotCRP supplementary option, and the availability- statement placement are 2027-cycle facts; page models have changed across ICSE editions before. Confirm the current call before deciding any placement that would be expensive to reverse in deadline week.

Output format

[Body/package split] each relocated item -> residue sentence + pointer present?
[Load-bearing check] any criterion-relevant content living only in the package? (must be none)
[Package audit] layout roles present; claims map covers every headline number
[Anonymity + runnability] scrub done; entry points re-run post-scrub
[Reviewer path] smoke test minutes, laptop-feasible, credential-free