| name | revedres-transparency-and-reproducibility |
| description | Use when settling coding reliability, open materials, PRISMA/MARS reporting, and a reproducible meta-analysis for a Review of Educational Research (RER) manuscript. Makes the review auditable and re-runnable; it does not run the substantive robustness analyses (revedres-comprehensiveness-and-balance) or design the exhibits (revedres-tables-figures). |
Transparency & Reproducibility (revedres-transparency-and-reproducibility)
When to trigger
- The analysis is done and you must make the review auditable and re-runnable
- Coding reliability has not been quantified or reported
- You have not decided what search files, codebook, data, or scripts to share
- A reviewer is likely to ask "could someone reproduce this synthesis from what you report?"
Why transparency IS the contribution at RER
Because RER reviews are submitted and peer-reviewed (not invited), a reader cannot take the synthesis on the author's authority — they must be able to check it. The credibility of a systematic review or meta-analysis is exactly the credibility of its process documentation: the search, the screening, the coding, and the analysis must be reported so completely that an independent reader could, in principle, re-run them and land in the same place. This is not an appendix afterthought; it is the spine of the review's claim to be systematic.
Reporting standards RER points to
RER directs authors of systematic reviews and meta-analyses to specific reporting standards (检索于 2026-06；以官网为准):
- PRISMA — the systematic-review/meta-analysis reporting checklist and flow diagram. Report against every applicable item; include the flow diagram (see
revedres-tables-figures).
- APA "Reporting Standards for Research in Psychology" (the JARS family, incl. MARS for meta-analyses) — RER's submission guidance cites the American Psychologist reporting-standards article. Use MARS to ensure the meta-analysis reports its eligibility, search, coding, effect-size metric, model, and heterogeneity/bias analyses completely.
- PRISMA-P for the protocol (see
revedres-proposal-and-commissioning); PROSPERO/OSF registration cited.
Coding reliability: report it, don't assert it
- Double-code a defined share of studies (or all), independently.
- Quantify agreement with the appropriate statistic — Cohen's/Fleiss' κ for categorical codes, ICC for continuous extractions — and report the value, not just "two coders agreed."
- State conflict resolution (consensus, third coder) and re-check after disagreements.
- Provide the codebook so a reader knows exactly what was coded and how.
Open materials: make the synthesis re-runnable
- Share the search artifacts: full strings per database, dates, and the screening decisions/PRISMA counts.
- Share the coding dataset and codebook (de-identified as needed) on OSF or a repository, with a DOI.
- Share the analysis code (R
metafor/meta, Stata meta, or Python) so the pooled estimates, moderators, and bias diagnostics are reproducible from the shared data.
- Distinguish preregistered from exploratory analyses, and flag/justify every deviation from the protocol.
- Note: RER publishes no original empirical data of yours; the open-materials obligation is on the review's own search, coding, and synthesis — your reproducible contribution.
Checklist
Anti-patterns
- "Two coders agreed" with no κ/ICC value (unauditable reliability)
- A meta-analysis whose model, weights, or effect-size metric a reader cannot reconstruct
- Search strings paraphrased, not reported verbatim per database
- Silent deviations from the registered protocol
- Treating open materials as optional polish rather than the basis of the systematic claim
- Sharing nothing because "it's only a review" — the review's own process is the data
Output format
【PRISMA】checklist + flow complete? Y/N
【MARS/JARS】meta-analysis reporting standards walked? Y/N
【Registration】PROSPERO/OSF cited; deviations flagged? Y/N
【Coder reliability】κ/ICC value reported = <value>; conflict resolution stated? Y/N
【Codebook】included? Y/N
【Search artifacts】strings + dates + screening counts reported in full? Y/N
【Open materials】dataset + code shared (DOI)? Y/N
【Preregistered vs exploratory】distinguished? Y/N
【Next step】→ revedres-editor-strategy (scope negotiation + reviewer expectations)
