| name | alterlab-results-transparency |
| description | Enforces results-reporting transparency as a discipline gate built on the Iron Law "NO RESULTS CLAIM WITHOUT REPORTING EVERY ANALYSIS RUN" — a numbered Gate Function (IDENTIFY the claim, LIST every test actually run including the ones that did not "work", CHECK assumptions were reported, CHECK effect size with 95% CI is present, CHECK pre-registration deviations are disclosed, ONLY THEN write the sentence), plus an Excuse-vs-Reality table and Red-Flags-STOP list for selective reporting, cherry-picking, and bare p-values. Use when writing up Results, claiming a finding from a subset of analyses, reporting a p-value without an effect size or confidence interval, dropping outliers post hoc, or omitting analyses that did not pan out. Orchestrates alterlab-statistical-analysis (tests, effect sizes), alterlab-preregistration-discipline (the frozen plan), and alterlab-open-science (TOP, disclosure); it does not run the tests itself. Part of the AlterLab Academic Skills suite. |
| license | MIT |
| allowed-tools | Read Bash(python:*) |
| compatibility | No API key required. Judgment/discipline skill; the optional reporting-gate helper runs locally via `uv run python` (stdlib only). |
| metadata | {"skill-author":"AlterLab","version":"1.0.0"} |
Results Transparency — The Reporting Discipline Gate
Skill type: DISCIPLINE-ENFORCING. This skill is not a how-to for running
statistics. It is a gate that stands between analyses you ran and the
sentence you are about to write in the Results section. It exists to stop the
single most common integrity failure in quantitative write-ups: reporting the
analyses that "worked" and quietly dropping the rest.
It orchestrates, it does not reimplement:
- REQUIRED BACKGROUND —
alterlab-statistical-analysis runs the tests,
checks assumptions (its alterlab-statistical-analysis/scripts/assumption_checks.py),
computes effect sizes and CIs, and produces the APA-formatted numbers. This
skill checks that those outputs are all present and disclosed before a claim
is written.
- REQUIRED BACKGROUND —
alterlab-preregistration-discipline owns the
frozen plan. This skill checks that confirmatory claims match it and that
deviations are labelled.
alterlab-open-science owns the disclosure mechanics (TOP Guidelines,
registration links, data/code availability).alterlab-scientific-thinking owns judging evidence quality; this skill
only enforces reporting completeness.
The Iron Law
NO RESULTS CLAIM WITHOUT REPORTING EVERY ANALYSIS RUN
Violating the letter of full reporting is violating the spirit of the science.
"Reporting every analysis" means every test you actually executed against the
data for this hypothesis — including the ones that returned p > .05, the
specification you abandoned, and the outlier rule you tried first. A Results
section is a census of what you did, not a highlight reel of what confirmed you.
When to Use This Skill
Invoke it the moment a results claim is being written or defended:
- About to write a sentence in the Results / Findings section.
- Stating "we found that…", "X significantly predicted Y", "there was no effect".
- Reporting a p-value, a coefficient, a group difference, or a correlation.
- Defending a finding drawn from a subset of the analyses that were run.
- Tempted to drop outliers, switch a test, or add a covariate after seeing the result.
- Writing the abstract's results sentence or a figure caption that asserts a finding.
If you are choosing which test to run, checking assumptions, or
an effect size, that is — come back here when you
move from computing numbers to claiming findings.
