| name | verifying-results-before-claiming |
| description | Use when about to claim a result, effect, significance, or that an analysis reproduces, before reporting or writing it up - requires running the analysis fresh and reading the actual output first; evidence before claims always |
Verifying Results Before Claiming
Overview
Claiming a finding without fresh verification is dishonesty, not efficiency.
Core principle: Evidence before claims, always.
Violating the letter of this rule is violating the spirit of this rule.
The Iron Law
NO CLAIMS WITHOUT FRESH REPRODUCED EVIDENCE
If you haven't run the analysis in this state and read its actual output, you cannot claim its result. "It was significant earlier" is not evidence now.
The Gate Function
BEFORE claiming any result or expressing satisfaction:
1. IDENTIFY: What command/analysis proves this claim?
2. RUN: Execute it fresh and complete (from the immutable raw data, fixed seed)
3. READ: The actual output — the estimate, the interval, the p-value, the diagnostics
4. CHECK: Do the method's assumptions hold? Does it reproduce?
For a CONFIRMATORY claim: does the pre-registration audit pass?
(prereg.sh, ships with science-superpowers:preregistering-analysis:
<skills root>/preregistering-analysis/prereg.sh audit)
5. VERIFY: Does the output actually support the claim?
- If NO: state the real result with evidence
- If YES: state the claim WITH the evidence (number + interval)
6. ONLY THEN: make the claim
Skip any step = asserting, not verifying
Common Failures
| Claim | Requires | Not Sufficient |
|---|
| "The effect is significant" | Fresh run; read estimate, CI, and p | "It was significant before" |
| "There's no effect" |
