verify-a-statistical-claim

name	verify-a-statistical-claim
category	data
description	Check a numerical claim from its original data and definitions through calculation, uncertainty, and presentation. Use when a percentage, trend, comparison, or risk figure matters.

verify-a-statistical-claim

Reconstruct the claim, not just the arithmetic. Most statistical errors begin with mismatched populations, time windows, denominators, or measures.

When to use

• Use for reported percentages, rates, averages, rankings, changes, correlations, forecasts, and risk statements.
• Increase rigor when decisions, money, health, safety, or public claims depend on the result.

Procedure

1. Copy the exact claim and parse its numerator, denominator, unit, population, geography, period, comparison, and implied causal strength.
2. Trace the number to the original dataset, table, analysis, or model and record its version.
3. Read definitions, sampling, exclusions, weighting, missing-data handling, revisions, and uncertainty notes.
4. Reproduce the calculation from the most granular authorized evidence available.
5. Check percentage points versus percent change, nominal versus real values, counts versus rates, and mean versus median.
6. Test alternative reasonable denominators, baselines, cutoffs, and time windows.
7. Assess precision, confidence intervals or error bounds, sample size, and practical significance.
8. Check whether aggregation hides subgroup differences or whether repeated observations are treated as independent.
9. State the verdict: verified, approximately supported, misleading, unsupported, or not reproducible, with corrected wording.

Failure plan

• Do not infer causation from correlation or a trend from two selected endpoints.
• Do not fabricate missing data, weights, uncertainty, or source revisions.
• If only a chart image is available, label any extracted value approximate.
• Preserve privacy and disclosure limits when checking small groups.

Done

• The claim is traceable to a versioned source and reproducible calculation
• Definitions, denominator, uncertainty, sensitivity, and causal limits are explicit
• The recommended wording matches the evidence