Menu
Audit a dataset or pipeline for the five leakages that inflate a metric: target, preprocessing, temporal, group, and sampling.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Establish a dumb baseline and an honest validation harness before any real model, so every later number means something.
Rigor for causal questions and A/B tests (the effect of acting on X): confounding, post-treatment bias, valid control groups.
Rigor for descriptive and diagnostic analytics (what happened and why): denominators, grain, and confounded slices, not model leakage.
EDA kickoff for a fresh dataset: fixed opening scaffold (goal, imports, load, sanity), then chapters derived from the data; scan leakage, land a baseline.
Rigor for statistical inference (is the difference real): hypothesis tests, power, multiple comparisons, effect size over p-value.
Final modeling once the baseline and feature set are locked: tune against validation, audit overfitting, touch the test set once, justify the complexity.
| name | lemma-leakage |
| description | Audit a dataset or pipeline for the five leakages that inflate a metric: target, preprocessing, temporal, group, and sampling. |
| homepage | https://github.com/tkpratardan/lemma |
| license | MIT |
Rank leakage findings by severity and show reproducible before/after scores, the contaminated path, the corrected validation design, and remaining risk.
Do not rationalize an implausible score, rely only on feature importance, randomly split temporal or repeated-entity data, or continue tuning before material leakage is resolved.
For the full leakage taxonomy, read references/deep-guide.md.
