pg-data-screening

name	pg-data-screening
description	Screen psychology datasets before Pingouin analysis: variable types, missingness, duplicates, long/wide shape, assumption checks, and safer preprocessing.

PG Data Screening

Use before analysis when data has not been inspected in the current thread, or when a test fails because of missing values, shape, categories, or assumptions.

Load

Read:

• ../../references/supervision-gates.md
• ../../references/pingouin-api-quickref.md

Workflow

1. Inspect data columns, dtypes, sample size, missingness, duplicates, and group sizes.
2. Confirm long versus wide format for the target analysis.
3. Confirm repeated-measures identifiers are unique enough for the design.
4. Run only relevant assumption checks.
5. Produce a short analysis-readiness summary and minimal corrective code.

Standard Code Skeleton

import pandas as pd
import pingouin as pg

df = pd.read_csv("data.csv")
print(df.info())
print(df.describe(include="all"))
print(df.isna().mean().sort_values(ascending=False).head(20))
print(df.duplicated().sum())

For group balance:

print(df.groupby("group", dropna=False).size())

For normality by group:

pg.normality(data=df, dv="score", group="group")

For homogeneity:

pg.homoscedasticity(data=df, dv="score", group="group")

For repeated-measures sphericity:

pg.sphericity(data=df, dv="score", within="condition", subject="id")

For robust outliers (MAD-median rule) and a normality Q-Q plot:

mask = pg.madmedianrule(df["score"].to_numpy())   # True = flagged outlier
print({"n_outliers": int(mask.sum())})

import matplotlib.pyplot as plt                    # qqplot needs matplotlib
pg.qqplot(df["score"], dist="norm")
plt.savefig("qqplot.png", dpi=150, bbox_inches="tight")

Reporting

Report:

• Number of rows and unique participants.
• Missingness by required variable.
• Any impossible values or suspicious categories.
• Whether the data shape matches the intended Pingouin function.
• Assumption diagnostics and practical implication.
• Exact rows removed if listwise deletion is used.

Guardrails

• Assumption tests are diagnostic; do not mechanically switch tests solely from a significant normality test in a large sample.
• If repeated measures exist, never analyze rows as independent.
• If IDs repeat unexpectedly within a condition, stop and flag the design/data issue.
• If transformations or exclusions are proposed, state whether they were planned or post hoc.
• Flag outliers with a rule chosen in advance (e.g. MAD-median); never delete points silently.
• End with a compact audit line using S0-S5 gate codes.