| name | alterlab-ssci-measurement-gate |
| description | Gates measurement quality before a scale or instrument is trusted — checks that each construct is defined and operationalized, that reliability is evidenced with McDonald omega (not Cronbach alpha alone, which assumes tau-equivalence and is only a lower bound), that reliability is not confused with validity (content, criterion, convergent/discriminant construct validity), and that measurement invariance is tested before comparing groups. Use when asking whether a scale or survey instrument is valid, reporting a Cronbach alpha, building or adopting a multi-item measure, or comparing a latent construct across groups. For designing the questionnaire items prefer alterlab-survey-design; for running the confirmatory factor analysis prefer alterlab-sem-psychometrics; to execute basic statistics prefer alterlab-statistical-analysis. Part of the AlterLab Academic Skills suite. |
| license | MIT |
| allowed-tools | Read Bash(python:*) |
| compatibility | No API key required. A discipline-enforcing measurement-quality skill; reliability/CFA execution is handed to sibling skills that run locally via `uv run python`. |
| metadata | {"skill-author":"AlterLab","version":"1.0.0","depends_on":"alterlab-survey-design (item design), alterlab-sem-psychometrics (CFA/invariance), alterlab-statistical-analysis"} |
Measurement Gate — Reliability Is Not Validity
Skill type: DISCIPLINE-ENFORCING. Before any hypothesis test consumes a scale, this gate
checks that the scale measures what it claims, reliably, and comparably across groups. It does
not fit factor models itself — it requires the right evidence and routes execution to the
psychometrics skill.
The Core Rule
A RELIABLE MEASURE OF THE WRONG CONSTRUCT IS STILL WRONG. RELIABILITY ≠ VALIDITY.
High internal consistency only means the items move together; it says nothing about whether
they capture the intended construct. Evidence for validity (content, criterion, construct) is
separate and mandatory. And the usual reliability number — Cronbach alpha — rests on an
assumption (tau-equivalence: equal item loadings) that real scales rarely meet, so alpha is a
lower bound that can under- or (with correlated errors) over-state reliability.
When to Use This Skill
- "Is my scale / survey instrument valid?"
- "My Cronbach alpha is 0.78 — is the scale good to use?" (← alpha alone is not enough)
- "I built a new 6-item measure of burnout; how do I show it works?"
- "I want to compare a latent trait (e.g. trust) across countries/groups."
Does NOT Trigger
| The request is really about… | Route to | Why not this skill |
|---|
| Writing / wording the questionnaire items, response scales | alterlab-survey-design | Instrument construction, upstream of validation. |
| Running the CFA / SEM / IRT / invariance model | alterlab-sem-psychometrics | This gate requires CFA; that skill fits it. |
| Basic descriptive/inferential stats execution | alterlab-statistical-analysis | Computation, not measurement discipline. |
| Which statistical test to run on a measured outcome | alterlab-test-selection-guard | Test choice, not measurement quality. |
| Sampling frame / how many respondents | alterlab-ssci-sampling-gate | Sampling, not measurement. |
Reliability: report alpha AND omega
Cronbach's alpha assumes tau-equivalence (all items load equally on one factor) and is a
lower bound on reliability under that model. McDonald's omega relaxes tau-equivalence by
weighting items by their factor loadings and should be reported alongside alpha (McNeish, 2018,
, "Thanks coefficient alpha, we'll take it from here"). In practice the
alpha-vs-omega gap for well-constructed unidimensional scales is often small (Warne, 2025 —
around 4.5% underestimate), so the discipline is: , plus item-total correlations
and a dimensionality check — not to fetishize a single number, but to show the estimate is not
resting on an unexamined assumption.
