| name | ai-input-data-and-design-audit |
| description | Use when auditing the data and foundational design of an AI/ML personnel assessment — Components 1-2 of the Landers & Behrend (2023) framework. Covers input-data population, sampling, range restriction, and incumbent-vs-applicant generalizability; and model design: how the criterion ("ground truth") is defined and its construct validity, why each predictor/feature was included, and whether choices were theory-driven or empirically derived. Triggers: "audit training data", "is the training sample representative", "ground truth validity", "why are these features predictors", "criterion definition in AI hiring", "range restriction in algorithm training data". |
| version | 1.0.0 |
| author | OpenMatter-Network |
| license | MIT |
| category | research |
| tags | ["Community","io-psychology","ai-assessment","auditing"] |
| permissions | [] |
AI input-data & model-design audit (Components 1–2)
The two far-reaching, hard-to-reverse decisions at the base of any AI assessment. Get these wrong
and everything built on top inherits the flaw — yet they're often the least documented. Define your
fairness standard first (ai-fairness-lenses) and scope via ai-audit-planning.
Component 1 — Input data
What it is: the validity and generalizability of the dataset(s) used to train the model,
relative to the population where predictions will actually be generated and used.
Questions to ask: How were input data collected, in terms of population and research
design? How did those factors affect data quality? Evaluate the population the data were drawn
from, the sampling techniques, and the adequacy of the data for the analyses later used to support
claims.
Apply it (focal example — algorithmic video interview): Were input data collected from job
incumbents? How do incumbents differ from applicants? Is there range restriction on
variables of interest? A model trained on a convenient source of existing candidates/incumbents is
no guarantee of representativeness for future applicants — so a key audit goal is to evaluate
how the developers investigated that assumption and concluded it was not a problem.
Audit emphases:
- This is the AI face of classic validation concerns: population selection and the
concurrent-vs-predictive design choice directly shape what can be inferred (see SIOP Principles
via
criterion-related-validation).
- Sampling bias (Lens 3, statistics): nonrandom under-/over-representation (e.g., undersampled
minorities) makes the model weight patterns of the most-represented group; "we'll just collect a
more diverse database / oversample" is not guaranteed to fix it.
- Every training dataset counts — including later subset datasets used for more specific
modeling challenges. Each must meet the same quality standard, embedded in a meaningful data
engineering framework. Don't let a well-vetted primary dataset launder a sloppy secondary one.
Component 2 — Model design
What it is: the initial modeling decisions that are difficult to test or change later. The
three most consequential:
- the nature of the criterion modeled,
- the set of predictors/features provided to the model,
- the initial set of algorithms used.
Even when the data are adequate (Component 1), the criterion and the available predictor set drive
many subsequent steps. These choices may be made conceptually, empirically, or heuristically — ask
which, and demand the rationale.
Questions to ask: What drove the initial model choices (criterion, predictor set, algorithm)?
Were they informed by theory or empirically derived? If empirically derived, from what data —
and of what quality — were they derived?
Apply it (focal example):
- Criterion / "ground truth" validity is the crux. "Performance data were available" is not
enough — audit the construct validity of that performance measure as a valid representation of
performance, including a clear, unambiguous, scientifically defensible definition of what
performance is. In computer science this is framed as choosing a "ground truth," often without any
explicit consideration of construct validity — name and close that gap.
- Predictor inclusion needs a rationale. Why are word choice, answer content, voice patterns,
and visual data included as predictors? Are those decisions theoretically supported? Even
though weak features can be dropped in later analytic stages, their initial inclusion is a
design-stage decision that creates technical debt and sunk-cost pressure — and, in the
focal example, drags in construct-irrelevant correlates (wrinkles→age, skin tone→race/ethnicity,
makeup→gender) that the designer may not even realize the model is exploiting.
Audit emphases:
- Tie criterion definition back to a real analysis of work
(
work-analysis) — a defensible ground truth requires it.
- Watch criterion reconceptualization / weighting decisions: down-weighting task vs. up-weighting
contextual performance can reduce adverse impact, but such weighting has validity implications
(Sackett et al., 2008) and must be carefully articulated and defended, not silently chosen.
- Flag predictors that proxy protected characteristics; inclusion-by-default is a finding.
Pitfalls
- Accepting "data were available" without auditing the criterion's construct validity.
- Treating incumbent-trained models as valid for applicants without addressing range restriction.
- Letting a clean primary dataset excuse unvetted subset datasets.
- Reading "features can be pruned later" as a reason not to justify their initial inclusion.
- Missing predictor proxies for protected class because the developer "didn't intend" them.
Checklist
See also
ai-model-development-audit · ai-model-outputs-audit · ai-fairness-lenses ·
work-analysis · criterion-related-validation
(criterion development, range restriction, concurrent/predictive)
Source: Landers & Behrend (2023), Table 1 (Components 1–2) and "Components Relating to Models" prose.
