structured-interviewing

name	structured-interviewing
description	Use when designing or auditing interview loops — covers job analysis, behavioral and case interviewing, work samples, scorecards, and calibration.

Structured Interviewing

Why It Matters

The evidence on selection methods is unambiguous (Schmidt & Hunter, 1998; updated Schmidt, Oh, & Shaffer, 2016). Validity coefficients for predicting job performance:

• Unstructured interview: r ≈ .20–.38
• Structured interview: r ≈ .46–.58
• Work sample / job audition: r ≈ .44–.54
• General mental ability: r ≈ .51–.65 (with disparate impact concerns)

Across thousands of hires, the difference between structured and unstructured methods is enormous in mis-hire cost, ramp time, and disparate impact.

Six Components of a Structured System

1. Job Analysis

• Outcomes: what must this person produce in 12 months?
• Competencies: 4–7 capabilities required
• Counter-indicators: what disqualifies (especially for senior roles)
• Stage fit: what differs vs. the same role at another stage

2. Loop Design

• 5–7 stages typical for senior IC; 4–6 for mid-level
• Each interviewer assigned 1–2 specific competencies
• Non-redundant coverage
• Total candidate time bounded (typically 4–8 hours including any take-home)

3. Scorecards

• Filled out before debrief
• Specific evidence cited
• 4-point scale with anchored descriptions
• One independent rating per interviewer

4. Question Bank

• Behavioral questions ("Tell me about a time when...")
• Probe with STAR (Situation, Task, Action, Result; emphasize Action)
• Anchored ratings: what does 5/5 sound like? 1/5?
• Calibrated periodically

5. Work Samples

• Representative of actual job, not contrived
• Time-bounded (60–120 min onsite, capped take-homes)
• Compensated when meaningfully extensive (3+ hours)
• Same prompt for every candidate
• Multi-evaluator review

6. Calibrated Debrief

• Scorecards in before debrief
• Each interviewer presents evidence, not just verdict
• Decision criteria stated up-front
• Hiring manager owns the call; bar raisers / senior leaders may have veto

Common Behavioral Question Anchors

For "leadership at senior IC":

• 5: Has built and reorganized teams; concrete frameworks for performance management; specific examples of difficult feedback delivered well
• 3: Mentored ICs; mixed examples; clear views on hiring
• 1: No mentorship experience; weak feedback philosophy; couldn't give concrete examples

For "judgment under ambiguity":

• 5: Walked through a recent ambiguous decision with explicit consideration of tradeoffs and evidence; admitted what they didn't know; followed up post-decision
• 3: Articulated the decision but glossed over tradeoffs; some hindsight reasoning
• 1: "I just went with my gut"; unable to articulate decision process

Avoid

• Brain teasers (no validity for most roles; high bias)
• Same questions every interviewer (redundant; coverage gaps)
• "Conversational" interviews
• Verdict-first scorecards
• "Culture fit" without operationalization
• Long unpaid take-homes (>3 hours) — deters underrepresented candidates

Hiring for Culture-Add, Not Culture-Fit

"Culture fit" correlates with homogeneity and weak performance prediction. Replace with two distinct concepts:

• Values alignment: behavioral evidence of operating consistent with stated company values
• Culture-add: complementary perspective, lived experience, or strength absent from the existing team

Funnel Audit

Quarterly, by stage and demographic:

• Application → recruiter screen → onsite → offer → accept
• Quality-of-hire: 90-day rating, retention 12/24 months
• Demographic conversion at each stage

Drop-offs at a single stage indicate process problem at that stage. Demographic drop-offs are bias signal — investigate cause.

Key References

• Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology. Psychological Bulletin, 124(2), 262–274.
• Schmidt, F. L., Oh, I.-S., & Shaffer, J. A. (2016). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings.
• Bock, L. (2015). Work Rules! Twelve.
• Bohnet, I. (2016). What Works: Gender Equality by Design. Harvard University Press.
• Smart, G., & Street, R. (2008). Who: The A Method for Hiring. Ballantine.

Pitfalls

• Building structure without training interviewers
• Rubrics on paper that nobody uses in debrief
• "Bar raisers" without accountability
• Skipping work samples for senior roles where they're most valuable
• Audit data not segmented by demographic