engagement-survey-design

# Engagement Survey Design ## Listening System, Not Just Surveys A survey is a tool. A *listening system* is the practice of asking the right questions, at the right cadence, with protections that make truth-telling safe, interpreting honestly, acting on findings, and closing the loop. Vendors sell surveys. Consultants build listening systems. ## Cadence | Cadence | Purpose | |---------|---------| | Annual full-engagement | Comprehensive read; benchmarking; trend | | Pulse (quarterly) | Track key metrics between annuals | | Onboarding (Day 7/30/90/365) | New hire experience | | Exit | Why people leave | | Stay | Why people stay or might leave (predictive) | Always-on sentiment monitoring (passive Slack analysis, etc.) is increasingly available. Default position: skeptical. Surveillance frames feel different from listening. ## Question Selection Avoid: vendor-default 60-item bank; generic "I have a best friend at work"; aspirational items nobody could disagree with. Prefer: questions tied to outcomes the company can act on. Core constructs to consider: - Psychological safety (Edmondson 7-item scale) - Manager effectiveness (3–5 items) - Belonging - Career growth - Compensation fairness perception - Workload sustainability - Senior leadership trust - eNPS (one signal, not the headline) - Open-ended (1–3 questions for actual sentences) Edmondson's psychological safety items (paraphrased): 1. If you make a mistake on this team, it is not held against you 2. Members of this team are able to bring up problems and tough issues 3. People on this team sometimes accept others for being different 4. It is safe to take a risk on this team 5. It isn't difficult to ask other members of this team for help 6. No one on this team would deliberately act in a way that undermines my efforts 7. Working with members of this team, my unique skills and talents are valued and utilized ## Anonymity Protections - **Cell-size minimums**: n ≥ 5–8 at every cut, including intersectional - **Free-text scrubbing**: re-identification possible from quotes; scrub before sharing - **No HR-record linkage**: don't combine with individual records without explicit consent - **Vendor agreements**: prevent vendor sharing identifiable data back to company ## Response Rate Drivers - Trust that responses are anonymous - Visible action from prior surveys (closed loop) - Manager visibility without coercion - Length and design (shorter, well-designed beats longer) Targets: - < 60%: low; signal biased toward respondents - 60–80%: moderate - 80%+: strong ## Interpreting Signal Look for: - **Surprises**: results that contradict leadership's mental model — most valuable - **Trends**: same item declining for 3+ quarters — investigate - **Gaps**: differences across demographics, levels, functions, geos - **Open-ended themes**: actual sentences in employees' words Be cautious about: - Absolute scores (less meaningful than trends and gaps) - Cross-company benchmarks (companies measure differently) - Single-quarter changes (noise vs. signal) ## Closed Loop The single biggest predictor of program longevity: 1. Survey 2. Analyze (within 2 weeks) 3. Share results back to employees (within 4 weeks) 4. Identify 2–3 specific actions 5. Communicate actions and timeline 6. Execute 7. Reference actions in next survey communication 8. Repeat Skipping step 3 or 4 destroys the program faster than anything else. ## Manager-Team Reporting At scale, team-level results enable manager accountability: - Managers see their own team results (where n permits) - Managers do NOT see individual responses - Manager-of-manager reviews trends; coaches improvement - Persistent issues escalate Watch for: managers who pressure their team to score them well destroy the signal. ## Common Failures - Survey without acting (kills program faster than anything) - Vendor question bank verbatim - Cell-size violations (anonymity broken) - Always-on surveillance (feels invasive; trust drops) - Demographic linkage allowing re-identification - Manager pressure to score well - Skipping unfavorable findings in leadership reports - Surveying too often (fatigue) - Single-question programs (eNPS only, too thin) ## Cross-References - `engagement-researcher` agent - `psychological-safety` skill - `inclusion-belonging` skill - `culture-articulation` skill ## Key References - Edmondson, A. (1999). Psychological safety and learning behavior in work teams. - Reichheld, F. (2003). The one number you need to grow (NPS / eNPS origins). - Practitioner work from Culture Amp, Lattice, Glint