Use this skill when planning user research, conducting usability tests, creating journey maps, or designing A/B experiments. Triggers on user interviews, usability testing, user journey maps, A/B test design, survey design, persona creation, card sorting, tree testing, and any task requiring user experience research methodology or analysis.
Use this skill when planning user research, conducting usability tests, creating journey maps, or designing A/B experiments. Triggers on user interviews, usability testing, user journey maps, A/B test design, survey design, persona creation, card sorting, tree testing, and any task requiring user experience research methodology or analysis.
Research questions before methods - Define what decisions your research must
inform before choosing a method. "We will run interviews" is not a research plan.
"We need to understand why users abandon the checkout flow" is.
5 users find 80% of issues - Jakob Nielsen's landmark finding still holds for
formative usability testing. Recruit 5 representative participants per distinct
user segment. More sessions do not linearly increase insight - they surface the
same issues repeatedly.
Triangulate across methods - No single method answers everything. Pair
interviews (why) with analytics (how many) with usability tests (can they do it).
Convergent findings across methods are high-confidence findings.
Recruit representative users - Recruiting convenience samples (colleagues,
power users, friends) produces data that does not generalize. Screeners must filter
for the behaviors and contexts that match your target segment, not just demographics.
Synthesis is where value lives - Raw notes and recordings are not insights.
Value is created in the synthesis step: clustering observations into patterns,
naming themes, and connecting evidence to design implications. Budget as much time
for synthesis as for fieldwork.
Core concepts
Generative vs. evaluative research
Type
Goal
When to use
Example methods
Generative
Discover problems, needs, and opportunities
Early in a project, before solutions exist
User interviews, diary studies, contextual inquiry
Evaluative
Test whether a solution works for users
After a design exists, before or after launch
Usability tests, A/B tests, first-click tests
Running evaluative research too early (testing mockups of unvalidated concepts)
wastes cycles. Running generative research too late (interviewing users after building)
surfaces insights you cannot act on.
Qualitative vs. quantitative
Dimension
Qualitative
Quantitative
Question type
Why? How? What is the experience?
How many? How often? What percentage?
Sample size
5-20 participants
Hundreds to thousands
Output
Themes, quotes, behavioral patterns
Statistics, rates, significance
Risk
Hard to generalize; researcher bias
Misses "why" behind numbers
Neither is superior. Qualitative research generates hypotheses; quantitative research
tests them at scale.
Research ops
Research operations (ResearchOps) is the infrastructure that makes research repeatable:
participant panels, consent templates, recording tools, repositories, and synthesis
workflows. Without it, research knowledge lives in individual researchers' heads and
dissipates when they leave.
Bias types to mitigate
Bias
Description
Mitigation
Confirmation bias
Seeking evidence that supports existing beliefs
Define hypotheses before fieldwork; use a co-researcher to challenge interpretations
Leading bias
Questions that suggest the desired answer
Use open-ended, neutral phrasing; pilot-test your guide
Sampling bias
Participants who do not represent target users
Write behavioral screeners; recruit outside your network
Social desirability bias
Participants saying what they think you want to hear
Ask about past behavior, not hypothetical preferences; observe over asking
Recency bias
Over-weighting the last sessions in synthesis
Synthesize incrementally; weight all sessions equally
Common tasks
Plan a research study
Use this template before any study begins:
RESEARCH PLAN
=============
Project: [Name]
Date: [Start - End]
Researcher: [Name]
RESEARCH QUESTIONS
1. [Primary question the research must answer]
2. [Secondary questions]
DECISIONS THIS RESEARCH INFORMS
- [Specific product/design/business decision]
METHOD
[Selected method and why it fits the research questions]
PARTICIPANTS
- Target segment: [Description]
- Number: [N per segment]
- Screener criteria: [Behavioral criteria, not just demographics]
TIMELINE
- Recruiting: [Dates]
- Fieldwork: [Dates]
- Synthesis: [Dates]
- Share-out: [Date]
MATERIALS NEEDED
- [Discussion guide / task scenarios / prototype / survey link]
SUCCESS CRITERIA
[How will we know the research answered the questions?]
Conduct user interviews
Discussion guide structure:
Warm-up (5 min) - Rapport-building; ask about their role and context. Never start with your main topic.
Topic exploration (30-40 min) - Open-ended questions about behavior, not opinion.
Specific scenarios (10-15 min) - "Tell me about a time when..." to get concrete stories.
Wrap-up (5 min) - "Is there anything important I didn't ask about?"
Probing techniques:
Probe
When to use
Example
The silent probe
After a short answer; pause 3-5 seconds
(silence)
Echo probe
Repeat the last few words as a question
"You said it was confusing?"
Elaboration probe
When an answer needs depth
"Can you tell me more about that?"
Example probe
When an answer is abstract
"Can you give me a specific example?"
Clarification probe
When a term is ambiguous
"When you say 'complicated,' what do you mean?"
Impact probe
To understand consequences
"What happened as a result of that?"
Rules for interviewers:
Ask one question at a time. Never stack questions.
Never suggest an answer in the question.
Prioritize "what did you do?" over "what would you do?"
Take sparse notes during the session; full notes immediately after.
Run moderated usability tests
Task design rules:
Tasks must be scenario-based, not feature-based. "You want to send $50 to a friend" not "Use the transfer feature."
Tasks must have a clear, observable completion state.
Order tasks from low to high complexity.
Include one task you expect to fail - it will reveal the most.
Key metrics per task:
Metric
What it measures
How to collect
Task completion rate
Can users do it at all?
Binary success/failure per task
Time on task
Efficiency
Timer from task start to success
Error count
Where the design breaks down
Count distinct wrong paths taken
Satisfaction (SEQ)
Perceived ease
Single Ease Question (1-7 scale) after each task
Think-aloud protocol: Ask participants to narrate their thoughts while working.
Do not help them when they struggle - that is your signal. Only intervene if they are
completely stuck for more than 3 minutes.
Debrief questions:
"What was the most confusing part?"
"If you could change one thing, what would it be?"
"What did you expect to happen when you clicked X?"
Create user journey maps
Use this template for each journey:
JOURNEY MAP: [User goal / scenario]
=====================================
Persona: [Name and segment]
Scenario: [Context and starting point]
STAGES: [Awareness] → [Consideration] → [Decision] → [Use] → [Advocacy]
For each stage:
ACTIONS: What is the user doing?
THOUGHTS: What are they thinking?
EMOTIONS: [Frustrated / Neutral / Delighted] + why
TOUCHPOINTS: [Channel: website / app / email / support / etc.]
PAIN POINTS: What is going wrong or creating friction?
OPPORTUNITIES: Design interventions to improve this stage
Tips:
Base journeys on real research data, not assumptions. Every cell should be
traceable to a quote or observation.
Map the current-state journey before designing a future-state journey.
Emotion is the most actionable row - peaks and valleys show where to invest.
Design an A/B test
Hypothesis template:
We believe that [change to control]
will result in [expected outcome]
for [target user segment]
because [rationale from research or data].
Null hypothesis: There is no difference between control and variant.
Metrics:
Metric type
Examples
Notes
Primary
Conversion rate, task completion, sign-up
One metric only - the one the decision rests on
Guardrail
Revenue per user, support ticket rate
Must not degrade; test stops if they do
Secondary
Click-through rate, scroll depth
Directional signal; not decision criteria
Sample size calculation:
Before running any test, calculate the required sample size using:
Baseline conversion rate (from analytics)
Minimum detectable effect (MDE) - the smallest change worth acting on
Statistical power: 80% (standard)
Significance level: 95% (p < 0.05)
Use a sample size calculator (e.g., Evan Miller's). A common mistake is ending a
test as soon as significance is reached - this inflates false positives (peeking problem).
Set the duration before the test starts and do not stop early.
Duration rule: Run for at least one full business cycle (usually 2 weeks) to
capture weekly behavior variation, regardless of when significance is reached.
Synthesize findings with affinity mapping
Data dump - Write one observation per sticky note (physical or digital). Include a participant ID on each note.
Silent sort - Each team member groups notes without discussion.
Cluster and name - Groups become themes. Name themes as insights ("Users do not trust the price until they see a breakdown") not categories ("Pricing").
Count and rank - Note how many participants contributed to each theme. Themes supported by 4 of 5 participants are high-confidence.
Extract implications - For each theme, write: "This means we should consider [design implication]."
Write a research report
Template:
RESEARCH REPORT: [Study name]
==============================
Date: [Date]
Researcher: [Name]
Method: [Methods used]
Participants: [N, segment description]
EXECUTIVE SUMMARY (3-5 sentences)
[Most important finding and recommended action]
RESEARCH QUESTIONS
[Restate from the plan]
KEY FINDINGS
Finding 1: [Insight statement]
Evidence: [Quotes and observations]
Implication: [What this means for the product]
Finding 2: ...
RECOMMENDATIONS
Priority 1 (do now): [Specific action]
Priority 2 (consider): [Specific action]
Priority 3 (monitor): [Watch metric or re-research]
LIMITATIONS
[Sample size constraints, recruitment bias, prototype fidelity issues]
APPENDIX
- Discussion guide
- Participant screener
- Raw notes / recording links
Anti-patterns
Anti-pattern
Why it is wrong
What to do instead
Validating rather than learning
Designing research to confirm a decision already made; ignoring contradictory findings
Define what would change your mind before starting; share raw data with stakeholders
One-method thinking
Using only surveys or only interviews for everything
Match method to the research question; triangulate across methods
Recruiting power users
Power users have different mental models and error tolerance than average users
Write screeners that target typical usage frequency and context
Skipping synthesis
Sharing raw quotes and session recordings as "insights"
Cluster, theme, and interpret data; insights require analysis
Testing too late
Running usability tests after engineering is complete, when changes are expensive
Integrate research at every stage; paper prototypes are testable
Asking hypothetical questions
"Would you use a feature that..." elicits aspirational, inaccurate answers
Ask about past behavior: "Tell me about the last time you did X"
Gotchas
Stopping an A/B test when significance is first reached inflates false positive rate - This is the "peeking problem." With continuous monitoring, you will reach p<0.05 by chance on roughly 1 in 20 tests even when there is no real effect. Set the test duration before launch based on sample size calculation and do not stop early regardless of when significance is reached.
Usability test participants who are too polite produce misleading data - Many participants will complete tasks while struggling rather than say they are confused, to avoid seeming incompetent. Watch behavior (hesitation, wrong clicks, backtracking) more than verbal reports. Silence or slow movement is a signal; "yeah, that was fine" may not be.
Journey maps built from assumptions rather than data entrench existing beliefs - A journey map created in a workshop without participant quotes attached to each cell is a hypothesis map, not a research artifact. Every pain point and emotion in a journey map must be traceable to a specific observation or quote.
Survey questions with "usually" or "typically" elicit aspirational, not actual behavior - "How do you typically research products before buying?" invites respondents to describe their ideal selves. Ask about the last specific instance: "Think about the last time you bought something over $50 online. Walk me through what you did before purchasing." Specific past behavior is more accurate than general habits.
Recruiting from your own user base misses non-users and churned users - If you only recruit current active users, you systematically exclude people who tried and left, people who never signed up, and people in adjacent segments. For generative research, recruit from the broader target population, not just existing customers.
References
For detailed content on specific topics, read the relevant file from references/:
references/research-methods.md - Catalog of 15+ UX research methods with when-to-use, sample size, and effort level
Only load a references file if the current task requires deep detail on that topic.
Companion check
On first activation of this skill in a conversation: check which companion skills are installed by running ls ~/.claude/skills/ ~/.agent/skills/ ~/.agents/skills/ .claude/skills/ .agent/skills/ .agents/skills/ 2>/dev/null. Compare the results against the recommended_skills field in this file's frontmatter. For any that are missing, mention them once and offer to install:
