| name | ab-test-design-rituals |
| description | Use when designing A/B tests. Covers hypothesis-driven testing, sample-size calculation, the test-design template, and the cultural rituals that make testing productive.
|
Ab Test Design Rituals
A/B tests answer questions. Run a test without a clear question and you'll have a clear non-answer.
Hypothesis template
We believe [change] will result in [outcome] for [audience], measured by [metric]. We'll know we're right if [primary metric] improves by [X%] within [Y days].
Sample size
Use a calculator (e.g., evanmiller.org). 5% baseline rate + 10% relative lift = ~7,000 visitors per arm.
Test duration
Minimum 1 week (captures day-of-week variation). Don't peek and stop early.
Test types
A/B (two variants). A/B/n (multiple variants — more samples required). Multivariate (combinations — even more).
Common mistakes
Stopping early when 'looks good'. Multiple metrics without primary. Underpowered tests (false negatives). 'Winning' variant that doesn't ship (no cultural follow-through).
Where this fits in X3 Compass
Applied across the X3 Compass design system.