Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
Design and analyze A/B tests and controlled experiments for data-driven decisions. Use for: experiment design, hypothesis formulation, sample size calculation, statistical significance testing, multivariate testing, test implementation, results analysis, avoiding common pitfalls, sequential testing, and experimentation platforms.
A/B Testing
Design and analyze controlled experiments to make data-driven decisions about product changes and optimizations.
Overview
A/B testing (split testing) is a rigorous method for comparing two or more variants to determine which performs better on a defined metric. This skill covers statistical foundations, experimental design principles, implementation strategies, and analysis techniques for running effective experiments.
A/B Testing Fundamentals
Core Concepts
Control (A): Current version or baseline
Variant (B): New version being tested
Randomization: Users randomly assigned to control or variant
Metric: Quantifiable measure of success (conversion rate, revenue, engagement)
Statistical Significance: Confidence that difference is not due to chance
When to Use A/B Testing
Good Use Cases:
Testing website/app design changes
Optimizing email subject lines or content
Comparing pricing strategies
Evaluating feature changes
Testing marketing messages
Not Suitable For:
Low-traffic pages (insufficient sample size)
Long-term strategic decisions
Understanding "why" (use qualitative research)
Testing many changes simultaneously (use multivariate testing)
Experimental Design
Hypothesis Formulation
Good Hypothesis:
Specific: "Changing button color from blue to green will increase click-through rate"
Measurable: Clear metric (CTR)
Relevant: Tied to business goal
Falsifiable: Can be proven wrong
Bad Hypothesis:
Vague: "New design will be better"
Unmeasurable: "Users will like it more"
Multiple changes: "New layout, colors, and copy will improve conversions"
Simpson's Paradox: Overall result may differ from segment results
Best Practices
Before Testing
Define clear hypothesis and success metrics
Calculate required sample size
Set up proper tracking
Document experiment design
Get stakeholder alignment
During Testing
Don't peek at results (or use sequential methods)
Monitor for technical issues
Ensure balanced traffic allocation
Run for predetermined duration
After Testing
Analyze primary and secondary metrics
Check for segment differences
Document learnings
Communicate results clearly
Implement winning variant or iterate
Using the Reference Files
When to Read Each Reference
/references/statistical-methods.md — Read when calculating sample sizes, choosing statistical tests, or understanding p-values and confidence intervals.
/references/experiment-design-patterns.md — Read when designing complex experiments, implementing multivariate tests, or using advanced techniques like CUPED.
/references/common-mistakes.md — Read when troubleshooting experiments, avoiding pitfalls, or understanding why results may be misleading.
