experimentation

name	experimentation
description	Canonical experimentation and CRO layer. Defines the hypothesis template, statistical-significance primer, A/B infrastructure contract, and quarterly review cadence that turn retainer work into a structured learning loop rather than opinion exchange. Every retainer client runs at least one test per quarter; every test has a documented hypothesis and a decision rule.

Experimentation

Acknowledgement: Shared by Peter Bamuhigire, techguypeter.com, [phone].

Use when

• A retainer client has a conversion, engagement, or retention question that cannot be answered by existing analytics alone.
• A strategist or SEO operator wants to test a hypothesis before rolling a change across a site.
• A client has shipped a major redesign and needs the "did it work" question answered with evidence, not opinion.

Do not use when

• The team has not agreed on the metric. Run ux-psychology or cro-audit to shape the question first.
• The site has insufficient traffic. Under ~1,000 primary-CTA events per week, experimentation power is too low to beat noise; use qualitative methods instead.
• The change is a bug fix. Fixes ship without experimentation.

Core contract

Every experiment honours five properties:

1. Documented hypothesis using references/hypothesis-template.md.
2. Minimum detectable effect computed before start, not after.
3. Stopping rule declared before the test begins — sample-size or calendar-date based.
4. Guardrail metrics named so a winner on the primary metric that harms a guardrail still loses.
5. Decision rule written before the test — how the operator decides to ship, kill, or iterate regardless of result.

A test without these five is not an experiment; it is opinion with numbers attached.