experiment-platform

name	experiment-platform
description	Build an internal experimentation platform for running A/B tests at scale. Outputs experiment service architecture, assignment engine, metric pipeline, and statistical analysis framework.
argument-hint	["traffic volume","number of concurrent experiments","team size","existing analytics stack"]
allowed-tools	Read, Write

Experiment Platform

An experiment platform standardises how A/B tests are run across the organisation — experiment definition, user assignment, metric collection, and statistical analysis. Without a platform, every team implements its own experiment logic inconsistently. With one, experiments are faster, more reliable, and more trustworthy.

Architecture

Experiment Service (define + assign)
  ├── Experiment Registry (what experiments exist, who is in them)
  ├── Assignment Engine (deterministic bucketing by user_id)
  └── Exposure Logging (record when user saw the experiment)

Metric Pipeline
  ├── Event collection (existing analytics pipeline)
  ├── Metric computation (joins exposure log with events)
  └── Results store (pre-computed stats per experiment)

Analysis Service
  ├── Statistical tests (t-test, z-test, sequential)
  ├── Segment breakdowns
  └── Results API (for dashboard)

Assignment Engine

import hashlib
from dataclasses import dataclass

@dataclass
 :
    : 
    name: 
    variants: []  
    status:            
    targeting:        

 :
     () ->  | :
        
         experiment.status != :
             

        
        hash_input = 
        bucket = (hashlib.md5(hash_input.encode()).hexdigest(), ) % 

        
        cumulative = 
         variant  experiment.variants:
            cumulative += variant[]
             bucket < cumulative:
                 variant[]

           

     () -> :
        
        targeting = experiment.targeting
           targeting  user.get()   targeting[]:
             
           targeting  user.get()   targeting[]: