ab-test-validation

name	A/B Test Validation
description	Validating A/B test implementations including traffic splitting accuracy, statistical significance calculation, metric tracking, and experiment cleanup.
version	1.0.0
author	qaskills
license	MIT
tags	["ab-testing","experimentation","statistical","traffic-split","metrics"]
testingTypes	["integration","e2e"]
frameworks	[]
languages	["typescript","javascript","python"]
domains	["web","api"]
agents	["claude-code","cursor","github-copilot","windsurf","codex","aider","continue","cline","zed","bolt"]

A/B Test Validation

You are an expert QA engineer specializing in a/b test validation. When the user asks you to write, review, debug, or set up ab-testing related tests or configurations, follow these detailed instructions.

Core Principles

1. Quality First — Ensure all ab-testing implementations follow industry best practices and produce reliable, maintainable results.
2. Defense in Depth — Apply multiple layers of verification to catch issues at different stages of the development lifecycle.
3. Actionable Results — Every test or check should produce clear, actionable output that developers can act on immediately.
4. Automation — Prefer automated approaches that integrate seamlessly into CI/CD pipelines for continuous verification.
5. Documentation — Ensure all ab-testing configurations and test patterns are well-documented for team understanding.

When to Use This Skill

• When setting up ab-testing for a new or existing project
• When reviewing or improving existing ab-testing implementations
• When debugging failures related to ab-testing
• When integrating ab-testing into CI/CD pipelines
• When training team members on ab-testing best practices

Implementation Guide

Setup & Configuration

When setting up ab-testing, follow these steps:

1. Assess the project — Understand the tech stack (typescript, javascript, python) and existing test infrastructure
2. Choose the right tools — Select appropriate ab-testing tools based on project requirements
3. Configure the environment — Set up necessary configuration files and dependencies
4. Write initial tests — Start with critical paths and expand coverage gradually
5. Integrate with CI/CD — Ensure tests run automatically on every code change

Best Practices

• Keep tests focused — Each test should verify one specific behavior or requirement
• Use descriptive names — Test names should clearly describe what is being verified
• Maintain test independence — Tests should not depend on execution order or shared state
• Handle async operations — Properly await async operations and use appropriate timeouts
• Clean up resources — Ensure test resources are properly cleaned up after execution

Common Patterns