ab-testing

name	ab-testing
description	Set up and evaluate A/B tests using feature-flag-server. Use when you need to run a multivariate experiment, assign users to variants, track which variant they received, or analyze variant distribution. Triggers include "A/B test", "split test", "experiment", "variant", "multivariate", "rollout percentage", or any task involving exposing different experiences to different users.

ab-testing

Run A/B tests and multivariate experiments using feature-flag-server segment flags.

When to use

• Running a two-variant A/B test (control vs treatment)
• Multivariate tests with more than two variants
• Gradual feature rollout with percentage-based targeting
• Segment-targeted experiments (e.g., only beta users see the variant)

Flag types for experiments

Type	Use case
percentage	Random percentage split (e.g., 50% see the feature)
segment	Target specific user groups with different variants

For A/B tests, use a segment flag with weighted variants. This gives full control over the variant split.

Creating an A/B test via REST API

# Step 1: Create the flag
curl -s -X POST http://localhost:7777/api/flags \
  -H "Authorization: Bearer ffs_..." \
  -H "Content-Type: application/json" \
  -d '{
    "key": "button-color-test",
    "name": "Button Color A/B Test",
    "flagType": "segment"
  }'
# Response: { "id": "550e8400-e29b-...", "key": "button-color-test", ... }

# Step 2: Configure variants and weights for production
curl -s -X PATCH http://localhost:7777/api/flags/550e8400-e29b-.../environments/production \
  -H "Authorization: Bearer ffs_..." \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": 1,
    "variants": [
      { "key": "control",   "value": "#3B82F6", "weight": 50 },
      { "key": "variant-a", "value": "#10B981", "weight": 30 },
      { "key": "variant-b", "value": "#F59E0B", "weight": 20 }
    ]
  }'

Note: variant weights must sum to 100.

Evaluating a variant (SDK)

import { FlagClient } from '@ffs/js-sdk';

const client = new FlagClient({
  baseUrl: 'http://localhost:7777',
  apiKey: process.env.FFS_API_KEY!,
  environment: 'production',
});

const result = await client.getVariant('button-color-test', {
  userId: req.user.id,
  plan: req.user.plan,
  country: req.user.country,
});

// result.value   => "#10B981"      (the variant's value)
// result.variant => "variant-a"   (the variant key)
// result.reason  => "segment_match"

// Use the value in your UI
const buttonStyle = { backgroundColor: result.value };

Checking variant distribution via API

curl -s http://localhost:7777/api/analytics/flags/550e8400-e29b-.../variants \
  -H "Authorization: Bearer ffs_..."
# Response:
# {
#   "total": 111432,
#   "variants": [
#     { "key": "control",   "count": 55716, "pct": 50.0 },
#     { "key": "variant-a", "count": 33430, "pct": 30.0 },
#     { "key": "variant-b", "count": 22286, "pct": 20.0 }
#   ],
#   "periodDays": 7
# }

Percentage rollout (gradual launch)

For simple gradual rollouts without variants, use a percentage flag:

# Create percentage flag, start at 0%
curl -s -X POST http://localhost:7777/api/flags \
  -H "Authorization: Bearer ffs_..." \
  -H "Content-Type: application/json" \
  -d '{ "key": "feature-x-rollout", "name": "Feature X Rollout", "flagType": "percentage" }'

# Enable at 10% in production
curl -s -X PATCH http://localhost:7777/api/flags/<id>/environments/production \
  -H "Authorization: Bearer ffs_..." \
  -H "Content-Type: application/json" \
  -d '{ "enabled": 1, "rolloutPct": 10 }'

# Ramp to 50%
curl -s -X PATCH http://localhost:7777/api/flags/<id>/environments/production \
  -H "Authorization: Bearer ffs_..." \
  -H "Content-Type: application/json" \
  -d '{ "rolloutPct": 50 }'

Evaluating a percentage flag returns true if the user is within the rollout, false otherwise. The assignment is deterministic and stable for each user.

Segment-targeted experiments

Target a variant only at specific users by combining a segment with a flag:

1. Create a segment: POST /api/segments with rules (e.g., user.plan eq "enterprise")
2. Create a segment flag
3. Add a rule that assigns variant-a to users matching the segment
4. Set a default variant (returned for users who do not match any rule)

{
  "rules": [
    {
      "segmentId": "segment-uuid-here",
      "variant": "variant-a"
    }
  ],
  "defaultVariant": "control"
}

Behavior

• Determinism: The same userId + flagKey combination always produces the same variant. Changing the flag weights may re-assign some users.
• Context isolation: Context values are only used for segment rule matching. They are not stored or logged in full.
• Audit trail: Every variant weight or rule change is recorded in the audit log with a diff.