full-api-test

skill	yaml-scenarios
version	1.0.0
trigger	user asks to write, edit, or validate a load test scenario YAML file

yaml-scenarios Skill

When to Use

Use this skill when the user wants to:

• Write a new YAML scenario file for load-test-runner
• Add or modify request steps in an existing scenario
• Configure thresholds for pass/fail determination
• Use template variables in headers or request bodies
• Validate scenario YAML without running it
• Understand which fields are required vs optional

Scenario Structure

A scenario YAML file has four top-level sections:

1. Run parameters - name, target, ramp_up_seconds, duration_seconds, max_vus
2. Steps - the request steps executed by virtual users
3. Thresholds - pass/fail criteria evaluated after the run
4. Variables (optional) - static values substituted into step fields

Complete Example with All Fields

name: Full API Test
description: Comprehensive scenario showing all available fields
target: https://api.example.com

ramp_up_seconds: 60
duration_seconds: 300
max_vus: 100

variables:
  TOKEN: abc123
  USER_ID: 42

steps:
  # Simple GET
  - name: Health check
    method: GET
    path: /health
    expected_status: 200
    weight: 5

  # GET with auth header using variable
  - name: Get user profile
    method: GET
    path: /users/${USER_ID}
    headers:
      Authorization: Bearer ${TOKEN}
    expected_status: 200
    weight: 20

  # POST with JSON body
  - name: Create order
    method: POST
    path: /orders
    headers:
      Authorization: Bearer ${TOKEN}
      Content-Type: application/json
    body: '{"user_id": ${USER_ID}, "item": "widget"}'
    expected_status: 201
    weight: 30

  # PUT
  - name: Update order status
    method: PUT
    path: /orders/1/status
    headers:
      Authorization: Bearer ${TOKEN}
      Content-Type: application/json
    body: '{"status": "shipped"}'
    expected_status: 200
    weight: 10

  # DELETE
  - name: Delete draft order
    method: DELETE
    path: /orders/draft
    headers:
      Authorization: Bearer ${TOKEN}
    expected_status: 204
    weight: 5

thresholds:
  p95_response_ms: 300
  error_rate: 0.005
  rps_min: 200

Step Weight Explained

Weights determine how often each step is executed relative to others. Virtual users pick a step randomly, weighted by these values.

weight: 10  ->  10 / (10+40+30) = 12.5% of requests
weight: 40  ->  40 / (10+40+30) = 50.0% of requests
weight: 30  ->  30 / (10+40+30) = 37.5% of requests

If all steps have equal weight (or weight is omitted), they are distributed evenly.

Template Variables

Variables defined under variables: are substituted using ${VAR_NAME} syntax in:

• path
• body
• headers values

Variables can also be set from environment variables at run time. The server substitutes env vars for any variable not defined in the YAML:

variables:
  TOKEN: default-value   # overridden by TOKEN env var if present

Runtime override via CLI (planned):

load-test run ./scenario.yaml --var TOKEN=my-token --var USER_ID=99

Thresholds Reference

Thresholds are evaluated after the run completes (or at abort). All defined thresholds must pass for the overall run to be marked PASS.

Threshold	Type	Description	Example
p95_response_ms	integer	p95 latency must be below this value (milliseconds)	500
error_rate	float	Error rate must be below this fraction	0.01 (= 1%)
rps_min	number	Average RPS must be at or above this value	100

Any subset of thresholds can be defined. A scenario with no thresholds always passes.

Validation Rules

The scenario is validated before a run starts. Validation errors:

Rule	Error
name missing	name is required
target missing or not a URL	target must be a valid URL
ramp_up_seconds >= duration_seconds	ramp_up_seconds must be less than duration_seconds
max_vus < 1	max_vus must be at least 1
steps empty	at least one step is required
step method not in allowed set	method must be GET, POST, PUT, PATCH, or DELETE
step path does not start with /	path must start with /
step weight < 0	weight must be a positive integer
expected_status not a valid HTTP status	expected_status must be a valid HTTP status code
error_rate threshold > 1.0	error_rate must be between 0 and 1

Run validation without executing:

load-test validate ./api-smoke.yaml

Common Scenario Patterns

Smoke test (quick sanity check)

ramp_up_seconds: 10
duration_seconds: 60
max_vus: 10

Load test (realistic production traffic)

ramp_up_seconds: 60
duration_seconds: 300
max_vus: 100

Stress test (find breaking point)

ramp_up_seconds: 120
duration_seconds: 600
max_vus: 500

Soak test (sustained load over time)

ramp_up_seconds: 60
duration_seconds: 3600
max_vus: 50

Validate a File

# Validate syntax and schema
load-test validate ./api-smoke.yaml

# Example output
ok  scenario   API Smoke Test
ok  target     https://api.example.com
ok  steps      3 steps defined
ok  thresholds 3 thresholds defined
ok  VALID

# Example failure output
FAIL  steps[1].path must start with /
FAIL  ramp_up_seconds (120) must be less than duration_seconds (60)
2 validation errors