test-automation

name	test-automation
description	All testing layers — Vitest unit, Playwright E2E deep-dive (configs, page objects, debugging, traces), Cucumber BDD, mobile (Appium/Detox), API (Supertest/Newman), k6 load, CI/CD integration, credential management. Use when writing or fixing tests, setting up test infrastructure, chasing a flaky test, or establishing performance baselines. One skill for every layer — there is no second testing toolkit to consult.
category	Testing & Debug
status	active
license	MIT
version	2.0.0

Test Automation (QA Engineering)

The single testing reference for this kit — every layer, whether you are the app developer validating your own change or the QA engineer building reusable infrastructure. Unit (Vitest) · E2E (Playwright, canonical) · BDD (Cucumber) · mobile (Appium/Detox) · API (Supertest/Newman) · load (k6).

Pick the layer by what you are trying to learn, not by your job title: a function's logic → unit; a user's path through the app → E2E; a contract between services → API; behaviour under concurrency → load.

When to Use

• Writing or fixing unit tests for functions, components, utilities
• Setting up E2E test automation from scratch (web/mobile/API)
• Writing Playwright tests with page objects + advanced configs
• Implementing Cucumber/BDD scenarios with Gherkin
• Integrating tests with CI/CD pipelines (GitHub Actions, GitLab CI)
• Creating maintainable, flaky-resistant test suites
• Managing test credentials across local / CI / cloud
• Debugging test failures with traces, snapshots, codegen
• Establishing performance baselines and catching regressions under load

Do NOT use when: driving a browser for a long autonomous session (use [[agent-browser]]) · profiling a live page's runtime, network or Core Web Vitals (use [[chrome-devtools]]) · deriving which cases to test (use [[scenario]]) · practising red-green discipline on a bug fix (use [[tdd]]) · testing third-party SaaS behaviour you do not control.

Supported Frameworks

E2E Testing

• Playwright - Primary E2E framework (recommended)
• Cypress - Alternative E2E framework
• Puppeteer - Low-level browser automation

BDD/Behavior Testing

• Cucumber - Gherkin-based BDD
• SpecFlow - Cucumber for .NET

Mobile Testing

• Appium - Cross-platform mobile automation
• Detox - React Native E2E testing

API Testing

• Supertest - HTTP assertions
• Rest Assured - Java API testing
• Newman - Postman CLI

Unit Testing

• Vitest - Primary unit runner (Jest-compatible API, TypeScript-native, smart watch)
• Jest - Alternative; still the default in older repos
• xUnit - .NET services (see the csharp-developer skill)

Load Testing

• k6 - Scriptable load tests with thresholds as pass/fail gates

Unit Testing (Vitest)

The cheapest layer — reach for it first. Anything an E2E test could prove about a pure function, a unit test proves faster and points straight at the cause.

1. Configure — vitest.config.ts sets the environment (jsdom for browser APIs), coverage thresholds, reporters.
2. Write — .test.ts / .spec.ts colocated with the source file.
3. Run — npm run test; npm run test:watch re-runs only the tests affected by the files you touched.
4. Coverage — npm run test:coverage writes an HTML report to coverage/index.html.

// vitest.config.ts
export default defineConfig({
  test: {
    globals: true,
    environment: 'jsdom',
    coverage: {
      provider: 'v8',
      reporter: ['text', 'html'],
      include: ['src/**/*.ts'],
      exclude: ['src/**/*.test.ts'],
    },
  },
});

Playwright Quick Start

Installation

npm init playwright@latest
# OR
npm install -D @playwright/test playwright
npx playwright install --with-deps

Basic Test Structure

import { test, expect } from '@playwright/test';

test.describe('Login Feature', () => {
  test('should login with valid credentials', async ({ page }) => {
    await page.goto('/login');
    await page.fill('[data-testid="email"]', '[e-mail]');
    await page.fill('[data-testid="password"]', 'password123');
    await page.click('[data-testid="login-button"]');
    
    await expect(page).toHaveURL('/dashboard');
    await expect(page.locator('[data-testid="user-name"]')).toBeVisible();
  });
  
  test('should show error with invalid credentials', async ({ page }) => {
    await page.goto('/login');
    await page.fill('[data-testid="email"]', '[e-mail]');
    await page.fill('[data-testid="password"]', 'wrongpassword');
    await page.click();
    
     (page.())
      .();
  });
});

Playwright Configuration

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
    {
      name: 'Mobile Chrome',
      use: { ...devices['Pixel 5'] },
    },
  ],
  webServer: {
    : ,
    : ,
    : !process..,
  },
});

Cucumber/BDD Quick Start

Installation

npm install -D @cucumber/cucumber @playwright/test

Feature File

Feature: Login functionality

  Scenario: Successful login with valid credentials
    Given the user is on the login page
    When the user enters "[e-mail]" in the email field
    And the user enters "password123" in the password field
    And the user clicks the login button
    Then the user should be redirected to the dashboard
    And the user should see their username

  Scenario: Failed login with invalid credentials
    Given the user is on the login page
    When the user enters "[e-mail]" in the email field
    And the user enters "wrongpassword" in the password field
    And the user clicks the login button
    Then the user should see an error message

Step Definitions

import { Given, When, Then } from '@cucumber/cucumber';
import { expect, Page } from '@playwright/test';

let page: Page;

Given('the user is on the login page', async function() {
  await page.goto('/login');
});

When('the user enters {string} in the email field', async function(email: string) {
  await page.fill('[data-testid="email"]', email);
});

When('the user enters {string} in the password field', async function(password: string) {
  await page.fill('[data-testid="password"]', password);
});

When('the user clicks the login button', async function() {
  await page.click('[data-testid="login-button"]');
});

Then('the user should be redirected to the dashboard',  () {
   (page).();
});

(,  () {
   (page.()).();
});

Best Practices

1. Test Data Management

// Use fixtures for test data
test.use({
  testData: {
    user: { email: '[e-mail]', password: 'password123' },
    admin: { email: '[e-mail]', password: 'admin123' },
  },
});

// In tests
test('admin can access settings', async ({ page }) => {
  const admin = test.config.testData.admin;
  await page.login(admin.email, admin.password);
});

2. Page Object Model

// pages/LoginPage.ts
import { Page, Locator } from '@playwright/test';

export class LoginPage {
  readonly page: Page;
  readonly emailInput: Locator;
  readonly passwordInput: Locator;
  readonly loginButton: Locator;
  readonly errorMessage: Locator;

  constructor(page: Page) {
    this.page = page;
    this.emailInput = page.locator('[data-testid="email"]');
    this.passwordInput = page.locator('[data-testid="password"]');
    this.loginButton = page.locator('[data-testid="login-button"]');
    this.errorMessage = page.locator('[data-testid="error-message"]');
  }

  async goto() {
    await this..();
  }

   () {
     ..(email);
     ..(password);
     ..();
  }
}


 {  }  ;

(,  ({ page }) => {
   loginPage =  (page);
   loginPage.();
   loginPage.(, );
});

3. Handle Dynamic Content

// Wait for elements properly
await page.waitForSelector('[data-testid="loaded-content"]');
await expect(page.locator('[data-testid="spinner"]')).toBeHidden();

// Handle navigation
await page.click('[data-testid="submit"]');
await page.waitForURL('/success');

// Handle dialogs
page.on('dialog', async dialog => {
  await dialog.accept();
});

4. Reduce Flakiness

// Use locators that don't change
// ✅ Good: data-testid, role, text
await page.click('[data-testid="submit-button"]');
await page.getByRole('button', { name: 'Submit' });

// ❌ Avoid: indices, partial text
await page.locator('button').nth(0);
await page.locator('button:has-text("Sub")');

// Use proper waits instead of sleep
await page.waitForLoadState('networkidle');
await expect(locator).toBeVisible({ timeout: 10000 });

5. Parallel Execution

// Playwright runs tests in parallel by default
// Configure in playwright.config.ts
export default defineConfig({
  fullyParallel: true,
  workers: process.env.CI ? 1 : 4, // Limit on CI
});

CI/CD Integration

GitHub Actions

# .github/workflows/test.yml
name: E2E Tests

on: [push, pull_request]

jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Install Playwright browsers
        run: npx playwright install --with-deps
      
      - name: Run tests
        run: npx playwright test
      
      - name: Upload test results
        if: 
         
        
           
           

Run Specific Tests

# Run by tag
npx playwright test --grep @smoke
npx playwright test --grep @regression

# Run by file pattern
npx playwright test "tests/login/**"
npx playwright test "tests/login/login.spec.ts"

# Run specific suite
npx playwright test --project=chromium

Test Maintenance

Debug Failed Tests

# Interactive mode
npx playwright test --debug

# UI mode
npx playwright test --ui

# With trace
npx playwright test --trace on
# View trace: npx playwright show-trace trace.zip

Update Tests After UI Changes

# Update snapshots
npx playwright test --update-snapshots

# Codegen new selectors
npx playwright codegen

Test Reports

# HTML report
npx playwright show-report

# JSON report for CI
npx playwright test --reporter=json

Load Testing (k6)

1. Write the script — virtual users, request pattern, and thresholds. Put the thresholds in the script: they are the pass/fail gate, not a number someone eyeballs in the output.
2. Run locally — k6 run script.js prints throughput, p95 latency, error rate.
3. Scale — raise vus and ramp duration gradually. A cold jump to peak load measures the ramp, not the system.
4. Compare against a baseline — a load run with nothing to compare to cannot detect a regression. Record the numbers.

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  vus: 10,
  duration: '30s',
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.1'],
  },
};

export default function () {
  const res = http.get('https://api.example.com/data');
  check(res, { 'status is 200': (r) => r.status === 200 });
  sleep(1);
}

Command Cheat-Sheet

# Unit: run all, watch only what changed
npm run test
npm run test:watch
npm run test:coverage            # HTML report at coverage/index.html

# E2E: one spec, headed / Inspector / UI mode
npx playwright test tests/e2e/login.spec.ts --headed
npx playwright test tests/e2e/checkout.spec.ts --debug
npx playwright test --ui
npx playwright test --trace on    # then: npx playwright show-trace trace.zip

# Load: 50 users, 5-minute ramp
k6 run -e USERS=50 -e RAMP_UP=5m load-test.js

Common Pitfalls

• Over-testing in E2E — E2E covers user workflows, not every button. If a unit test can prove it, the E2E test is slower and flakier for no gain.
• Brittle selectors — CSS/XPath break on layout changes. Use data-testid.
• Non-deterministic waits — never waitForTimeout(ms). Use waitForSelector / expect(...).toBeVisible().
• Failures without context — "test failed" alone is useless. Capture traces, screenshots, console logs (--trace on).
• Load-test spikes — too short a ramp measures the ramp, not steady state.
• No baseline metrics — without recorded numbers, no regression is detectable.

Resources

• Playwright Docs: https://playwright.dev/docs/intro
• Vitest Docs: https://vitest.dev/
• Grafana k6 Docs: https://grafana.com/docs/k6/latest/
• Cucumber Docs: https://cucumber.io/docs/
• Playwright Best Practices: https://playwright.dev/docs/best-practices
• MS Playwright Testing: https://learn.microsoft.com/en-us/playwright/

Credential Management

See: references/credential-management.md for detailed guide on storing and managing test credentials across local, CI/CD, and cloud environments.

project/
├── .env                    # Root credentials - NEVER commit
├── .env.example            # Template - safe to commit
├── .gitignore             # Must include .env
└── tests/
    ├── .env              # Test-specific credentials
    └── fixtures/
        └── accounts.json # Encrypted or gitignored

Environment Variables Setup

1. Create .env file:

# .env - Local development (DO NOT COMMIT)
TEST_USER=[e-mail]
TEST_PASS=SecurePassword123
API_KEY=sk_test_xxx
ADMIN_ID=12345

# Staging environment
STAGING_URL=https://staging.example.com
STAGING_USER=[e-mail]
STAGING_PASS=staging_pass

2. Create .env.example (template - commit):

# .env.example - Copy this and fill in your values
TEST_USER=
TEST_PASS=
API_KEY=
ADMIN_ID=

3. Add to .gitignore:

.env
.env.local
.env.*.local
tests/.env
tests/fixtures/credentials.json
playwright/.auth/

Playwright Config with Credentials

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import dotenv from 'dotenv';

// Load environment variables
dotenv.config({ path: '.env.test' });

export default defineConfig({
  use: {
    // Use environment variables in tests
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
  },
  // Pass credentials to tests via environment
});

Using Credentials in Tests

import { test, expect } from '@playwright/test';

test('login with stored credentials', async ({ page }) => {
  // Credentials from environment variables
  const user = process.env.TEST_USER;
  const pass = process.env.TEST_PASS;
  
  await page.goto('/login');
  await page.fill('[data-testid="email"]', user);
  await page.fill('[data-testid="password"]', pass);
  await page.click('[data-testid="login-button"]');
});

test('admin operations with stored ID', async ({ page }) => {
  const adminId = process.env.ADMIN_ID;
  
  // Navigate to admin with stored ID
  await page.goto(`/admin/users/${adminId}`);
});

CI/CD Integration

GitHub Actions:

# .github/workflows/test.yml
jobs:
  e2e:
    steps:
      - name: Inject credentials
        run: |
          echo "TEST_USER=${{ secrets.TEST_USER }}" >> $GITHUB_ENV
          echo "TEST_PASS=${{ secrets.TEST_PASS }}" >> $GITHUB_ENV
          echo "ADMIN_ID=${{ secrets.ADMIN_ID }}" >> $GITHUB_ENV
      
      - name: Run tests
        run: npx playwright test

GitLab CI:

test:
  variables:
    TEST_USER: $TEST_USER
    TEST_PASS: $TEST_PASS
  script:
    - npx playwright test

User-Provided Credentials Workflow

When running tests that require user credentials:

Step 1: Check for stored credentials

# Check if credentials exist
if [ -f .env ]; then
  echo "Using stored credentials"
else
  echo "No stored credentials found"
fi

Step 2: Prompt user (if not found)

echo "Enter test credentials (will be saved for next run):"
read -p "Test User: " USER
read -s -p "Test Password: " PASS
read -p "Admin ID: " ADMIN_ID

# Save to .env for next run
echo "TEST_USER=$USER" > .env
echo "TEST_PASS=$PASS" >> .env
echo "ADMIN_ID=$ADMIN_ID" >> .env

echo "✅ Credentials saved to .env for future runs"

Step 3: Use in test automation

// Auto-load from .env if exists
import dotenv from 'dotenv';
import path from 'path';

const envPath = path.resolve(__dirname, '../../.env');
if (require('fs').existsSync(envPath)) {
  dotenv.config({ path: envPath });
}

Security Best Practices

1. NEVER commit credentials - Add to .gitignore
2. Use different accounts - Separate test accounts from production
3. Rotate credentials - Update periodically
4. Limit permissions - Test accounts should have minimal access
5. Use secrets manager - For enterprise, use AWS Secrets Manager, etc.
6. Encrypt at rest - If storing locally, consider encrypting

Regenerate/Update Credentials

# When credentials expire or change
# 1. Ask user for new credentials
# 2. Update .env file
# 3. Run login test to verify
npx playwright test tests/auth/login.spec.ts

# 4. Re-authenticate if session expired
# Delete old auth state
rm -rf playwright/.auth/
# Run tests to create new session
npx playwright test

---

Integration with Tester Agent

When the tester agent runs tests:

1. This skill covers every layer — unit, E2E, BDD, API, load. There is no second testing skill to consult.
2. Start at the cheapest layer that can prove the claim; escalate only as needed.
3. Prioritize Playwright over Cypress (better cross-browser)
4. Use BDD for acceptance criteria tests
5. Always include data-testid attributes in development code
6. On a failure, root-cause it with the debugging skill before changing the test — a test edited until it passes proves nothing.
7. Check for stored credentials in .env before prompting user