e2e-playwright-runbook

name	e2e-playwright-runbook
description	Playwright E2E testing for Volatio using e2e.sh script, timeout handling, port zombies, lock files, storageState auth, test projects. Use when running E2E tests, debugging flaky tests, or auth issues.

e2e-playwright-runbook

⛔ CRITICAL: PREVENT HANGS

E2E tests WILL hang forever without correct flags. ALWAYS use:

# ✅ CORRECT - exits cleanly
./scripts/e2e.sh --reporter=list
./scripts/e2e.sh e2e/my-test.spec.ts --reporter=list --timeout=60000

# ❌ HANGS - opens blocking HTML server, no terminal output
./scripts/e2e.sh
pnpm exec playwright test

If you see no output for 30+ seconds, it's hung. Kill with Ctrl+C and add --reporter=list.

When to Use

• Running E2E tests
• Debugging flaky or failing tests
• Auth/login test issues
• Tests hanging or timing out
• Port conflicts or zombie processes

Golden Path

ALWAYS use e2e.sh script - never run Playwright directly.

# Run all E2E tests (use --reporter=list to avoid HTML server blocking)
./scripts/e2e.sh --reporter=list

# Run specific test file with timeout
./scripts/e2e.sh e2e/strategies.spec.ts --timeout=60000 --reporter=list

# Run specific project
./scripts/e2e.sh --project=auth-tests --reporter=list

# Run tests matching a pattern
./scripts/e2e.sh --grep="unauthenticated" --reporter=list

CRITICAL: Use --reporter=list

The default HTML reporter opens a blocking server that requires Ctrl+C:

# ❌ BLOCKS - opens server at http://localhost:9323
./scripts/e2e.sh

# ✅ NON-BLOCKING - outputs results and exits
./scripts/e2e.sh --reporter=list

What e2e.sh Does Automatically

1. Starts Docker services (postgres, redis)
2. Waits for PostgreSQL to be ready
3. Runs database migrations
4. Seeds admin test user ([e-mail] / Trading2024!)
5. Kills zombie processes on E2E ports (3100, 3101)
6. Starts API on port 3101
7. Starts Dashboard on port 3100, warms up Next.js
8. Runs Playwright tests with correct env vars
9. Cleans up servers on exit

CRITICAL: Always Use Timeouts

Tests can hang forever without timeouts:

# ✅ ALWAYS use --timeout flag
./scripts/e2e.sh e2e/my-test.spec.ts --timeout=60000 --reporter=list

# ❌ NEVER run without timeout protection
pnpm exec playwright test  # Wrong!

Test Projects

Project	Description	Auth State
chromium	Most tests (default)	Pre-authenticated via storageState
auth-tests	Login/logout/redirect	No saved auth
setup	Auth setup (runs first)	Creates storageState

When to Use --project=auth-tests

• Testing login page behavior
• Testing unauthenticated redirect behavior
• Testing logout flows
• Any test needing fresh browser without cookies

Port Allocation

Service	Dev Port	E2E Port
Dashboard	3000	3100
API	3001	3101

Writing E2E Tests: Auth for API Requests

Tests that make API calls need auth cookies. The chromium project provides storageState for page navigation, but API requests via request fixture need explicit Cookie headers.

import * as fs from 'node:fs';
import * as path from 'node:path';

const AUTH_FILE = path.join(__dirname, '.auth/user.json');

// Helper to get auth cookie from storageState
function getAuthCookieHeader(): string {
  try {
    const storageState = JSON.parse(fs.readFileSync(AUTH_FILE, 'utf-8'));
    const sessionCookie = storageState.cookies.find(
      (c: { name: string }) => c.name === 'authjs.session-token',
    );
    if (!sessionCookie) return '';
    return `authjs.session-token=${sessionCookie.value}`;
  } catch {
    return '';
  }
}

// Usage in tests:
test.beforeAll(() => {
  authCookie = getAuthCookieHeader();
});

test('makes authenticated API call', async ({ request }) => {
  const response = await request.post(`${API_BASE}/v1/paper/sessions`, {
    headers: { Cookie: authCookie },  // ← Required!
    data: { name: 'Test', startingBalance: 10000 },
  });
});

API Response Format

Many API endpoints return wrapped responses:

// ❌ Wrong - assumes array directly
const strategies = await response.json();
strategies.find(...)  // TypeError if response is {strategies: [...]}

// ✅ Correct - handle both formats
const data = await response.json();
const strategies = data.strategies || data;
strategies.find(...)

Troubleshooting

⛔ NEVER Use waitForLoadState('networkidle')

The app uses SSE (Server-Sent Events) connections that NEVER close. networkidle waits for all network activity to stop, which never happens with SSE.

// ❌ TIMES OUT AFTER 30s - SSE keeps network active forever
await page.waitForLoadState('networkidle');

// ✅ Wait for specific elements instead
await expect(page.getByRole('button', { name: 'New Session' })).toBeVisible();
await page.waitForTimeout(2000);  // Simple delay if needed

⛔ Playwright Strict Mode - Locators Must Match Exactly 1 Element

When using .or() or broad locators, Playwright's strict mode fails if multiple elements match:

// ❌ FAILS if BOTH button AND skeleton exist (strict mode violation)
await expect(
  page.getByRole('button', { name: /Submit/i }).or(page.locator('.loading'))
).toBeVisible();

// ✅ Check each separately - avoids strict mode issues
const hasButton = await page.getByRole('button', { name: /Submit/i }).isVisible().catch(() => false);
const hasLoading = await page.locator('.loading').isVisible().catch(() => false);
expect(hasButton || hasLoading).toBe(true);

Lock File Errors (Next.js)

# "Unable to acquire lock" error
rm -f apps/dashboard/.next/dev/lock
./scripts/e2e.sh

Zombie Processes on Ports

# Kill manually
lsof -ti:3100 | xargs kill -9 2>/dev/null || true
lsof -ti:3101 | xargs kill -9 2>/dev/null || true

# Or use Makefile
make kill-servers

"net::ERR_CONNECTION_REFUSED"

Servers aren't running. The CI was likely interrupted. Re-run with e2e.sh:

./scripts/e2e.sh e2e/auth.spec.ts --reporter=list

Don't use E2E_SKIP_SERVER=1 unless servers are already running.

Strict Mode Violations (Multiple Elements)

// ❌ BAD: Matches multiple elements
page.getByRole('link', { name: /paper trading/i })

// ✅ GOOD: Be specific
page.getByRole('link', { name: /paper trading/i }).nth(1)
// or
page.getByTestId('header').getByRole('link', { name: /paper trading/i })

Auth Failures ("CredentialsSignin")

1. Check PLAYWRIGHT=1 env var is set
2. Check test user is seeded
3. Check AUTH_SECRET is set
4. Don't "fix" by adding retries - fix auth setup

"Unauthorized" API Errors

API requests need explicit auth cookies - see "Auth for API Requests" section above.

Tests That Require External Resources

Tests that depend on external state (fixtures with data files, database data) should skip gracefully:

// ❌ BAD: Silently passes when condition isn't met
if (response.ok()) {
  // test logic
}

// ✅ GOOD: Use test.skip() to indicate why test was skipped
if (!response.ok()) {
  test.skip(true, 'Resource not available - skipping test');
  return;
}
// test logic

UI Text Matching

Be careful with regex patterns for UI text:

// ❌ May fail if exact text differs
.getByText(/no trading sessions|create one to get started/i)

// ✅ Match the exact text from the source
.getByText(/No trading sessions yet/i)

Fixture and Route Sync Issues

Fixture File Name Mismatches

Common failure pattern: Tests expect fixture files that don't exist or have different names.

// ❌ Test expects generic name
loadFixture('entry/time-of-day.jsonl')

// ✅ Actual fixture has descriptive name
loadFixture('entry/time-of-day-reject.jsonl')

Debug strategy:

1. List actual fixtures: ls e2e/fixtures/entry/
2. Check fixture content matches test expectations
3. Fixtures may contain only rejections OR only entries, not both

Fixture Location - CRITICAL

Test fixtures MUST be in e2e/fixtures/ - the data/ directory is gitignored.

# ❌ WRONG - gitignored, won't be in CI
data/fixtures/test/combined/my-fixture.jsonl

# ✅ CORRECT - committed with tests
e2e/fixtures/combined/my-fixture.jsonl

If CI fails with "Fixture not found" but works locally:

1. Check if fixture is in data/ (gitignored)
2. Move to e2e/fixtures/
3. Update loadFixture() path if needed

API Route Mismatches

Common failure pattern: Tests use API routes that don't exist.

// ❌ Route doesn't exist
request.post(`${API_BASE}/v1/replay/start`)

// ✅ Actual route location
request.post(`${API_BASE}/admin/synthetic/replay/start`)

Debug strategy:

1. Search for route definition: grep -r "replay/start" apps/api/src/
2. Check route file exports match Hono router registration
3. Use API health/docs endpoints to discover available routes

Resource Contention in Parallel Runs

Individual tests may pass but fail when run together due to:

• Port conflicts between test suites
• Database connection pool exhaustion
• SSE connection limits

# ✅ Run specific test files individually for debugging
./scripts/e2e.sh entry-filters.spec.ts --reporter=list
./scripts/e2e.sh exit-strategies.spec.ts --reporter=list

# ⚠️ Combined runs may show ECONNREFUSED - retry or reduce parallelism
./scripts/e2e.sh --workers=1 --reporter=list

Don'ts