| name | e2e-test-debugging |
| description | Systematic methodology for running, diagnosing, and fixing Playwright E2E test failures in this project. Use this when running E2E tests, investigating flaky or failing E2E tests, verifying infrastructure readiness, or debugging issues where UI elements behave unexpectedly (buttons stay disabled, pages redirect to error, config values are wrong). Also use this when the user mentions "e2e", "playwright", "end-to-end", or asks to verify that the app works. |
E2E Test Debugging
Systematic methodology for running, diagnosing, and fixing Playwright E2E test failures.
Project E2E Architecture
The E2E suite uses Playwright across three browsers (Chromium, Firefox, WebKit) with parallel workers. The test infrastructure involves:
- Vite dev server (
localhost:5173) — serves the Vue SPA and proxies /api, /config.js, /analytics.js to the backend
- ASP.NET Core backend (
https://localhost:8001) — the API server, started with the e2e-local launch profile
- Docker services — KeyDB (Redis), HashiCorp Vault, S3 mock (Adobe S3Mock)
- Config pipeline — backend
appsettings.json → options binding → /config.js endpoint → window.appConfig → Vue composables/components
Key Files
| File | Purpose |
|---|
Warp.ClientApp/playwright.config.ts | Test config: timeouts, browsers, web server |
Warp.ClientApp/e2e/global-setup.ts | Verifies dev server and backend health before tests |
Warp.ClientApp/e2e/locators.ts | Centralized Playwright locator definitions |
Warp.ClientApp/e2e/utils.ts | Shared helpers: clickElement, fillTextAndVerify, saveAndAwaitPostSaveButtons |
Warp.ClientApp/e2e/*.spec.ts | Test specs |
Warp.WebApp/Properties/launchSettings.json | Launch profiles including e2e-local |
Warp.WebApp/appsettings.E2ELocal.json | Backend config for local E2E (localhost addresses) |
docker-compose.e2e.yml | Vault, vault-init, S3 mock services |
docker-compose.yml | KeyDB service |
Pre-Flight: Infrastructure Verification
Before investigating test failures, confirm the entire stack is operational. Many failures that look like test bugs are actually infrastructure problems.
Step 1: Docker Services
docker compose -f docker-compose.yml -f docker-compose.e2e.yml ps
Verify keydb, vault, and s3mock are all Up. If any are down:
docker compose -f docker-compose.yml -f docker-compose.e2e.yml up -d
Step 2: Vault Token
The backend needs a valid Vault token at .vault/warp-e2e.token. The vault-init Docker service generates this automatically. If missing, restart the vault services.
Step 3: Backend Server
The .NET backend must be running on https://localhost:8001 with the e2e-local profile:
dotnet run --project Warp.WebApp --launch-profile e2e-local
If the backend isn't running, the Vite proxy returns ECONNREFUSED for all API calls and the global setup will time out with "Vue app did not initialize within 120000ms" after showing repeated proxy errors.
Step 4: Playwright Browsers
If Playwright was recently updated, browsers may need reinstalling:
cd Warp.ClientApp && yarn playwright install
Error signature: browserType.launch: Executable doesn't exist at ...chrome-headless-shell.exe
Running the Tests
From Warp.ClientApp/:
yarn e2e
yarn e2e:headed
Diagnostic Workflow
1. Classify the Failure Pattern
Before reading individual error details, look at the full test summary to classify:
| Pattern | Likely cause | Where to look |
|---|
| Same failure across all 3 browsers | Code bug or config issue | Application code, config pipeline |
| Failure in one browser only | Timing/flakiness | Timeouts, parallel worker contention |
| All tests fail identically | Infrastructure or config | Pre-flight checks, /config.js output |
| Specific test group fails | Feature-specific bug | The feature's component and API |
This classification saves significant time — a config binding mismatch causing all UI validation to fail looks very different from a single flaky timeout.
2. Collect Error Context
Read the error message, call stack, and error-context artifacts produced by Playwright:
- Error message reveals which locator timed out and with what timeout.
- Call stack pinpoints the exact spec line and utility function involved.
error-context.md (or equivalent snapshot in test-results/) captures the actual DOM/accessibility tree at failure time — this is the single most valuable artifact.
3. Read the Failing Test and Its Helpers
Read the full test case, its locator definitions in e2e/locators.ts, and utility functions in e2e/utils.ts. Understand:
- The exact sequence of user actions the test performs.
- Which locators are used and how they resolve (role, text, test-id).
- Default timeouts baked into helper functions (e.g.,
clickElement uses 15s, saveAndAwaitPostSaveButtons uses 30s).
4. Analyze the Page Snapshot
Compare the expected UI state (what the test waits for) against the actual UI state (from the error-context snapshot):
| Symptom in snapshot | Likely cause |
|---|
| "saving..." / spinner visible, button shows "Pending…" | Async operation (API call) hasn't completed; timeout too short or request failed silently |
Error page or redirect to /error | API returned an error status or ECONNREFUSED; check backend is running |
| Element missing entirely | Wrong route, component not mounted, or conditional rendering (v-if) not satisfied |
| Element present but disabled | State flag not updated, validation failing, or config values wrong |
All pages show /error?status=500 | Backend not reachable; check Vite proxy and backend process |
5. Trace the Config Pipeline
When elements are stuck in wrong states (e.g., button permanently disabled despite valid input), trace the config value flow:
- Backend config: Check the relevant section in
appsettings.json / appsettings.E2ELocal.json (e.g., EntryValidatorOptions)
- Options binding: Check
ServiceCollectionExtensions.cs for .BindConfiguration(nameof(...)) — the section name must match the JSON key
/config.js endpoint: Check SpaExtensions.cs MapSpaConfigs — verify it reads from the correct config path or options object- Frontend consumption: Check
window.appConfig usage in composables (e.g., use-content-size-indicator.ts) and how components derive their state from these values
A mismatch at any step (e.g., /config.js reading Content:SizeLimits:MaxPlainTextSize when the actual config section is EntryValidatorOptions:MaxPlainTextSize) causes the frontend to receive default/zero values, breaking all validation-dependent UI.
6. Cross-Reference with Application Code
Trace the flow through the application:
- Component template: Check
:disabled bindings and v-if/v-else conditions that gate the target element's visibility or state.
- Component script: Follow the async handler (e.g.,
onSave) — identify all awaited calls and what sets the flag the template depends on.
- API layer: Check the request construction (FormData, headers, auth) and error handling (
try/catch/finally).
- Server-side: Verify the endpoint exists, check attribute filters (auth cookies, content-type validation), and confirm the request format matches.
7. Compare with Similar Passing Tests
Find other tests that perform the same flow (e.g., save → copy link) and compare:
- Do passing tests use explicit waits (
await expect(...).toBeVisible({ timeout })) that the failing test lacks?
- Do passing tests skip expensive operations (e.g., image uploads) that add latency?
Common Fix Patterns
Configuration Binding Mismatch
Problem: The /config.js endpoint reads config values from a path that doesn't match the actual appsettings section. The frontend receives 0 or null for limits/settings, causing validation to reject all input. Symptom: elements (e.g., Preview button) stay disabled across all browsers, even after entering valid content.
Diagnosis: Check if the config path in SpaExtensions.cs matches the section name in appsettings.json. Use strongly-typed IOptions<T> instead of raw IConfiguration.GetValue<>() calls with string paths — this catches mismatches at startup.
Fix: Inject the options object and read from it:
["maxPlainTextContentSize"] = configuration.GetValue<int>("Content:SizeLimits:MaxPlainTextSize")
["maxPlainTextContentSize"] = entryValidatorOptions.Value.MaxPlainTextSize
Insufficient Timeout After Expensive Operations
Problem: Default clickElement timeout (15s) is too short for operations involving file uploads, image processing, or heavy server load.
Fix: Use saveAndAwaitPostSaveButtons or create a dedicated helper that waits for the post-operation UI state with a generous timeout.
Key principle: Separate the action (click Save) from the assertion (wait for result) so the wait timeout can be tuned independently.
Missing Wait Between Sequential Actions
Problem: Two clickElement calls back-to-back where the second element only appears after the first action's async side-effect completes.
Fix: Insert an explicit visibility/state assertion between the two clicks:
await clickElement(getSaveButton(page))
await expect(getCopyLinkButton(page)).toBeVisible({ timeout: 30000 })
await clickElement(getCopyLinkButton(page))
Silent API Failures
Problem: The API call fails (network error, 401, 500) but the error is caught and only logged to console. The UI stays stuck in "saving" state forever.
Diagnosis: Check the error-context snapshot for redirect to error pages or stuck spinner states. Add page.on('console') or page.on('response') listeners in the test to capture API failures.
Worker Contention / Resource Exhaustion
Problem: Unlimited parallel workers (Playwright's default is half CPU cores) launch too many concurrent browser processes. The OS runs out of resources and individual tests fail randomly — a different test each run, almost always in only one browser (typically Chromium, which is heaviest). Serial test blocks cascade: one timeout causes all subsequent tests in the block to be skipped.
Symptoms:
- "Test timeout of Nms exceeded while setting up 'page'" — browser couldn't even create a page.
- "saving..." / "Pending…" stuck in the error-context snapshot — the API call hung under system load.
- Failures are non-deterministic: different tests fail on each run.
- Only one browser is affected (Chromium most often, being the most resource-intensive).
- Serial blocks show 1 failure + N skipped, inflating the apparent failure count.
Diagnosis: Run the same suite twice. If different tests fail each time, and only in one browser, this is almost certainly resource contention rather than a code defect. Check playwright.config.ts for workers: undefined (unlimited) and retries: 0 (no tolerance for transient failures).
Fix (in playwright.config.ts):
workers: process.env.CI ? 2 : undefined,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 2 : 4,
retries: process.env.CI ? 2 : 1,
Also audit the failing test for missing waits between sequential actions (see "Missing Wait Between Sequential Actions" above) — under load, a missing wait that normally succeeds by luck will start failing.
Global Setup Timeout
Problem: global-setup.ts times out with "Vue app did not initialize within 120000ms". Logs show repeated ECONNREFUSED errors from the Vite proxy.
Fix: This always means the backend is unreachable. Verify that dotnet run --project Warp.WebApp --launch-profile e2e-local is running and healthy on https://localhost:8001.
Checklist
