Write and maintain Playwright end-to-end tests for the Onyx application. Use when creating new E2E tests, debugging test failures, adding test coverage, or when the user mentions Playwright, E2E tests, or browser testing.
Write and maintain Playwright end-to-end tests for the Onyx application. Use when creating new E2E tests, debugging test failures, adding test coverage, or when the user mentions Playwright, E2E tests, or browser testing.
Playwright E2E Tests
Spec-authoring rules live in web/tests/e2e/README.md — the Page Object Model, locator
priority, and auto-retrying matchers. Read it before adding or changing a spec. This skill covers
the surrounding workflow: layout, environment, running tests, auth, and Onyx-specific utilities.
Always use absolute imports with the @tests/e2e/ prefix — never relative paths (../, ../../). The alias is defined in web/tsconfig.json and resolves to web/tests/.
Run Playwright against your local dev servers so the tests exercise your working-tree changes:
start ods web dev (frontend, :3000) and ods backend api (backend, :8080), then use the
config default BASE_URL=http://localhost:3000 — no override needed. In dev, next dev proxies
/api/* to the backend itself (the route handler web/src/app/api/[...path]/route.ts), so the UI
and /api both live on :3000; global setup needs the backend up too, since it registers/logs in
users over /api/auth/*.
The devcontainer image bakes in Playwright's OS deps + Node, but not the Chromium browser binary. If a run fails with a missing-browser error, install it once: bunx playwright install chromium.
Run commands from the web/ directory; global-setup.ts writes auth-state files (admin_auth.json, …) to the cwd. It's idempotent, so setup is fast.
web/.vscode/.env may be absent — dotenv skips it silently (fine for most tests). One exception: tests/e2e/mcp/mcp_oauth_flow.spec.tsthrows at import without MCP_OAUTH_* vars, which also breaks a whole-suite --list. Scope runs to specific files/dirs to avoid it.
Running Tests
cd web # from the repo root# Run a specific test file
bunx playwright test tests/e2e/chat/default_assistant.spec.ts
# Narrow to a single test by title (fast smoke check)
bunx playwright test tests/e2e/chat/welcome_page.spec.ts \
-g "chat input is visible and focusable" --project admin
Test Projects
Project
Description
Parallelism
admin
Standard tests (excludes @exclusive)
Parallel
exclusive
Serial, slower tests (tagged @exclusive)
1 worker
All tests use admin_auth.json storage state by default (pre-authenticated admin session).
Authentication
Global setup (global-setup.ts) runs automatically before all tests and handles:
Server readiness check (polls health endpoint, 60s timeout)
Provisioning test users: admin, admin2, and a pool of worker users (worker0@example.com through worker7@example.com) (idempotent)
API login + saving storage states: admin_auth.json, admin2_auth.json, and worker{N}_auth.json for each worker user
Setting display name to "worker" for each worker user
Promoting admin2 to admin role
Ensuring a public LLM provider exists
Both test projects set storageState: "admin_auth.json", so every test starts pre-authenticated as admin with no login code needed.
When a test needs a different user, use API-based login — never drive the login UI:
import { loginAs } from"@tests/e2e/utils/auth";
await page.context().clearCookies();
awaitloginAs(page, "admin2");
// Log in as the worker-specific user (preferred for test isolation):import { loginAsWorkerUser } from"@tests/e2e/utils/auth";
await page.context().clearCookies();
awaitloginAsWorkerUser(page, testInfo.workerIndex);
Test Structure
Tests start pre-authenticated as admin — navigate and test directly:
User isolation — tests that modify visible app state (creating assistants, sending chat messages, pinning items) should run as a worker-specific user and clean up resources in afterAll. Global setup provisions a pool of worker users (worker0@example.com through worker7@example.com). loginAsWorkerUser maps testInfo.workerIndex to a pool slot via modulo, so retry workers (which get incrementing indices beyond the pool size) safely reuse existing users. This ensures parallel workers never share user state, keeps usernames deterministic for screenshots, and avoids cross-contamination:
If the test requires admin privileges and modifies visible state, use "admin2" instead — it's a pre-provisioned admin account that keeps the primary "admin" clean for other parallel tests. Switch to "admin" only for privileged setup (creating providers, configuring tools), then back to the worker user for the actual test. See chat/default_assistant.spec.ts for a full example.
loginAsRandomUser exists for the rare case where the test requires a brand-new user (e.g. onboarding flows). Avoid it elsewhere — it produces non-deterministic usernames that complicate screenshots.
API resource setup — only when tests need to create backend resources (image gen configs, web search providers, MCP servers). Use beforeAll/afterAll with OnyxApiClient to create and clean up. See chat/default_assistant.spec.ts or mcp/mcp_oauth_flow.spec.ts for examples. This is uncommon (~4 of 37 test files).
Key Utilities
OnyxApiClient (@tests/e2e/utils/onyxApiClient)
Backend API client for test setup/teardown. Key methods:
THEMES — ["light", "dark"] as const array for iterating over both themes
setThemeBeforeNavigation(page, theme) — sets next-themes theme via localStorage before navigation
When tests need light/dark screenshots, loop over THEMES at the test.describe level and call setThemeBeforeNavigation in beforeEachbefore any page.goto(). Include the theme in screenshot names. See admin/admin_pages.spec.ts or chat/chat_message_rendering.spec.ts for examples:
TOOL_IDS — centralized data-testid selectors for tool options
openActionManagement(page) — opens the tool management popover
Waiting Strategy
// Wait for load state after navigationawait page.goto("/app");
await page.waitForLoadState("networkidle");
// Wait for specific elementawait page.getByTestId("chat-intro").waitFor({ state: "visible", timeout: 10000 });
// Wait for URL changeawait page.waitForFunction(() =>window.location.href.includes("chatId="), null, { timeout: 10000 });
// Wait for network responseawait page.waitForResponse(resp => resp.url().includes("/api/chat") && resp.status() === 200);
Best Practices
Descriptive test names — clearly state expected behavior: "should display greeting message when opening new chat"
API-first setup — use OnyxApiClient for backend state; reserve UI interactions for the behavior under test
User isolation — tests that modify visible app state (sidebar, chat history) should run as the worker-specific user via loginAsWorkerUser(page, testInfo.workerIndex) (not admin) and clean up resources in afterAll. Each parallel worker gets its own user, preventing cross-contamination. Reserve loginAsRandomUser for flows that require a brand-new user (e.g. onboarding)
DRY helpers — extract reusable logic into utils/ with JSDoc comments
No hardcoded waits — use the auto-retrying matchers (web/tests/e2e/README.md), waitFor, or waitForLoadState; never waitForTimeout
Parallel-safe — no shared mutable state between tests. Prefer static, human-readable names (e.g. "E2E-CMD Chat 1") and clean up resources by ID in afterAll. This keeps screenshots deterministic and avoids needing to mask/hide dynamic text. Only fall back to timestamps (\test-${Date.now()}``) when resources cannot be reliably cleaned up or when name collisions across parallel workers would cause functional failures
Error context — catch and re-throw with useful debug info (page text, URL, etc.)
Tag slow tests — mark serial/slow tests with @exclusive in the test title
Visual regression — use expectScreenshot() for UI consistency checks
Minimal comments — only comment to clarify non-obvious intent; never restate what the next line of code does
