writing-playwright-tests

# Writing Playwright Tests for ComfyUI_frontend The design rules that apply at every test level (behavioral assertions, lowest proving level, table-driven variants, isolation, no sleeps) are in `docs/guidance/testing-principles.md`. This skill covers the Playwright and ComfyUI mechanics. ## Golden Rules 1. **ALWAYS look at existing tests first.** Search `browser_tests/tests/` for similar patterns before writing new tests. 2. **ALWAYS read the fixture code.** The APIs are in `browser_tests/fixtures/` - read them directly instead of guessing. 3. **Use premade JSON workflow assets** instead of building workflows programmatically. - Assets live in `browser_tests/assets/` - Load with `await comfyPage.workflow.loadWorkflow('feature/my_workflow')` - Create new assets by starting with `browser_tests/assets/default.json` and manually editing the JSON to match your desired graph state 4. **Seed starting settings with `test.use({ initialSettings })`.** The main fixture replaces backend settings before every test; do not add settings resets in `afterEach`. Keep runtime setters for changes under test. Nested `test.use` replaces the parent's settings object, so include inherited overrides. See [Starting settings and isolation](../../../browser_tests/README.md#starting-settings-and-isolation). ## Vue Nodes vs LiteGraph: Decision Guide Choose based on **what you're testing**, not personal preference: | Testing... | Use | Why | | ---------------------------------------------- | -------------------------------- | ---------------------------------------- | | Vue-rendered node UI, DOM widgets, CSS states | `comfyPage.vueNodes.*` | Nodes are DOM elements, use locators | | Canvas interactions, connections, legacy nodes | `comfyPage.nodeOps.*` | Canvas-based, use coordinates/references | | Both in same test | Pick primary, minimize switching | Avoid confusion | Always add `{ tag: '@vue-nodes' }` to the test or `test.describe` when it needs Vue Nodes. The fixture enables the renderer before boot and waits for the nodes. Never manually set `Comfy.VueNodes.Enabled`, including through `initialSettings`, or call `comfyPage.vueNodes.waitForNodes()` in tests. **Vue Node state uses CSS classes:** ```typescript const BYPASS_CLASS = /before:bg-bypass\/60/ await expect(node).toHaveClass(BYPASS_CLASS) ``` ## Common Issues These are frequent causes of flaky tests - check them first, but investigate if they don't apply: | Symptom | Common Cause | Typical Fix | | ----------------------------------- | ------------------------- | -------------------------------------------------------------------------------------- | | Test passes locally, fails in CI | Missing nextFrame() | Add `await comfyPage.nextFrame()` after canvas ops (not needed after `loadWorkflow()`) | | Keyboard shortcuts don't work | Missing focus | Add `await comfyPage.canvas.click()` first | | Double-click doesn't trigger | Timing too fast | Add `{ delay: 5 }` option | | Elements end up in wrong position | Drag animation incomplete | Use `{ steps: 10 }` not `{ steps: 1 }` | | Widget value wrong after drag-drop | Upload incomplete | Add `{ waitForUpload: true }` | | Test fails when run with others | Test pollution | Add `afterEach` with `resetView()` | | Local screenshots don't match CI | Platform differences | Screenshots are Linux-only, use PR label | | `subtree intercepts pointer events` | Canvas overlay (z-999) | Use `dispatchEvent` on the DOM element to bypass overlay | | Context menu empty / wrong items | Node not selected | Select node first: `vueNodes.selectNode()` or `nodeRef.click('title')` | | `navigateIntoSubgraph` timeout | Node too small in asset | Use node size `[400, 200]` minimum in test asset JSON | ## Test Tags Add appropriate tags to every test: | Tag | When to Use | | ------------- | ----------------------------------------- | | `@smoke` | Quick essential tests | | `@slow` | Tests > 10 seconds | | `@screenshot` | Visual regression tests | | `@canvas` | Canvas interactions | | `@node` | Node-related | | `@widget` | Widget-related | | `@mobile` | Mobile viewport (runs on Pixel 5 project) | | `@2x` | HiDPI tests (runs on 2x scale project) | ```typescript test.describe('Feature', { tag: ['@screenshot', '@canvas'] }, () => { ``` ## Retry Patterns **Never use `waitForTimeout`** - it's always wrong. | Pattern | Use Case | | ------------------------ | ---------------------------------------------------- | | Auto-retrying assertions | `toBeVisible()`, `toHaveText()`, etc. (prefer these) | | `expect.poll()` | Single value polling | | `expect().toPass()` | Multiple assertions that must all pass | ```typescript // Prefer auto-retrying assertions when possible await expect(node).toBeVisible() // Single value polling await expect.poll(() => widget.getValue(), { timeout: 2000 }).toBe(100) // Multiple conditions await expect(async () => { expect(await node1.getValue()).toBe('foo') expect(await node2.getValue()).toBe('bar') }).toPass({ timeout: 2000 }) ``` ## Screenshot Baselines - **Screenshots are Linux-only.** Don't commit local screenshots. - **To update baselines:** Add PR label `New Browser Test Expectations` - **Mask dynamic content:** ```typescript await expect(comfyPage.canvas).toHaveScreenshot('page.png', { mask: [page.locator('.timestamp')] }) ``` ## CI Debugging 1. Download artifacts from failed CI run 2. Extract and view trace: `pnpm dlx playwright show-trace trace.zip` 3. CI deploys HTML report to Cloudflare Pages (link in PR comment) 4. Reproduce CI: `CI=true pnpm test:browser` 5. Local runs: `pnpm test:browser:local` ## Anti-Patterns Avoid these common mistakes: 1. **Arbitrary waits** - Use retrying assertions instead ```typescript // ❌ await page.waitForTimeout(500) // ✅ await expect(element).toBeVisible() ``` 2. **Implementation-tied selectors** - Use test IDs or semantic selectors ```typescript // ❌ page.locator('div.container > button.btn-primary') // ✅ page.getByTestId('submit-button') ``` 3. **Missing nextFrame after canvas ops** - Canvas needs sync time ```typescript await node.drag({ x: 50, y: 50 }) await comfyPage.nextFrame() // Required ``` 4. **Shared state between tests** - Tests must be independent ```typescript // ❌ let sharedData // Outside test // ✅ Define state inside each test ``` ## Quick Start Template ```typescript // Path depends on test file location - adjust '../' segments accordingly import { comfyPageFixture as test, comfyExpect as expect } from '../fixtures/ComfyPage' test.describe('FeatureName', { tag: ['@canvas'] }, () => { test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() }) test('should do something', async ({ comfyPage }) => { await comfyPage.workflow.loadWorkflow('myWorkflow') const node = (await comfyPage.nodeOps.getNodeRefsByTitle('KSampler'))[0] // ... test logic await expect(comfyPage.canvas).toHaveScreenshot('expected.png') }) }) ``` ## Finding Patterns ```bash # Find similar tests grep -r "KSampler" browser_tests/tests/ # Find usage of a fixture method grep -r "loadWorkflow" browser_tests/tests/ # Find tests with specific tag grep -r '@screenshot' browser_tests/tests/ ``` ## Key Files to Read | Purpose | Path | | ----------------- | ------------------------------------------ | | Main fixture | `browser_tests/fixtures/ComfyPage.ts` | | Helper classes | `browser_tests/fixtures/helpers/` | | Component objects | `browser_tests/fixtures/components/` | | Test selectors | `browser_tests/fixtures/selectors.ts` | | Vue Node helpers | `browser_tests/fixtures/VueNodeHelpers.ts` | | Test assets | `browser_tests/assets/` | | Existing tests | `browser_tests/tests/` | **Read the fixture code directly** - it's the source of truth for available methods.