pmx-canvas-testing

# PMX Canvas Testing Use this skill when changing the PMX Canvas product checkout and you need a consistent verification path. The commands and paths below refer to that source checkout, not to a workspace that merely has the published package and skills installed. For an isolated consumer workspace, do not expect the package to contain this checkout's `tests/`, source-only scripts, or Playwright configuration. Validate package transports and host integration through their public CLI, MCP, HTTP, and SDK surfaces; use the `published-consumer-e2e` skill when a packed-install outside-in check is required. ## When To Use - Any code change that should be validated before handoff - Adding or updating tests - Debugging a regression or flaky behavior - Updating CI or coverage commands - Deciding the minimum acceptable verification for a task ## Default Verification Ladder Pick the narrowest command that proves the change, then escalate if the change crosses layers. ```bash bun run test # Fast Bun suite for server/state/API coverage bun run test:coverage # Same Bun suite with coverage output bun run test:web-canvas # Browser smoke against a real running app bun run test:all # Bun suite + browser smoke ``` **Direct `bun test` is side-effect-guarded by `tests/preload.ts`** (wired via `bunfig.toml` `[test].preload`): it defaults `PMX_CANVAS_DISABLE_BROWSER_OPEN=1` so the suite's open-as-site tests cannot launch the developer's real browser, whichever way the tests are invoked. The preload covers `bun test` only — when you boot a server yourself (`bun run src/cli/index.ts`), pass `--no-open` and set `PMX_CANVAS_DISABLE_BROWSER_OPEN=1` explicitly. **Never pipe a gating command through `tail`, `head`, or `grep`.** The pipeline reports the exit code of the LAST command, so `bun run test | tail -5` exits 0 on a red suite. Redirect to a file and check `$?`, then read the file. ## Which Command To Run - Server/state/API-only changes: run `bun run test` - Test-only changes: run `bun run test` and `bun run test:coverage` if coverage matters - Client/UI/browser interaction changes: run `bun run test:web-canvas` - Cross-stack or non-trivial changes: run `bun run test:all` - Before changing browser-visible behavior under `src/client/`: rebuild with `bun run build` Manual browser validation also requires a fresh client bundle. `bun run test:web-canvas` already does this for you. ## Coverage Notes - `bun run test:coverage` covers the Bun unit suite under `tests/unit/` - Coverage output is written to `coverage/lcov.info` and also printed as a text summary - CI currently uses that same unit-test coverage command, then runs browser smoke separately - Do not describe `test:coverage` as full-stack coverage; Playwright coverage is not wired in here ## Current Project Test Surface - In the product source checkout, Bun tests live under `tests/unit/`. - In the product source checkout, Playwright browser smoke lives under `tests/e2e/`. - Product CI runs those checkout suites: Bun coverage plus the browser smoke flow. - An isolated installed consumer instead tests the package's public transports and host behavior; it does not run or depend on the product checkout's source suites. ## WebView Automation Caveat - Some Linux/CI environments expose `Bun.WebView` but still cannot start a usable automation session within the timeout window - When testing WebView automation, treat a cleanly reported unsupported/timeout runtime boundary as distinct from a product regression Prefer extending the existing suites before inventing a one-off script. ## Test Authoring Rules - Keep unit tests isolated. Reset singleton server state between tests. - Test public behavior first: HTTP endpoints, persisted state, visible UI outcomes - Use browser tests for interactions the user actually performs: node creation, pins, snapshots, loading the workbench, and other sync-sensitive flows - Avoid brittle selectors. Prefer stable text, roles, titles, or deliberate component hooks - If a change spans server and client, add at least one server-side assertion and one browser or API-level proof - MCP harnesses using the official SDK must wind down BOTH halves: call `client.close()` AND `transport.close()` on the `StdioClientTransport`. Closing only the client leaves the spawned stdio server process alive after the test exits (0.4.0 cycle finding) - Before a version-test cycle, run `pmx-canvas skills sync --check` in the consumer workspace — exit 1 means the installed skill copies are stale against the package; run `pmx-canvas skills sync --yes` to refresh them (whole trees, whatever agent layout owns them) before trusting skill-guided results ## Tests That Cannot Fail A test that passes against the broken code is worse than no test — it certifies the bug. Every regression test must be shown to discriminate: run it against the unfixed code (or hard-code the broken value) and watch it go red BEFORE you trust the green. Four ways a test silently stops discriminating in this repo: - **Polled assertions pass on the first sample.** `expect.poll(fn).toBeGreaterThanOrEqual(180)` succeeds the instant one sample qualifies, so it never observes the failure window. Wait for the settle condition, then assert once. - **Committed fixtures embed generated output.** `src/server/demo-state.json` carries each primitive's generated markup; count and coverage assertions are satisfied by stale markup just as well as fresh. Guard generated fixtures by rebuilding from the generator's INPUT and byte-comparing against the current renderer. - **Absolute assertions about globally-wired side effects are order-dependent.** Anything flowing through a single-slot listener (architecture rule 8 in `CLAUDE.md`) is wired or not depending on whether an earlier test file booted a server — so the test passes alone and fails in the suite, or vice versa. Assert DIFFERENTIALLY: run the action disarmed and armed, require the deltas to match. Always run the full suite before trusting a new test; a single-file run hides this. - **A new node type passes every test and renders nothing.** Server, API, and client-unit assertions all pass while `isCanvasNodeType` drops the type during layout apply. Only a DOM census on a live board catches it. ## Layout And Embedded Content Checks - For seeded or generated boards, add API-level geometry assertions: expected node/edge counts, group counts, valid edge endpoints, no visible node overlaps, and group children contained with header/padding space. - For grouped layouts, test non-group node overlap separately from group containment. Group frames are allowed to contain children; children should not overlap each other or collide with headers. - For edge-heavy layouts, assert endpoints exist and long cross-board edges are intentional. If a user says an edge “comes from nowhere,” add a regression check for missing endpoints or excessive edge distance in that board. - For `graph`, `json-render`, `mcp-app`, webpage, and image nodes, API geometry is not enough. Verify the rendered browser frame when changing sizing: iframe/body `scrollHeight` and `scrollWidth` should fit the available frame unless scrolling is the intended behavior. - When checking embedded frame fit manually, start from a clean seeded state, rebuild stale bundles, and inspect the actual iframe document in a browser. Server dimensions can look correct while the embedded content is still clipped. - **A node's stored content is not render evidence.** An html/primitive node's stored HTML holds every conditional branch — hidden banners, error states, empty states — so a content-level read (search hit, pinned context, `node get`, any text summary) quotes strings the user never saw. The 0.4.7 report filed an "AX bridge unavailable" banner as a live product gap on exactly this basis; the served surface had the bridge injected ahead of the check and the banner was `hidden` in the document. Before filing a rendering bug, fetch the served surface or inspect the DOM. - User-facing creation flows must end with the new nodes visible: after the create (plus the skill-mandated focus/fit), a screenshot must show them without any manual pan. A `panned: true` API result alone is not proof — verify the frame is on-screen and unobscured. ## Failure Handling - Never wave away a failure without checking whether your change caused it - If the failure is truly pre-existing, say that explicitly and include the failing command - If a command cannot run in the environment, say what blocked it - If browser tests fail after a client change, confirm the bundle was rebuilt and the server started from the updated code ## Handoff Standard Before marking work done, report: - Which verification command(s) you ran - Whether they passed - Any meaningful gaps, skipped checks, or known pre-existing failures For non-trivial changes, the default expectation is `bun run test:all` unless there is a clear reason to scope verification more narrowly. ## Presence, sessions, and the redesigned chrome (rail-chrome-v2) - Agent/human presence is in-memory and TTL-swept. An e2e reset must detach any attached session (`POST /api/canvas/ax/presence { attached: false }` per attached presence), clear the scope fence, and mark its own writes with `x-pmx-workbench: 1` — otherwise the harness reads as an external writer and a leftover fence refuses the next test's `clear`. Unattached writers from earlier tests can still be live (90 s): assert about YOUR writers, not exact totals. - Assert SSE frames as received (`readSseEvent` in `tests/unit/agent-presence-api.test.ts`), not the object handed to the emitter — the envelope overwrites `sessionId` and `timestamp`. - Two-tab behaviour (human cursors, the edit lock) needs two browser contexts; name them with `/workbench?name=…` so assertions can target a cursor by its tag. - The Browser pane reports `visibility: hidden` and never fires rAF: drags, drop pills, edge previews and presence animations only work under Playwright. Use the pane for static checks. - Selection is shift-click on the node BODY (the titlebar drags); groups have no ports and their drag handle is the edge row; the selection bar and command bar both float bottom-center (the selection bar lifts during a session).