pwa-device-testing

# PWA Device Testing Headless Playwright tests cover logic and layout but cannot validate: - Chrome password manager / autofill behavior - WebAuthn biometric (fingerprint, face) - PWA install-to-homescreen and standalone mode - Real touch gestures, virtual keyboard, IME - Service worker update UX on actual Chrome - CSS safe-area-inset rendering on notched devices This skill provides the correct setup, known pitfalls, and ready-to-use templates for testing on real Chrome via Android emulator. ## Quick Start ```bash # First time only: scripts/setup-avd.sh # Every session (Appium tests -- primary): scripts/run-appium-tests.sh # full suite (31 tests) scripts/run-appium-tests.sh --suite my-label # tagged archive directory # Legacy (CDP emulator tests): npm run test:emulator ``` **IMPORTANT:** ALWAYS run Appium/emulator tests via `scripts/run-appium-tests.sh`, never bare `npx playwright test --config=playwright.appium.config.js`. The script handles screen recording, ANR dialog dismissal, archival to `test-history/`, and ffprobe validation. Without it, test runs produce no video evidence for review. Fast-gate tests (tsc, eslint, headless `npx playwright test`) are fine to run directly -- they don't touch the emulator and don't need recording. `npm run test:emulator` (via `scripts/run-emulator-tests.sh`) handles the legacy CDP path: boots emulator if needed, enables Chrome debugging, sets up port forwarding, runs Playwright over CDP. ## Architecture ``` Host machine Android Emulator (Pixel 7, API 35) +-----------------------+ +---------------------------+ | MobiSSH server :8081 |<--adb rev-->| Chrome tab: localhost:8081| | Playwright test runner|--CDP:9222-->| Chrome DevTools socket | +-----------------------+ +---------------------------+ ``` - **CDP connection**: Playwright `connectOverCDP()` to real Chrome via ADB-forwarded DevTools port - **Port forwarding**: `adb reverse tcp:8081 tcp:8081` so emulator's localhost reaches host - **Single worker CDP**: One CDP connection per test file, fresh tab per test ## Interaction Design Principles (learned the hard way) ### Emulator tests must be faithful proxies for human interaction Every dialog, prompt, and overlay that appears during a test flow must be handled the way a real user would handle it: see it, understand what it's asking, and dismiss it appropriately. This includes both app-owned dialogs (host key accept, vault setup) and Chrome-native UI (save password bar, add username suggestion, notification prompt). If a test can't click a button, a real user can't either. The test is surfacing a real UX bug. ### Never use `force: true` to work around click interception When Playwright reports `<div class="vault-dialog">...</div> intercepts pointer events`, the correct response is to fix the CSS/layout so the button is actually clickable, NOT to bypass Playwright's actionability checks with `{ force: true }`. Using `force: true`: - Hides real interaction bugs from the test suite - Creates a test that passes while the actual user flow is broken - Masks layout overflow on mobile viewports Common causes of click interception on mobile: - `align-items: center` on `position: fixed; inset: 0` overlays -- when the dialog content is taller than the viewport, content overflows and sibling elements intercept clicks. Fix: `align-items: flex-start` + `overflow-y: auto` on the overlay, `margin: auto 0` on the dialog for vertical centering that still allows scrolling. - Chrome-native UI (password save bar, username suggestion) appearing as a layer between your app dialog and the click target. Fix: suppress Chrome autofill on test fields with `autocomplete="off"`, pre-grant notifications, use `--disable-fre` flag. - Keyboard pushing elements up so labels overlap buttons. Fix: ensure sufficient spacing, or scroll the button into view first. ### Feature removal is a valid outcome The selection overlay feature (#55) went through 6+ commits, was feature-flagged off, still caused interference with core scroll behavior (#143), and was ultimately removed entirely (600 lines deleted in 0ac4010). This validates the "know when to quit" principle: if every fix introduces a new bug, the abstraction is wrong. Strip it, ship the working core, and re-approach later with a cleaner design. ### Always verify server version before asking user to test ``` scripts/verify-test-ready.sh [--url https://mobissh.tailbe5094.ts.net] ``` Checks server currency, HTTP endpoint hash match, and prints the `?reset=1` URL for SW cache busting. Exit 1 = not ready. ## Critical Pitfalls (learned the hard way) ### Chrome DevTools socket requires `set-debug-app` Handled automatically by `scripts/run-emulator-tests.sh` and `scripts/run-appium-tests.sh`. If running manually: `adb shell am set-debug-app --persistent com.android.chrome` then force-stop and relaunch Chrome. ### No `browser.newContext()` on Android Chrome Android Chrome's CDP exposes a single default browser context. Calling `browser.newContext()` throws: `Protocol error (Target.createBrowserContext): Failed to create browser context.` Correct pattern: ```javascript const context = browser.contexts()[0]; // use the default const page = await context.newPage(); // new tab within it ``` ### Shared localStorage across tests Since all tabs share the single default context, localStorage is shared. Every test fixture MUST clear localStorage AND reload before the test runs. The app reads localStorage on init (panel state, vault, profiles), so clearing alone doesn't help if the app already initialized with stale state: ```javascript await page.goto(BASE_URL, { waitUntil: 'domcontentloaded' }); await page.evaluate(() => localStorage.clear()); await page.reload({ waitUntil: 'domcontentloaded' }); // app re-inits with clean state ``` ### Vault snapshot fixture (skip per-test vault setup) Creating the vault from scratch per test wastes ~5s (keyboard dismiss, modal wait, form fill). The `vaultSnapshot` worker-scoped fixture creates the vault once per test file and snapshots the localStorage keys. The `emulatorPage` fixture restores the snapshot and auto-unlocks: ```javascript // Worker-scoped: creates vault once, snapshots localStorage keys vaultSnapshot: [async ({ cdpBrowser }, use) => { // ... create vault with password 'test', snapshot localStorage ... await use(snapshot); // { vaultMeta, vaultData, ... } }, { scope: 'worker' }], // Per-test: restores snapshot, unlocks vault via addInitScript hook emulatorPage: async ({ cdpBrowser, vaultSnapshot }, use) => { // ... restore snapshot to localStorage ... await page.addInitScript(() => { // Hook into __appReady to unlock vault before promptVaultSetupOnStartup Object.defineProperty(window, '__appReady', { /* ... fill pw + click unlock */ }); }); await page.reload({ waitUntil: 'domcontentloaded' }); await page.waitForFunction(() => window.__vaultUnlocked === true); // ... }, ``` **Key insight:** The vault unlock bar doesn't appear on startup — it appears later when `ensureVaultKeyWithUI()` is called during profile operations. Checking for the bar after reload will find `needsUnlock=false` (race condition). The `addInitScript` + `__appReady` hook approach intercepts the app boot to unlock the vault before any UI flow triggers it. Use `emulatorPage` for tests that need a pre-configured vault. Use `cleanPage` for tests that exercise vault setup from scratch. ### Playwright outputDir isolation (CRITICAL) Playwright clears its `outputDir` on each run. The default is `test-results/`, which wipes emulator recordings, report.json, frames, and any other non-Playwright artifacts. Every Playwright config MUST set a dedicated `outputDir`: - `playwright.config.js` → `test-results/headless` - `playwright.emulator.config.js` → `test-results/playwright-emulator` - `playwright.appium.config.js` → `test-results-appium` - `playwright.browserstack.config.js` → `test-results/browserstack` Without this, `run-emulator-tests.sh` writes `recording.mp4` and `report.json` to `test-results/emulator/`, then the next Playwright run wipes them. ### workers: 1 is mandatory for CDP Parallel Playwright workers each try to interact with the same single Chrome instance over CDP. This causes "Target page, context or browser has been closed" across all tests. Always set `workers: 1` in the emulator config: ```javascript module.exports = defineConfig({ workers: 1, // single Chrome instance via CDP // ... }); ``` ### Inject page state AFTER navigation, not before Any `page.evaluate()` state injection (WS spies, test globals) done before `page.goto()` gets destroyed by the navigation. Always inject on the live, already-loaded page: ```javascript // WRONG: spy gets destroyed by goto() await page.evaluate(() => { window.__spy = []; }); await page.goto(BASE_URL); // RIGHT: inject after the page is loaded await page.goto(BASE_URL, { waitUntil: 'domcontentloaded' }); await page.evaluate(() => { window.__spy = []; }); ``` ### Use actionTimeout for fast selector failure Without `actionTimeout`, a bad selector (e.g. `#connectBtn` that doesn't exist) waits the full test timeout (60s), then cleanup closes the page, producing a misleading "Target page closed" error. Set a short action timeout so bad selectors fail fast with the actual error: ```javascript use: { actionTimeout: 10_000, // fail fast on bad selectors } ``` ### Elements may not have IDs Don't assume HTML elements have IDs. Use semantic/structural selectors: ```javascript // BAD: #connectBtn doesn't exist await page.locator('#connectBtn').click(); // GOOD: target by form context + type await page.locator('#connectForm button[type="submit"]').click(); ``` ### Page.screencastFrame CDP doesn't work on Android emulator The `Page.startScreencast` / `Page.screencastFrame` CDP API returns 0 frames on Android emulator Chrome. It works on desktop Chrome but the emulator's GPU pipeline doesn't produce screencast frames. Don't rely on frame count assertions. Screenshots via `page.screenshot()` work fine as an alternative. ### Worker-scoped CDP connection is mandatory Creating a new `connectOverCDP()` per test destabilises the DevTools socket. After ~4-5 connect/disconnect cycles, the connection drops with "Target page, context or browser has been closed." Use a worker-scoped fixture: ```javascript cdpBrowser: [async ({}, use) => { const browser = await chromium.connectOverCDP(`http://127.0.0.1:${CDP_PORT}`); await use(browser); browser.close(); }, { scope: 'worker' }], ``` ### Chrome nag modals block test visibility on first launch Handled by `scripts/run-appium-tests.sh` (pre-grants notifications, sets Chrome flags) and the Appium fixture's modal-dismiss step. If writing a new test runner, replicate the three-layer defense: 1. `adb shell pm grant com.android.chrome android.permission.POST_NOTIFICATIONS` 2. Chrome `--disable-fre --no-first-run --no-default-browser-check` flags 3. Fixture-level modal dismiss with 2s timeout fallback ### KVM group membership requires session reload After `sudo usermod -aG kvm $USER`, the current shell doesn't pick up the new group. Use `sg kvm -c 'emulator ...'` or start a new login session. ### AVD config uses ` = ` (with spaces) The `config.ini` generated by `avdmanager` uses `key = value` (space-equals-space), not `key=value`. Sed patterns without spaces silently fail and the fallback `echo` creates duplicate keys. The setup script uses a `set_avd_prop` helper that handles both formats. ### WebAuthn biometric toggle in headless test browsers `prfAvailable()` returns `true` in Playwright's headless Chromium (PublicKeyCredential API exists) but `navigator.credentials.create()` hangs forever (no authenticator). In headless tests, uncheck the biometric toggle via `page.evaluate`: ```javascript await page.evaluate(() => { const cb = document.getElementById('vaultEnableBio'); if (cb) cb.checked = false; }); ``` The CSS toggle hides the checkbox with `opacity:0; width:0; height:0`, so Playwright's `isVisible()` returns false and `uncheck()` silently skips it. Always use `page.evaluate` for CSS-hidden form elements. ## Real SSH Integration via Docker For features that need a live SSH connection (gestures, terminal buffer, command execution), use the Docker test-sshd container instead of WebSocket mocks: ```bash docker compose -f docker-compose.test.yml up -d test-sshd # simple test container, no lifecycle script ssh -p 2222 testuser@localhost # password: testpass ``` The `sshd-fixture.js` helper starts the container automatically and exposes credentials to tests. The `setupRealSSHConnection(page, sshServer)` helper in `fixtures.js` handles: SSRF bypass for localhost, WS URL rewriting (localhost→10.0.2.2), connect form fill, host key acceptance, and waiting for connected state. Vault setup is handled by the `vaultSnapshot` + `emulatorPage` fixture (no per-test vault creation). ## Touch Gesture Testing Gesture helpers use CDP `Input.dispatchTouchEvent` which goes through Chrome's real input pipeline and fires DOM touch events faithfully. An in-page touch visualizer draws green dots/trails at finger positions so gestures are visible in screen recordings (Android's `pointer_location` overlay doesn't register CDP touches). Helpers in `tests/emulator/fixtures.js`: - `swipe(page, selector, startX, startY, endX, endY, steps)` -- single-finger swipe via CDP - `pinch(page, selector, startDist, endDist, steps)` -- two-finger pinch via CDP - `sendCommand(page, cmd)` -- type into IME input char-by-char The touch visualizer is injected automatically by `swipe()` and `pinch()`. Green dots show current finger positions, small trail dots persist for 2s to show the gesture path. Verify gesture effects through app state, not visual diffs: ```javascript // Scroll: check xterm buffer position const vp = await page.evaluate(() => window.__testTerminal.buffer.active.viewportY); // Swipe: check WS spy for tmux commands const msgs = await page.evaluate(() => window.__mockWsSpy.filter(...)); // Pinch: check terminal font size const font = await page.evaluate(() => window.__testTerminal.options.fontSize); ``` ## Exploratory Interaction Testing (script-first, then assert) Before writing test assertions, use the emulator to explore what actually happens when a user performs a gesture. This inverts the typical TDD approach: instead of starting with expected behavior and checking if it matches, start by simulating the physical interaction, capturing what the device shows, and reviewing the visual evidence to discover the domain of possibilities. ### The workflow **1. Script the interaction textually.** Before writing any test code, describe the interaction as a sequence of physical actions: ``` - Navigate to Settings panel - Place two fingers on screen, 200px apart - Move fingers apart to 400px (pinch out / zoom in) - Observe: does the bottom bar stay anchored? Does content scale? - Release fingers - Observe: does layout return to normal? ``` **2. Translate to emulator script.** Use the gesture helpers to replay the interaction on the emulator, with screen recording running but NO assertions: ```javascript test('explore: pinch zoom on settings panel', async ({ emulatorPage: page }) => { await page.locator('[data-panel="settings"]').click(); await page.waitForSelector('#panel-settings.active'); await screenshot(page, testInfo, '01-before-pinch'); // Simulate pinch-out (zoom in) await pinch(page, '#panel-settings', 100, 300); await page.waitForTimeout(500); await screenshot(page, testInfo, '02-during-zoom'); // Release and observe await page.waitForTimeout(1000); await screenshot(page, testInfo, '03-after-zoom'); });