Menu
Validate Prose features against the web mode build using Playwright. Use for CI-compatible QA that doesn't require Electron or a display server.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | qa-web |
| description | Validate Prose features against the web mode build using Playwright. Use for CI-compatible QA that doesn't require Electron or a display server. |
Automated QA testing for Prose using Playwright against the web mode build (http://localhost:5174). Unlike Circuit Electron, this runs headlessly in CI — no display server or Electron required.
/qa-web
Or target a specific PR branch:
/qa-web <branch-name>
Playwright browsers must be installed once:
npx playwright install --with-deps chromium
If testing a specific PR branch:
git fetch origin
git checkout <branch-name>
With auto-managed dev server (recommended for local development):
npm run test:web
The Playwright config automatically starts npm run dev:web and waits for http://localhost:5174 to be ready.
Against an already-running dev server (if npm run dev:web is already running):
PLAYWRIGHT_TEST_BASE_URL=http://localhost:5174 npx playwright test
Against the production build (recommended for CI — faster, no HMR overhead):
npm run build:web
npm run preview:web &
npx playwright test --config playwright.config.ts
| Status | Meaning |
|---|---|
| ✅ passed | Feature works as expected |
| ❌ failed | Bug detected — check error message and screenshot |
| ⚠️ flaky | Timing issue — re-run with --retries=2 |
On failure, Playwright saves a screenshot and trace under test-results/. Open the trace viewer:
npx playwright show-trace test-results/<test-name>/trace.zip
Post a comment on the PR with results:
gh pr comment <pr-number> --body "$(cat <<'EOF'
## Web QA Results
**Branch**: \`<branch>\`
**Tested**: $(date -u +"%Y-%m-%dT%H:%M:%SZ")
### Results
| Test | Status | Notes |
|------|--------|-------|
| App loads without errors | ✅ | |
| File explorer renders files | ✅ | |
| Clicking file loads content | ✅ | |
| Editor accepts input | ✅ | |
| Settings panel opens | ✅ | |
| Theme toggle works | ✅ | |
### Recommendation
✅ **Ready for Review** — All smoke tests pass.
EOF
)"
The smoke test suite at tests/smoke.spec.ts covers:
dark class on <html>Add tests to tests/smoke.spec.ts or create new files under tests/.
// Editor
const editor = page.locator('.ProseMirror')
// Toolbar buttons
page.getByRole('button', { name: /show files/i }) // file list toggle
page.getByRole('button', { name: /hide files/i })
page.getByRole('button', { name: /toggle theme/i })
page.getByRole('button', { name: /more options/i }) // overflow menu
// File items in sidebar
page.getByText('Welcome to Prose.md')
page.getByText('Formatting Examples.md')
// Dialogs
page.getByRole('dialog') // any dialog
page.getByRole('alertdialog') // alert dialogs (consent, delete confirm)
Prose shows a consent dialog on first launch in a fresh browser context. Dismiss it before testing:
async function dismissAIConsent(page: Page) {
const dialog = page.getByRole('alertdialog').filter({ hasText: 'AI Writing Assistance' })
if (await dialog.isVisible({ timeout: 3_000 }).catch(() => false)) {
// "Use Without AI" closes in one click. "Enable AI Features" advances
// OSS/web builds to a second step (skill download) that would also need
// dismissal — avoid by declining.
await page.getByRole('button', { name: 'Use Without AI' }).click()
await dialog.waitFor({ state: 'hidden' })
}
}
async function waitForAppReady(page: Page) {
await page.locator('[aria-label="Toggle theme"]').waitFor({ state: 'visible', timeout: 15_000 })
await dismissAIConsent(page)
}
Unlike Electron, browser keyboard shortcuts work reliably with Playwright:
// Ctrl+/ to open chat (web mode uses Ctrl instead of Cmd)
await page.keyboard.press('Control+/')
// Type in editor
await editor.click()
await page.keyboard.type('Hello world')
The web mock pre-loads these files at /Documents/:
| File | Description |
|---|---|
Welcome to Prose.md | Intro document, exercises basic editor |
Formatting Examples.md | Markdown formatting showcase |
Meeting Notes/Weekly Standup.md | Subdirectory — tests folder expansion |
Meeting Notes/Q1 Planning.md | Frontmatter parsing |
Blog Drafts/AI Writing Tools.md | Longer content |
Empty Note.md | Empty file edge case |
playwright.config.ts at the project root:
http://localhost:5174 (web mode dev server)json (outputFile) + github (inline PR annotations)list (human-readable terminal output)net::ERR_CONNECTION_REFUSEDThe dev server isn't running. Either:
npm run test:webnpm run dev:web then re-run tests.ProseMirrorThe editor may be blocked by the AI consent dialog or recovery dialog. Make sure you call dismissAIConsent() in your test setup.
The file list auto-initializes from getDocumentsPath(). If files aren't showing:
npm run dev:web console for errors[aria-label="Show files"] button was clicked firstThe initial theme (dark class on <html>) may differ if a previous test changed it. Use isolated browser contexts or clear localStorage between test runs:
test.beforeEach(async ({ page }) => {
await page.addInitScript(() => localStorage.clear())
})
Validate all LLM-accelerated GitHub issues through systematic QA testing via Circuit Electron.
Human-in-the-loop QA loop for verifying a large feature or refactor against the locally-running app before merge/release. Use when automated tests (e2e, web-e2e) don't cover the behavioral/UX surface and you need a structured, interactive "you drive / I observe" verification pass with in-session bug fixes. Complements qa-pr (automated, Circuit) and qa-web (Playwright).
Automated QA testing for Prose PRs using Circuit Electron. Use when testing pull requests before merge.
Implement one solo-ist/prose GitHub issue end-to-end and open a PR ready for review. Use when an Oz child agent is assigned exactly one issue to fix or build in this repo.
Reference and checklist for building and maintaining CI/CD workflows, inter-workflow communication, dispatch scripts, and cloud agent infrastructure. Use when modifying any workflow YAML, dispatch script, or sentinel-based communication.
Merge scorer and PE signals to route PRs. Applies routing matrix (hitl-light / hitl-full) based on score thresholds and risk levels.