メニュー
Validate Prose features against the web mode build using Playwright. Use for CI-compatible QA that doesn't require Electron or a display server.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Validate all LLM-accelerated GitHub issues through systematic QA testing via Circuit Electron.
Human-in-the-loop QA loop for verifying a large feature or refactor against the locally-running app before merge/release. Use when automated tests (e2e, web-e2e) don't cover the behavioral/UX surface and you need a structured, interactive "you drive / I observe" verification pass with in-session bug fixes. Complements qa-pr (automated, Circuit) and qa-web (Playwright).
Automated QA testing for Prose PRs using Circuit Electron. Use when testing pull requests before merge.
Implement one solo-ist/prose GitHub issue end-to-end and open a PR ready for review. Use when an Oz child agent is assigned exactly one issue to fix or build in this repo.
Reference and checklist for building and maintaining CI/CD workflows, inter-workflow communication, dispatch scripts, and cloud agent infrastructure. Use when modifying any workflow YAML, dispatch script, or sentinel-based communication.
Merge scorer and PE signals to route PRs. Applies routing matrix (hitl-light / hitl-full) based on score thresholds and risk levels.
| name | qa-web |
| description | Validate Prose features against the web mode build using Playwright. Use for CI-compatible QA that doesn't require Electron or a display server. |
Automated QA testing for Prose using Playwright against the web mode build (http://localhost:5174). Unlike Circuit Electron, this runs headlessly in CI — no display server or Electron required.
/qa-web
Or target a specific PR branch:
/qa-web <branch-name>
Playwright browsers must be installed once:
npx playwright install --with-deps chromium
If testing a specific PR branch:
git fetch origin
git checkout <branch-name>
With auto-managed dev server (recommended for local development):
npm run test:web
The Playwright config automatically starts npm run dev:web and waits for http://localhost:5174 to be ready.
Against an already-running dev server (if npm run dev:web is already running):
PLAYWRIGHT_TEST_BASE_URL=http://localhost:5174 npx playwright test
Against the production build (recommended for CI — faster, no HMR overhead):
npm run build:web
npm run preview:web &
npx playwright test --config playwright.config.ts
| Status | Meaning |
|---|---|
| ✅ passed | Feature works as expected |
| ❌ failed | Bug detected — check error message and screenshot |
| ⚠️ flaky | Timing issue — re-run with --retries=2 |
On failure, Playwright saves a screenshot and trace under test-results/. Open the trace viewer:
npx playwright show-trace test-results/<test-name>/trace.zip
Post a comment on the PR with results:
gh pr comment <pr-number> --body "$(cat <<'EOF'
## Web QA Results
**Branch**: \`<branch>\`
**Tested**: $(date -u +"%Y-%m-%dT%H:%M:%SZ")
### Results
| Test | Status | Notes |
|------|--------|-------|
| App loads without errors | ✅ | |
| File explorer renders files | ✅ | |
| Clicking file loads content | ✅ | |
| Editor accepts input | ✅ | |
| Settings panel opens | ✅ | |
| Theme toggle works | ✅ | |
### Recommendation
✅ **Ready for Review** — All smoke tests pass.
EOF
)"
The smoke test suite at tests/smoke.spec.ts covers:
dark class on <html>Add tests to tests/smoke.spec.ts or create new files under tests/.
// Editor
const editor = page.locator('.ProseMirror')
// Toolbar buttons
page.getByRole('button', { name: /show files/i }) // file list toggle
page.getByRole('button', { name: /hide files/i })
page.getByRole('button', { name: /toggle theme/i })
page.getByRole('button', { name: /more options/i }) // overflow menu
// File items in sidebar
page.getByText('Welcome to Prose.md')
page.getByText('Formatting Examples.md')
// Dialogs
page.getByRole('dialog') // any dialog
page.getByRole('alertdialog') // alert dialogs (consent, delete confirm)
Prose shows a consent dialog on first launch in a fresh browser context. Dismiss it before testing:
async function dismissAIConsent(page: Page) {
const dialog = page.getByRole('alertdialog').filter({ hasText: 'AI Writing Assistance' })
if (await dialog.isVisible({ timeout: 3_000 }).catch(() => false)) {
// "Use Without AI" closes in one click. "Enable AI Features" advances
// OSS/web builds to a second step (skill download) that would also need
// dismissal — avoid by declining.
await page.getByRole('button', { name: 'Use Without AI' }).click()
await dialog.waitFor({ state: 'hidden' })
}
}
async function waitForAppReady(page: Page) {
await page.locator('[aria-label="Toggle theme"]').waitFor({ state: 'visible', timeout: 15_000 })
await dismissAIConsent(page)
}
Unlike Electron, browser keyboard shortcuts work reliably with Playwright:
// Ctrl+/ to open chat (web mode uses Ctrl instead of Cmd)
await page.keyboard.press('Control+/')
// Type in editor
await editor.click()
await page.keyboard.type('Hello world')
The web mock pre-loads these files at /Documents/:
| File | Description |
|---|---|
Welcome to Prose.md | Intro document, exercises basic editor |
Formatting Examples.md | Markdown formatting showcase |
Meeting Notes/Weekly Standup.md | Subdirectory — tests folder expansion |
Meeting Notes/Q1 Planning.md | Frontmatter parsing |
Blog Drafts/AI Writing Tools.md | Longer content |
Empty Note.md | Empty file edge case |
playwright.config.ts at the project root:
http://localhost:5174 (web mode dev server)json (outputFile) + github (inline PR annotations)list (human-readable terminal output)net::ERR_CONNECTION_REFUSEDThe dev server isn't running. Either:
npm run test:webnpm run dev:web then re-run tests.ProseMirrorThe editor may be blocked by the AI consent dialog or recovery dialog. Make sure you call dismissAIConsent() in your test setup.
The file list auto-initializes from getDocumentsPath(). If files aren't showing:
npm run dev:web console for errors[aria-label="Show files"] button was clicked firstThe initial theme (dark class on <html>) may differ if a previous test changed it. Use isolated browser contexts or clear localStorage between test runs:
test.beforeEach(async ({ page }) => {
await page.addInitScript(() => localStorage.clear())
})