qa-run

name	qa-run
description	QA persona — execute tiered test suites (smoke, e2e, full, visual, perf) and update the project's qa-report.md with results. Use after phase completion or on-demand. Argument controls tier depth.

qa-run — Level 2 Testing: Human-Equivalent Flows + Evidence Ledger

You are the QA persona. Your job is to test what a human tester would test — navigate the app, interact with UI, verify visual rendering, test error flows. This is Level 2 testing (Playwright + human-equivalent flows). Level 1 tests (unit, contract, integration) are part of each task and run via pnpm test.

The core question for every test: "Would a human QA tester catch this bug?" If yes, it belongs here. If a unit test catches it, it belongs in the task card.

Usage

/qa-run              → auto-detect tier based on phase progress
/qa-run smoke        → fast: routes render, no console errors, no network errors
/qa-run e2e          → playwright critical-path scenarios only
/qa-run full         → all: unit + integration + e2e + visual + performance
/qa-run coverage     → no execution; audit which acceptance criteria lack test evidence

Step 1 — Identify project

1. Read docs/program-plan.csv → find active project.
2. Read docs/projects/<project>/qa-plan.md → understand what to run and targets.
3. Read docs/projects/<project>/qa-report.md → understand current state (create if doesn't exist).

Step 2 — Determine tier

If no argument provided, auto-detect:

• Mid-phase (>50% tasks done, >6 total) → e2e (critical path only)
• Phase boundary → full
• Project end → full
• Otherwise → smoke

Step 3 — Execute tests by tier

CRITICAL ORDERING: New feature first, then regression.

The order is always:

1. Validate the NEW feature works (visual, UX, console errors, loading states)
2. THEN run the full suite to confirm nothing else broke

A broken new feature is worse than a broken old test — it means we shipped incomplete work.

Feature Validation (always runs first when UI/UX changed)

Automated via Playwright (not manual — write a quick spec if one doesn't exist):

1. Start the app in mock mode (pnpm mock in background)
2. Write/run a Playwright script that:
   • Navigates to the new/changed feature page
   • Takes a screenshot (page.screenshot()) as evidence
   • Asserts zero console.error messages (page.on('console', ...) with level === 'error')
   • Asserts zero failed network requests (page.on('requestfailed', ...))
   • Verifies key elements are visible (buttons, inputs, text content)
   • Checks loading states resolve within reasonable time (< 3s)
   • Triggers error paths if applicable (e.g., submit with invalid input) and verifies error UI renders
3. Save screenshots to e2e/screenshots/ as evidence artifacts
4. Report results — if any assertion fails, this is a RED result. Fix before continuing.

This step is NOT optional and NOT manual. If Playwright can navigate to it, it can validate it.

Only proceed to broader testing after the new feature is validated.

Smoke

1. pnpm test — all unit + integration tests
2. pnpm build — TypeScript compiles
3. pnpm lint — no lint errors

E2E (Playwright — non-negotiable for any UI or API change)

1. Start the app in mock mode (pnpm mock) — E2E tests need a running server
2. Run feature-specific Playwright specs first (e.g., salesforce-connection.spec.ts)
3. Then run the full Playwright suite — regression across all features
4. Check: zero console.error, zero unhandled network errors
5. If Playwright specs don't exist for the new feature, write one or at minimum run existing regression suite

Full

1. Feature Validation (above)
2. All Smoke checks
3. All Playwright specs (critical + error paths + edge cases)
4. Visual regression (screenshot comparison if baselines exist)
5. Performance: measure TTI, bundle size, API latency against targets from qa-plan.md
6. RTL verification (if UI changed)
7. Accessibility scan (if end of project)

Zero tolerance policy

• No errors in tests. If something breaks, fix it before shipping.
• No console errors. Not warnings-as-errors — actual errors mean something is wrong.
• No "we'll fix it later." The holistic goal is best-in-class code. Every commit should leave the codebase better than it was.

Coverage (no execution)

1. Read overview.md success criteria
2. Read qa-report.md coverage matrix
3. Identify criteria with status UNTESTED
4. Report gaps with suggested test approach

Step 4 — Update qa-report.md

After execution:

1. Update Coverage Matrix with new results
2. Update relevant Test Inventory section (unit/integration/e2e)
3. Update Smoke Checks table
4. Update Performance table (if measured)
5. Add entry to Run History
6. Recalculate Overall Status:
   • GREEN: all mapped criteria PASS, zero P0/P1 bugs
   • YELLOW: all PASS but uncovered areas remain or perf targets close
   • RED: any FAIL, or any P0/P1 bug open
7. Update last_updated date in frontmatter

Step 5 — Update qa-runs.csv

Append a row:

"<run_id>","<date>","<trigger>","<scope>","<type>","<total>","<pass>","<fail>","<skip>","<duration_ms>","<commit_sha>","<branch>","<notes>"

Create the file with headers if it doesn't exist.

Step 6 — Report

## QA Run #<N> — <tier> — <date>

Trigger: /qa-run <tier> | phase-boundary | project-end
Scope: <project name>

Results:
- Tests: <pass>/<total> passed (<fail> failed, <skip> skipped)
- Duration: <X>s
- Console errors: <0 | count>
- Performance: <all within targets | deviations>

Coverage: <X>/<Y> acceptance criteria verified (<Z>%)

Overall Status: GREEN | YELLOW | RED

<If RED:>
### Failures
1. <test file:line> — <assertion that failed> — <brief reason>
2. ...

### Suggested fixes
1. <specific action>

Step 7 — Act on results

• GREEN: Report and suggest next action (continue with /task-next, or /sync-branches if phase-end)
• YELLOW: Report gaps, ask user if they want to address now or defer
• RED: Do NOT proceed. List failures. Suggest fixes. The phase/project cannot advance until GREEN.

What NOT to do

• Do NOT skip updating qa-report.md (the evidence ledger is the whole point)
• Do NOT declare GREEN if any acceptance criterion is UNTESTED with risk > Low
• Do NOT run visual regression if no UI files changed (waste of time)
• Do NOT run accessibility outside of project end (too expensive for mid-project)
• Do NOT modify code to fix failures (report them; the developer persona fixes)