| name | qa-heal |
| description | Self-healing for the generated test suite. Use when previously-green Playwright tests are failing, the user says "my tests broke", "fix the tests", "/qa-heal", or after a UI refactor breaks selectors. Classifies each failure (real bug vs test drift) and repairs the drift — never papers over real bugs.
|
QA Heal — Test Suite Self-Healing
Prime directive: a failing test is a signal; healing must never silence a
real bug. You repair tests that broke because the app legitimately changed.
You report (not "fix") tests that broke because the app is broken.
Step 1 — Run and collect
npx playwright test --reporter=json (or the project's runner). Collect every
failure with its error, trace, and the spec source.
Step 2 — Classify each failure
Four drift types plus the one that matters most:
| Type | Signal | Action |
|---|
| Selector drift | Locator not found, but the page snapshot shows the same element with new role/name/structure | Update locator to match current UI, prefer role/label |
| Flow drift | Step order changed (new onboarding screen, moved button, extra confirmation dialog) | Re-walk the flow in the browser, update steps to the new legitimate path |
| Data drift | Fixture/test data stale (deleted user, expired token, changed enum) | Regenerate fixture or create data via API in setup |
| Environment drift | Port changed, server slower to start, new env var required | Fix playwright.config webServer / env setup |
| REAL BUG | The app behavior genuinely violates the assertion's intent | DO NOT touch the test. Report it. |
Classification method (intent-based, not guesswork): open the app at the
failing step (Playwright MCP or --headed trace), take a snapshot, and ask
"can the user still accomplish this test's INTENT?" If yes → drift, heal it.
If no → real bug, report it.
When ambiguous, treat as a real bug and ask the user — a wrongly-healed test
is worse than a noisy one.
Step 3 — Heal
- Apply minimal edits per drift type. Keep the test's intent and assertions —
healing changes HOW the test gets there, never WHAT it verifies.
- Re-run each healed test 2x (green twice = healed; intermittent = mark flaky
and investigate separately).
Step 4 — Learn
Append to qa-reports/qa-profile.md → Quirks & Learnings: what drifted and
why (e.g. "2026-06-12 qa-heal: signup flow gained email-verification step —
flows touching signup must handle it"). Future audits and test generation
read this and don't trip on the same change.
Step 5 — Final response
HEAL: X healed (Y selector, Z flow, ...), W real bugs found, V flaky- Real bugs first, with evidence.
- Diff summary of healed specs.
