e2e-retrofit

name

e2e-retrofit

description

Multi-agent workflow for retrofitting a coverage-matrix section with Playwright e2e tests. Drives the full lifecycle: scope confirm, live-UI recon, implementation in a sub-agent, fresh-context review, fix application, a single confirmation run per environment, and wrap-up (matrix rows updated, branch pushed / PR opened, and — if a tracker is configured — the issue commented). Use when the user says 'retrofit e2e tests for {ticket}', 'add e2e coverage for {section}', 'cover {section} with Playwright tests', or any variant of 'apply the multi-agent e2e workflow'. Designed to survive a context reset — the conventions encoded here and in patterns.md are the source of truth across conversations.

E2E Test Retrofit Workflow

A repeatable workflow for retrofitting Playwright e2e coverage onto a section of an app, one batch at a time. It assumes a working e2e suite, helpers, and the Playwright MCP are already in place in the target project.

First time on this repo / no Playwright yet? Run the e2e-setup skill first — it's the one-time bootstrap that stands up the e2e suite, installs Playwright, registers the MCP, and wires the harness to the app. This skill (e2e-retrofit) is the many-times workflow you run after setup. Phase −1 below detects a missing suite and sends you there.

This document is the operational manual. Two companions, alongside this file:

• patterns.md — the load-bearing rules the workflow enforces (spec structure, selector strategy, assertion coverage, comment style, SPA hazards). Read the relevant section before each phase.
• examples-from-the-field.md — composite, synthetic hazards representative of real retrofits, kept as illustrations of the generic rules. Not drawn from any specific application; never normative; your app will differ.

Assumptions & portability

This workflow is portable across stacks, but it does make a few assumptions. Know them before running:

• A seedable state channel. The assertion strategy seeds state through the fastest deterministic channel the app exposes, then asserts in the UI. Prefer a contracted channel, in this order: REST/GraphQL API (login → token → create/get/delete, waiting on save responses) → a test-only seeding endpoint → a direct DB write → client-side storage (localStorage/IndexedDB) for apps with no server contract. The default harness (ApiClient + created-id registry + dispose) is API-shaped; when you fall back to DB or client storage, keep the seed deterministic, accept the coupling to the app's internal representation (more brittle than an API contract — document it in the spec), and handle cleanup accordingly (client storage is usually per-context, so needs none; a DB seed still needs teardown). Pure-UI-only apps adapt the seeding step itself — the rest of the workflow still applies.
• A coverage matrix in a known shape. The skill parses ### {Section} headers, [x] [x] dual checkboxes, [-]/[~] markers, and priority tags. The worked example is harness/COVERAGE.md (in the e2e-setup skill). A different tracking format (spreadsheet, TestRail/Xray, other markdown) needs the matrix steps adapted.
• The project's existing conventions win. Every naming/layout default below (crud-/manage- spec names, tests/{slug}/, helper filenames) is just the default on a fresh e2e-setup repo. If the suite already has its own conventions, mirror those (patterns.md §1).
• Optional integrations, not requirements. Issue tracker ($TRACKER) and code host ($CODE_HOST) are opt-in. With no tracker, the ticket is skipped entirely and naming falls back to a section-based scheme; with no code host, the workflow pushes the branch and prints a PR-ready summary. Nothing in the core loop requires either.

Configuration — fill these in for your project

The workflow is toolchain-agnostic. Map each placeholder to your stack before running:

Placeholder	Meaning
$DEFAULT_BRANCH	Trunk to branch from and target (main, master, …)
$LOCAL_BASE_URL / $REMOTE_BASE_URL	Local app URL and optional shared staging/QA URL
$DEV_UP	Command that brings local services up (npm run dev, docker compose up, make dev, …). Omit if you target a running staging/remote env instead.
$TEST_CMD / $REMOTE_TEST_CMD	Command to run specs locally / against the remote target — match the project's config (e.g. npx playwright test / E2E_TARGET=remote npx playwright test)
$COVERAGE_MATRIX	Path to the coverage checklist (e.g. e2e/COVERAGE.md)
{entity} / {section}	The feature area a batch covers
{ticket} / $TRACKER	Optional. Issue-tracker ID + the CLI/step that reads & comments on it (Jira, Linear, GitHub Issues, …). Omit entirely if the project has no tracker.
$CODE_HOST	Optional. Tool that opens a PR/MR (gh pr create, glab mr create, …). Omit to just push the branch + print a summary.

What the skill needs — required structure

This skill operates on a live e2e suite in the target project — it does NOT run tests from inside the skill folder. The e2e-setup skill produces this suite; this skill consumes it. Every phase reads, writes, and runs against the project's e2e directory (e.g. e2e/ or tests/e2e/ at the repo root), never the skill folder.

That directory must contain, at minimum:

<project-e2e-dir>/            # contains playwright.config.ts; all commands run from here
├── playwright.config.ts      # required — defines projects/targets; presence is how the skill finds the suite
├── package.json              # @playwright/test installed (npm/pnpm/yarn/bun install + playwright install <browser> done)
├── .env                      # base URLs + a test user, per target. Never committed
├── $COVERAGE_MATRIX          # the coverage checklist (e.g. COVERAGE.md) containing the `### {Section}` to scope
├── tests/                    # specs live here; folder-per-section (tests/{slug}/)
│   └── smoke.spec.ts         # always present after setup. Feature specs may not exist yet (see note)
└── helpers/                  # shared API client, name generators, modal/date helpers, response predicates

If playwright.config.ts / package.json / helpers/ are missing, the project isn't set up yet — run e2e-setup first. Phase −1 only verifies setup is present; it does not perform it.

First-batch case — no feature specs yet. e2e-setup writes only smoke.spec.ts; it deliberately does not ship canonical CRUD/Manage specs. So on a freshly-set-up repo tests/ holds just the smoke test, and the first retrofit batch establishes the structural baseline rather than mirroring an existing one. When the project has no feature spec to copy, use the example specs bundled in the e2e-setup skill (<e2e-setup>/harness/tests/) + patterns.md as the pattern reference; your first spec then becomes the project's canonical baseline for every batch after.

Phase −1 — Preflight / readiness gate

Run BEFORE Phase 0. Actively verify the environment — do not assume. Probe the project's e2e directory and the configured tooling, then emit the readiness report below and STOP for the user to ack. This replaces a passive prerequisites checklist: the point is to fail fast with a clear report, not to discover a missing .env key three phases deep.

The ### {Section} lookup needs the ticket/section from Step 0, so gather those first — run Phase −1 and Step 0 together in the opening exchange (environment checks first, complete the section check once the section is named).

Checks

Setup detection (run first): is there a project e2e directory with a playwright.config.ts?

• No → the repo isn't set up. Stop and direct the user to the e2e-setup skill ("set up e2e in this repo"). Do not attempt to bootstrap from here.
• Yes → set up. Continue the checks below against it.

Hard blockers — stop and report if any fail:

• git available; repo is clean enough to branch ($DEFAULT_BRANCH exists, working tree not mid-conflict).
• Project e2e dir found; playwright.config.ts present; deps installed; $TEST_CMD --list parses without error.
• Structural baseline resolved — either the project's canonical CRUD/Manage specs Phase 3/4 will mirror (name the exact files), or, on a freshly-set-up repo with only smoke.spec.ts, note that this is the first batch: it will establish the baseline, using the e2e-setup harness examples (<e2e-setup>/harness/tests/) + patterns.md as the pattern reference. Either resolves this check — a fresh repo is not a blocker.
• Required config placeholders resolved — base URLs, $DEFAULT_BRANCH, $DEV_UP, $TEST_CMD/$REMOTE_TEST_CMD, and $COVERAGE_MATRIX map to real values. Tracker, code host, and ticket are optional (soft checks below) and may be unset.
• $COVERAGE_MATRIX exists AND contains the ### {Section} the user named. A matrix without the section is a blocker — there's nothing to scope.
• .env wired — exists and carries the keys (base URL, API base, test user) for every target you'll run against.

Soft / degraded — proceed with the capability flagged, don't stop:

• Playwright MCP connected — /mcp lists it and mcp__playwright__browser_* tools are available → if absent, register it (claude mcp add playwright -s user -- npx @playwright/mcp@latest; see the e2e-setup skill Step 4); recon degrades to code-only until then (note it).
• $TRACKER CLI present → if absent, skip the ticket-read/comment steps and ask the user for ticket details.
• $CODE_HOST CLI present → if absent, print the branch + PR summary instead of opening a PR.
• Target environment reachable — for each target you'll run: local is up ($DEV_UP succeeds) or the staging/remote base URL responds. The app must be running somewhere before specs run.

Readiness report format

Emit this, then wait for the user to ack before Phase 0:

Preflight — e2e-retrofit
Setup: [ready | needs-setup → run e2e-setup]
Project e2e dir: [path or "none found"]
Canonical baseline specs: [crud spec path] + [manage spec path] | [none yet — first batch establishes baseline; pattern ref: e2e-setup harness examples]
Coverage matrix: [path] — section "{Section}" [found | MISSING]

Hard blockers: [none | list each failing check + what's needed]
Degraded capabilities: [none | e.g. "no Playwright MCP → code-only recon", "no $TRACKER → manual ticket entry"]

Ready to proceed: [yes | no — blocked on the above]

Do not create a branch or spawn any sub-agent until the user acks a Ready to proceed: yes.

Step 0 — Always start with clarifying questions

Do NOT begin work until you have answers. Ask in one round:

1. Ticket (optional) — if the batch is tracked, the {ticket} ID; look it up via $TRACKER for the branch name and PR title. No tracker? Skip it — use the fallback naming in Phase 0 and don't ask for ticket details.
2. Coverage section — which ### {Section} block in $COVERAGE_MATRIX is in scope. Read the rows; list IN-scope (default: priority High + Medium) and OUT-of-scope.
3. Priority filter — default High + Medium. Confirm or override.
4. File structure — default crud-{slug}.spec.ts + manage-{slug}.spec.ts (CRUD = create/edit/delete; Manage = the section's stateful operations). Override only if the section is too small to split.
5. Helper file — default is NO new helper file. The bar is high: ≥3 multi-step UI flows that would each be ≥30 lines inline, OR a genuinely complex shape (two-step modal, route interception, nested-modal picker). Otherwise import shared helpers and inline the rest. Confirm before adding.
6. Branch base — default $DEFAULT_BRANCH after a fast-forward pull. Don't stack unless explicitly asked.
7. Environments — local only, remote only, or both. Default both if a remote target is configured.
8. Stabilization budget — default 1 confirmation run per environment. Don't bump unless asked (see patterns.md §7).

Restate the plan after their answers and only proceed once they confirm.

Workflow phases

Track each phase with a task tracker. Checkpoint to the user with a 3–5 line summary between phases.

Phase 0 — Branch

git checkout $DEFAULT_BRANCH && git pull --ff-only
git checkout -b {ticket}-retrofit-e2e-{slug}   # with a ticket
# git checkout -b e2e-retrofit-{slug}          # no tracker — section-based fallback

Phase 1 — Scope confirm

Re-read the in-scope rows from $COVERAGE_MATRIX and list them with line numbers. Confirm wording matches the file. No code yet.

Phase 2 — Recon on the live UI

Delegate to a sub-agent with the briefing template in patterns.md §9. The agent drives the live UI and returns structured workstream notes. Required validation steps (see patterns.md §2):

• (a) Cross-reference observed request bodies against the server's model/schema — fields the UI doesn't always send must be flagged.
• (b) Note selector ambiguities — nav text collisions, multiple buttons with the same accessible name, list rows where a row filter could match a wrapper.
• (c) Confirm each reported selector resolves to a single element under strict mode.
• (d) Capture the actual GET response shape for any field a test will assert on — typed value objects, nested entity references, list-envelope keys. One verification request up front saves a stabilization cycle.

Phase 3 — Implementation

Delegate to a fresh sub-agent. Brief with:

• Phase 2 workstream notes.
• Structural baseline (read FIRST, before any other reference): your suite's two cleanest, most-current CRUD specs + their shared helper, under the project's tests/. Pick the specs your team agrees are canonical — NOT whatever is structurally nearest. First batch on a freshly-set-up repo: there are no feature specs yet (setup writes only smoke.spec.ts), so read the example specs in the e2e-setup skill (<e2e-setup>/harness/tests/) + patterns.md as the pattern reference, and write this batch as the project's new canonical baseline.
• Anti-analog warning: do NOT pick a structurally similar but drifted feature as the template. If you catch yourself thinking "this is like feature X, let me copy X's pattern," STOP and use the canonical baseline. See patterns.md §1.1.
• Helpers to import directly (API client, modal/date helpers, name generators). Search for existing patterns before writing new ones.
• The conventions block below + the rules in patterns.md.

Anti-drift call-outs to include in the brief verbatim (full list in patterns.md §1.1):

• No suite-wide timeout overrides (test.describe.configure({ timeout })) — use the default; scope test.setTimeout to a single test only if genuinely needed.
• No test.setTimeout inside beforeAll.
• No macro uiCreate{Entity} helpers — inline the create flow in the test.
• No findAndOpen{Entity}FromSearch helpers — use direct URL navigation + a visibility assertion (or a page.reload() for forced fresh state).
• No granular uiEdit{Entity}{Field} helpers — inline click → fill → blur → waitForResponse(isXSave(id)) per field.
• No defensive .first() on locators that should be unique. Strict-mode failure is a signal to fix the locator, not paper over it. Reserve .first() for genuine list ambiguity.
• Prefer getByRole('textbox', { name, exact: true }) for labelled inputs. CSS attribute selectors only when no role/label exists.
• WHY comments are single-line. Multi-line "tutorial" comments belong in the PR description, not the code.

Phase 3.5 — Drift audit (orchestrator-side, before review)

Run the grep recipe from patterns.md §1.1.1 against the new files. Any hit is a drift item the orchestrator fixes BEFORE handing to the reviewer. Non-negotiable: the implementation agent had the rules in its brief and may still have drifted, so don't expect the reviewer to be the only line of defence. Post the audit results to the user verbatim.

Phase 4 — Review + best-practice audit (fresh sub-agent)

Brief a NEW sub-agent (no Phase 3 context) with the diff vs $DEFAULT_BRANCH + reference files. Output: numbered recommendations sorted blocker → should-fix → nit (plus pattern-drift, below). The reviewer must NOT edit files.

The brief MUST start with the drift checklist as Y/N items, not as one of many "evaluation axes." The reviewer answers each before considering anything else; any NO becomes a should-fix at minimum. Full checklist + eval axes in patterns.md §9.

Tri-source audit (build → audit → assess; patterns.md §14). Phase 3 built the spec against the stored patterns; this phase audits the written tests against three trusted sources: (1) Playwright's live documentation — fetch playwright.dev (the best-practices, locators, and assertions pages, plus the release notes for the installed @playwright/test version); (2) patterns.md — the cultivated conventions; (3) the repo's existing specs — the prevailing local idioms (locator/assertion/fixture/wait style), so usage doesn't drift batch to batch. Audit the tests, not the patterns: where a test that faithfully follows patterns.md contradicts current Playwright guidance, tag it [pattern-drift] — that contradiction signals a stale rule, not a spec bug. A pattern-drift finding is recommended at wrap-up, never silently spec-patched (don't rewrite the spec to break the documented convention). The live audit is prose; it adds no items to the §1.1.1 grep or the §9.3 Y/N checklist.

Phase 5 — Apply fixes

Orchestrator applies all blocker/should-fix items + meaningful nits. Note any rejected with reason. Re-run the spec parse ($TEST_CMD {path} --list) to catch syntax errors.

Phase 6 — Single confirmation run per environment

$TEST_CMD tests/{slug}            # local   (e.g. npx playwright test)
$REMOTE_TEST_CMD tests/{slug}     # remote  (if configured; e.g. E2E_TARGET=remote npx playwright test)

Per patterns.md §7: one run per env is enough — 3-run sweeps are statistical theatre. If a test fails, classify as real-bug | flake | env-issue and fix only flake/env. Stop and report on real bugs. Time-box per patterns.md §8 (~3 fix cycles per failing test before checkpointing).

Phase 7 — Wrap-up

• Mark in-scope rows in $COVERAGE_MATRIX [x] [x] (or [-] [x] for env-gated tests; [~] for known coverage gaps) with the spec path.
• Don't forget peripheral rows the round-trip transitively covers — e.g. a reopen-from-search flow exercises the Search row too; mark it if applicable.
• Recompute the coverage summary table if your matrix has one: per-priority and total. [-] and [~] rows stay in the denominator but don't count as covered; drop any rows for workflows that turned out not to exist.
• Commit per phase ideally; a final wrap-up commit is acceptable.
• Push the branch with -u.
• If $CODE_HOST is configured: open a PR (draft if the host supports it) titled {ticket}: {summary} — or e2e: {section} coverage when there's no ticket. Otherwise: print the branch name + a PR-ready summary for the user to open manually.
• If a tracker is configured: comment on {ticket} via $TRACKER with the PR URL + a short summary; leave the ticket status alone — let the user transition it. No tracker → skip this step.
• Assess pattern drift (patterns.md §14). If the Phase 4 audit raised any [pattern-drift] findings, surface each to the user with its trusted-source citation and a proposed patterns.md / examples-from-the-field.md update. These are recommendations the user gates — apply accepted ones per §13; never auto-edit the patterns as part of the retrofit. No drift found → say so in one line.
• Final summary to the user — required, not optional. Format below.

Final summary format

Open with the high-level state (branch, PR URL or "branch pushed", tracker commented if applicable, test counts per env). Then for each test in the new specs, in spec-file order:

1. Test name — quote it verbatim with the spec file + line number (e.g. crud-{slug}.spec.ts:112).
2. Steps — numbered, terse. The user-action chain: open this, click that, fill X, submit. Name helper calls where used.
3. Asserts (UI) — what's checked from the rendered DOM: input value, row visibility, pill text, URL pattern, count(0) for negative.
4. Asserts (API) — what's read back via the API client. Field path + expected value.
5. Pre-conditions if any — when state was set up via API, what was asserted before the action.

Close with an assertion-source split table (rows = tests, columns = "UI assertions" / "API assertions"). Makes it obvious at a glance whether each test follows the UI-primary pattern (patterns.md §3) or genuinely diverges (e.g. an unreliable DOM rendering, patterns.md §4).

The summary is the user's main artifact for the retrofit — they should be able to ack it without opening the spec files. Aim for ≤500 words; group by spec file with sub-headings if the section is large.

Conventions

Branch / folder / files

• The project's existing convention always wins. The names below are the defaults on a fresh e2e-setup repo; if the suite already uses different names or layout, mirror that instead (patterns.md §1).
• Branch: {ticket}-retrofit-e2e-{slug} (with a ticket) or e2e-retrofit-{slug} (no tracker) — slug is lowercase kebab.
• Folder: tests/{slug-lowercase}/ — match the existing convention (lowercase, no spaces).
• Spec files: crud-{slug}.spec.ts + manage-{slug}.spec.ts, mirroring the canonical reference specs. CRUD = create/edit/delete; Manage = the section's stateful operations (add line items, status changes, etc.). Don't pick a drifted feature as the structural reference (patterns.md §1.1).
• Helper: helpers/{slug}.ts only if the entity needs genuinely complex multi-step UI flows (two-step modals, route interception, two-step delete confirm). Most retrofits should NOT add a file — extend shared API helpers for seeding and inline single-flow UI driving in the test.

Seed data naming

All test data prefixed e2e- plus a timestamp for uniqueness, via generators in helpers/names.ts (or your suite's equivalent name-generator helper). Add new generators there when the section needs new data shapes — keep the convention. Populate every recon-catalogued field, not just the happy-path minimum (patterns.md §6.5).

Test describe + naming

• test.describe('{Section} > CRUD') and test.describe('{Section} > Manage') — match the coverage-matrix row prefixes.
• Test descriptions match the workflow row name where reasonable (use the user-facing term, not the internal field name).
• Use the default timeout. If one test genuinely needs longer, scope test.setTimeout inside it.

Assertion strategy (patterns.md §3–4)

• CRUD tests assert UI and persistence via close-and-reopen.
• Toasts are noise — don't assert them.
• Negative assertions explicit: toHaveCount(0) / not.toBeVisible().
• For state-mutation tests: assert the touched field changed AND invariants stayed put. Pre-condition assertions count.
• API reads complement UI assertions — they don't replace them. Use API-only when the DOM is genuinely unreliable.

Selector strategy (patterns.md §2)

Before writing any new locator: identify the element shape, grep helpers/ and tests/ for that shape, reuse the working pattern. Treat recon selectors as hypotheses to validate, not gospel.

Comments + scope

• Comments minimal, only WHY, never WHAT (patterns.md §5).
• Stay strictly in scope — don't bundle unrelated improvements (patterns.md §6). Flag them and let the user decide.

Sub-agent briefing templates

Full templates live in patterns.md §9. In short:

• Phase 2 (recon): drive the live UI, return structured workstream notes only, do NOT write test code. Close the browser when done. Prefix any created data with e2e-.
• Phase 3 (implement): Phase 2 notes + canonical reference files + conventions + patterns.md rules. Return files written, helpers reused vs added, decisions not dictated by the brief.
• Phase 4 (review + audit): fresh context, diff + reference files, drift checklist first, then the tri-source audit (Playwright live docs + patterns.md + repo specs); numbered [blocker|should-fix|nit|pattern-drift] output, no edits.

Wrap-up checklist

• Phase 3.5 drift audit ran clean (or fixes applied + re-ran clean)
• Phase 4 tri-source audit ran (Playwright live docs + patterns.md + repo specs); any [pattern-drift] findings surfaced to the user (§14)
• $COVERAGE_MATRIX — every in-scope row marked
• $COVERAGE_MATRIX — peripheral rows transitively covered (e.g. Search)
• $COVERAGE_MATRIX — summary table recomputed if present
• All commits pushed
• PR opened (draft if supported) with a summary + run-results table per env — or branch pushed + summary printed if no $CODE_HOST
• {ticket} commented with the PR link — only if a tracker is configured
• Confirmation runs green for every targeted env
• No scope-creep changes outside the section's helpers/specs
• Final per-test summary posted to the user

When the workflow goes off-rails

Stop and checkpoint with the user when:

• A real bug is found (not flake / env divergence) — let them decide if it's in scope.
• Phase 6 fix iteration exceeds ~3 cycles per failing test.
• Recon reveals a workflow needs configuration the target env lacks — confirm skip vs seed.
• Scope drift starts (new helpers needed for unrelated rows, refactors that touch other specs).