team-qa

# Team QA When this skill is invoked, orchestrate the QA team through a structured testing cycle. **Decision Points:** At each phase transition, use `AskUserQuestion` to present the user with the subagent's proposals as selectable options. Write the agent's full analysis in conversation, then capture the decision with concise labels. In `collaborative` mode, the user must approve before moving to the next phase. In `guided` mode the pipeline advances automatically unless a phase is BLOCKED; in `autonomous` mode it runs end to end, recording each phase outcome via `log_decision`. Decisions in `automation_always_ask` categories (`is_always_ask_category` helper) always prompt regardless of mode. See `.claude/docs/automation-modes.md`. ## Phase 0: Resolve Config !`bash "${CLAUDE_SKILL_DIR}/../../hooks/yaml-helper.sh" resolve_config --keys review_mode,automation,team.size,qa.level` Resolved above — use as-is; `--review` overrides `review_mode`. No block → defaults in `.claude/docs/config-resolution.md`. `qa.level` decides what a Logic, Integration or Config/Data story's evidence is: its automated test at `standard` and `full`; at `minimal`, where tests are waived (`.claude/docs/coding-standards.md`), the acceptance-criteria walk in Phase 5 — a story without a test is then WAIVED, not missing its evidence. The UI and Visual/Feel screenshots are required at every level. `review_mode` sets director-gate depth, and this pipeline has no director gate: no phase below spawns CD-, TD-, PR- or AD-PHASE-GATE, at any `review_mode`. Its phase gates are the pipeline's own decision points (defined under `team.size` below), and the agents that work at them are team members, not director gates. `automation` drives the Decision Points note above. See the Decision Points note above and `.claude/docs/automation-modes.md` for how each mode changes pipeline behavior. **`team.size`**: which agents are active (orthogonal to review_mode gate-depth and workflow docs). - **`individual`** (default): `qa-tester` only; `qa-lead` invoked at phase gates only. - **`small`**: `qa-lead` + `qa-tester` pipeline (as documented). - **`studio`**: `qa-lead` + per-story `qa-tester` spawn + sign-off. A non-core agent needed at `individual` routes through the nearest active core agent with an informational note. **"Phase gate" means any phase that ends in an `AskUserQuestion` decision point this pipeline itself lists** — a transition under Decision Points above, or a **Gate** step written into the pipeline below — **whatever the `automation` mode**: in `guided` or `autonomous`, where that question is skipped or logged, the phase is still a gate. Not every phase is one. Apply the test literally: if the phase below lists no decision point, it is not a gate, and an agent restricted to "phase gates only" is not spawned for it. This active-set scoping applies throughout the pipeline below: any phase that names an agent outside the active set routes through the nearest core agent rather than spawning it. **Announce the active set before Phase 1 — never let the collapse be silent.** Before spawning anything, state in one line which agents this run will actually spawn, and which the pipeline below names but will **not** spawn at the resolved `team.size`. For example: > `Active set (team.size: <resolved>): <the agents listed for that size above>.` > `Not spawned this run: <every other agent this pipeline names> — consulted` > `through <nearest active core agent>. Raise team.size (or modes.rigor) to widen.` Fill it from the `team.size` list directly above and the agents this file's own pipeline names — not from an example. Both sets differ per orchestrator. An agent the resolved size spawns only at phase gates (`qa-lead` at `individual`) goes in the active set with that qualifier — it is spawned, not consulted through another agent. The pipeline below reads as a multi-agent fan-out and at the shipped default it is one or two agents — `team-release` names ten and runs one, `team-narrative` names six across five phases and runs `writer` alone. **The collapse is correct**: `team.size` is rigor-fronted and the narrow default is the token lever. Without saying so, a reader cannot distinguish a correctly-collapsed run from a broken pipeline, and the per-agent "routes through the nearest core agent with an informational note" rule above fires at routing time and never states the shape of the run as a whole. This is the same rule as the skipped-check reporting elsewhere in this file: **a constraint that is enforced but never surfaced is indistinguishable, to the person reading the output, from one that was never enforced.** ## Team Composition - **qa-lead** — QA strategy, test plan generation, story classification, sign-off report - **qa-tester** — Test case writing, bug report writing, manual QA documentation ## How to Delegate Use the `Agent` tool to spawn each team member as a subagent: - `subagent_type: qa-lead` — Strategy, planning, classification, sign-off - `subagent_type: qa-tester` — Test case writing and bug report writing **Brief each agent — do not dump context.** Read the shared inputs **once** and pass a distilled brief inline: the lines each agent actually needs, never a file path for a document you have already read (an agent handed a path re-reads the whole file). Pass a path only for a document you have not read and only that agent needs. **End every agent prompt with a return contract:** "Write your full output to `[path]` — that named path is your write authorisation under the bounded exception below, so write it without a separate approval prompt. Return **only** (1) the path written, (2) a ≤5-bullet summary of decisions, (3) any BLOCKED/CONCERNS items, one line each. Do not restate the documents you read." Without it, an agent returns everything it read back into this session. > **Why this does not violate the Collaboration Protocol.** `CLAUDE.md` requires an agent to ask "May I write this to [filepath]?" before Write/Edit. A subagent spawned here writes **without** asking, and that is a deliberate, bounded exception rather than an oversight — the same call already made for `consistency-check` appending to `active.md`. The exception holds only when all three are true: (1) the path is one **you** named in the prompt, so the user approved the destination when they approved the phase; (2) it is a new artifact under `production/`, `docs/` or `tests/`, never an edit to existing source or config; (3) the phase that produced it is a phase gate as defined above — one the pipeline lists as ending in an `AskUserQuestion` before it advances, whatever the automation mode. Outside those three, the agent must ask. **Do not "fix" this by asking per subagent** — a prompt per agent per phase makes an orchestrator unusable, which is why the exception exists. At `team.size: studio`, Phase 4 spawns one qa-tester per story, in parallel; below that, one spawn covers the whole group. ## Pipeline ### Phase 1: Load Context Before doing anything else, gather the full scope: 1. Detect the current sprint or feature scope from the argument: - If argument is a sprint identifier (e.g., `sprint-03`): Glob `production/sprints/sprint-*.md` and match by sprint number — `sprint-3`, `sprint-03` and `sprint-003` all name sprint 3 (`/sprint-plan` writes `sprint-NNN.md`). Read the matched file. If multiple match, use the most recently modified. - If argument is `feature: [system-name]`: glob story files tagged for that system - If no argument: read `production/session-state/active.md` and `production/sprint-status.yaml` (if present) to infer the active sprint. If there is no sprint — at `rigor: minimal` there are none — ask which epic to cover and use `feature: [epic-slug]`; do not invent a sprint 2. Read `project.stage` from `project.yaml` (fallback `production/stage.txt`) to confirm the current project phase. 3. **Read the open bugs in scope** — the bugs already on file, before QA finds new ones: grep the `**Severity**`, `**Status**` and `**System**` lines of `production/qa/bugs/*.md`, as `/bug-triage` does. A bug is open unless its Status is `Closed` or `Verified Fixed`; it is in scope when its System is one the in-scope stories belong to, or it names an in-scope story — when unsure, list it. These go into the Phase 2 and Phase 6 briefs as **Open bugs in scope**: an open S1 or S2 here already makes the sign-off NOT APPROVED (Phase 6), whatever QA finds. No bug files → "Open bugs in scope: none on file". 4. Count stories found and report to the user: > "QA cycle starting for [sprint/feature]. Found [N] stories. Open bugs in scope: [N — S1: n, S2: n, or none on file]. Current stage: [stage]. Ready to begin QA strategy?" ### Phase 2: QA Strategy (qa-lead) Spawn `qa-lead` via `Agent` to review all in-scope stories and produce a QA strategy. Pass the qa-lead the **Open bugs in scope** from Phase 1 (ID, severity, story or system), then prompt it to: - Read each story file - Classify each story by type: **Logic** / **Integration** / **Visual/Feel** / **UI** / **Config/Data** - Identify which stories require automated test evidence vs. manual QA — at `qa.level: minimal` none requires a test: a Logic, Integration or Config/Data story without one reads `WAIVED` under Automated Required and gets its acceptance criteria walked in manual QA - Flag any stories with missing acceptance criteria or missing test evidence that would block QA - Estimate manual QA effort (number of test sessions needed) - **Before assessing smoke status, check for an existing smoke check report**: Glob `production/qa/smoke-*.md` and read the most recently modified file (if found). If a report exists, use its verdict and findings directly — do not re-interview the user. If no report exists, note: "No prior smoke check report found — run `/smoke-check sprint` before proceeding." and set smoke check status to UNKNOWN (treat as PASS WITH WARNINGS for the purpose of continuing). Produce a smoke check verdict: **PASS** / **PASS WITH WARNINGS [list]** / **NOT ASSESSED [what did not run]** / **FAIL [list of failures]** / **UNKNOWN (no report found)** - Produce a strategy summary table and smoke check result: | Story | Type | Automated Required | Manual Required | Blocker? | |-------|------|--------------------|-----------------|----------| **Smoke Check**: [PASS / PASS WITH WARNINGS / NOT ASSESSED / FAIL / UNKNOWN] — [source: `production/qa/smoke-[date].md` or "no report found"] — [details if not PASS] **Open bugs in scope**: [BUG-ID — S[n] — story or system, one per line; or "none on file"] — an open S1 or S2 is named here as a known failure If the smoke check result is **FAIL**, the qa-lead must list the failures prominently. QA cannot proceed past the strategy phase with a failed smoke check. Present the qa-lead's full strategy to the user, then use `AskUserQuestion`: ``` question: "QA Strategy Review" options: - "Looks good — proceed to test plan" - "Adjust story types before proceeding" - "Skip blocked stories and proceed with the rest" - "Smoke check failed — fix issues and re-run /team-qa" - "Cancel — resolve blockers first" ``` If smoke check **FAIL**: do not proceed to Phase 3. Surface the failures from the smoke check report and stop. The user must fix them, re-run `/smoke-check sprint`, and then re-run `/team-qa`. If smoke check **UNKNOWN**: surface a warning — "No smoke check report found. Recommend running `/smoke-check sprint` before QA. Proceeding with caution." If smoke check **NOT ASSESSED**: surface what the report says did not run — "The last smoke check could not assess [what did not run]. Recommend re-running `/smoke-check sprint` once it can. Proceeding with caution." — and continue as for UNKNOWN; the sign-off cannot be APPROVED on it (Phase 6). If smoke check **PASS WITH WARNINGS**: note the warnings for the sign-off report and continue. If blockers are present: list them explicitly. The user may choose to skip blocked stories or cancel the cycle. ### Phase 3: Test Plan Generation Using the strategy from Phase 2, produce a structured test plan document. The test plan should cover: - **Scope**: sprint/feature name, story count, dates - **Story Classification Table**: from Phase 2 strategy - **Automated Test Requirements**: which stories need test files, expected paths in `tests/` (none at `qa.level: minimal`: list the waived stories under Manual QA Scope instead) - **Manual QA Scope**: which stories need manual walkthrough and what to validate - **Out of Scope**: what is explicitly not being tested this cycle and why - **Entry Criteria**: what must be true before QA can begin. Always include: (1) Smoke check PASS or PASS WITH WARNINGS report exists at `production/qa/smoke-*.md`, (2) build is stable (no crashes on launch), (3) at sprint scope, all Must Have stories are `in-progress`, `review` or `done` in `production/sprint-status.yaml`; at `feature:` scope, every in-scope story file's Status is In Progress, In Review or Complete. Add any sprint-specific criteria beyond these. - **Exit Criteria**: what constitutes a completed QA cycle (all stories PASS or FAIL with bugs filed) Ask: "May I write the QA plan to `production/qa/qa-plan-[sprint]-[date].md`?" Write only after receiving approval. ### Phase 4: Test Case Writing (qa-tester) > **Smoke check** is performed as part of Phase 2 (QA Strategy). If the smoke check returned FAIL in Phase 2, the cycle was stopped there. This phase only runs when the Phase 2 smoke check was PASS, PASS WITH WARNINGS, NOT ASSESSED, or UNKNOWN. For each story requiring manual QA (Visual/Feel, UI, Integration without automated tests — and, at `qa.level: minimal`, where tests are waived, every Logic, Integration and Config/Data story without a test: its cases walk its acceptance criteria): Spawn `qa-tester` via `Agent` — one per story, in parallel, at `team.size: studio`; one spawn covering the whole group at `small` and `individual` — providing, for each story: - The story file path - The relevant section of the QA plan for that story - The GDD acceptance criteria for the system being tested (if available) - Instructions to write detailed test cases covering all acceptance criteria - **The output path: `production/qa/test-cases/[story-slug]-cases.md`.** Name it explicitly in the prompt, one per story. > **Why the path is stated here rather than left to the orchestrator.** The > bounded write exception above holds only when "the path is one **you** named in > the prompt". This is the phase that spawns agents *in parallel*, so it is where > an unnamed destination does the most damage: each agent improvises its own, and > two runs file the same artifact in two places. The phase reads correctly right > up until two agents need somewhere to put their output. Each test case set should include: - **Preconditions**: game state required before testing begins - **Steps**: numbered, unambiguous actions - **Expected Result**: what should happen - **Actual Result**: field left blank for the tester to fill in - **Pass/Fail**: field left blank Present the test cases to the user for review before execution. Group by story. Use `AskUserQuestion` per story group (batched 3-4 at a time): ``` question: "Test cases ready for [Story Group]. Review before manual QA begins?" options: - "Approved — begin manual QA for these stories" - "Revise test cases for [story name]" - "Skip manual QA for [story name] — not ready" ``` ### Phase 5: Manual QA Execution Walk through each story in the approved manual QA list. Batch stories into groups of 3-4 and use `AskUserQuestion` for each. **Wait for the tester's answer** and record only what they said: a result the tester did not give is never filled in for them. A story nobody answered for stays unexecuted, which makes the sign-off NOT ASSESSED — it is not BLOCKED unless they say so. ``` question: "Manual QA — [Story Title]\n[brief description of what to test]" options: - "PASS — all acceptance criteria verified" - "PASS WITH NOTES — minor issues found (describe after)" - "FAIL — criteria not met (describe after)" - "BLOCKED — cannot test yet (reason)" ``` After each FAIL result: use `AskUserQuestion` to collect the failure description, then spawn `qa-tester` via `Agent` to write a formal bug report in `production/qa/bugs/`, in `/bug-report`'s format — its Severity and Priority labels exactly as that template lists them (`S2-High`, `P1-Fix this sprint`, …), because `/bug-triage` parses them. **After each PASS or PASS WITH NOTES on a Visual/Feel or UI story, record its evidence in `production/qa/evidence/`.** The retained screenshot is the evidence: ask the tester to save one of each screen or effect the story touched under `production/qa/evidence/[story-slug]/`, and confirm it is on disk — `/story-done` checks for a retained image, not just the write-up. A UI story is closed by those screenshots; a Visual/Feel story also needs the lead sign-off. Then ask "May I write the evidence for [story titles] to `production/qa/evidence/`?", once per story group, and write the evidence artifact `production/qa/evidence/[story-slug]-evidence.md` from `.claude/docs/templates/test-evidence.md`, carrying the sign-off table intact and referencing the screenshots. If no screenshot was saved, name that story in the Phase 5 summary as missing its retained evidence rather than writing a doc that implies one. > **This is not optional bookkeeping — it is the artifact the next skill gates > on.** `/story-done` globs `production/qa/evidence/` for Visual/Feel and UI > stories and reads the sign-off table; `/story-readiness`, > `/test-evidence-review` and `gate-release` read the same directory. Write > anywhere else and a story can pass a full manual QA cycle here, then be told by > `/story-done` that no visual evidence exists. Visual/Feel and UI gates are > **BLOCKING by default**, so that is a deadlock — QA passed, story cannot > close. It is a merely confusing flag only where `testing.strict.ui` or > `.visual` has been explicitly set to `false`. > > Leave the sign-off rows **unchecked** unless the sign-off actually happened in > this session. An evidence file with pre-ticked approvals is worse than none: it > converts a missing signature into a recorded one. Bug report naming: `BUG-[NNNN].md`, numbered the way `/bug-report` numbers them — the highest ID in `production/qa/bugs/` plus one, zero-padded to four digits — so `/bug-report verify` and `close` can find it by its ID. After collecting all results, summarize: - Stories PASS: [count] - Stories PASS WITH NOTES: [count] - Stories FAIL: [count] — bugs filed: [IDs] - Stories BLOCKED: [count] ### Phase 6: QA Sign-Off Report Spawn `qa-lead` via `Agent` to draft the sign-off report using all results from Phases 2–5. **Here the return contract is the report text in its reply, not a file:** the orchestrator writes the sign-off after the ask at the end of this phase, so the qa-lead writes nothing. **Brief the qa-lead with the evidence and the verdict rules below — never a verdict.** Pass the strategy table, the smoke check verdict and its source, each story's result, the bugs filed in Phase 5 and the **Open bugs in scope** from Phase 1, and the verdict rules below, verbatim. Do not name a verdict, pre-fill the `### Verdict:` line, or say which fact decides it: the qa-lead applies the rules to the evidence. Then check its verdict against the same rules before asking to write; if they disagree, show both and ask — never silently substitute your own. The sign-off report format: ```markdown