| name | integration-test-review |
| description | [Code Quality] Use when you need to review integration tests for assertion quality, bug protection, repeatability, and test-spec traceability — AND verify the review target (changed production code) has test coverage (integration-first) with spec↔test↔code alignment. |
Codex compatibility note:
- Invoke repository skills with
$skill-name in Codex; this mirrored copy rewrites legacy Claude /skill-name references.
- Task tracker mandate: BEFORE executing any workflow or skill step, create/update task tracking for all steps and keep it synchronized as progress changes.
- User-question prompts mean to ask the user directly in Codex.
- Ignore Claude-specific mode-switch instructions when they appear.
- Strict execution contract: when a user explicitly invokes a skill, execute that skill protocol as written.
- Subagent authorization: when a skill is user-invoked or AI-detected and its protocol requires subagents, that skill activation authorizes use of the required
spawn_agent subagent(s) for that task.
- Do not skip, reorder, or merge protocol steps unless the user explicitly approves the deviation first.
- For workflow skills, execute each listed child-skill step explicitly and report step-by-step evidence.
- If a required step/tool cannot run in this environment, stop and ask the user before adapting.
Codex Project-Reference Loading (No Hooks)
Codex uses static project-reference loading instead of runtime-injected project docs.
When coding, planning, debugging, testing, or reviewing, open project docs explicitly using this routing.
Always read:
docs/project-config.json (project-specific paths, commands, modules, and workflow/test settings)
docs/project-reference/docs-index-reference.md (routes to the full docs/project-reference/* catalog)
docs/project-reference/lessons.md (always-on guardrails and anti-patterns)
Missing/stale context route: If docs/project-config.json, the docs index, lessons.md, CLAUDE.md, AGENTS.md, or any task-required reference doc is missing or stale, auto-run $project-init or the narrow setup route ($project-config, $docs-init, $scan-all, $scan --target=<key>, $claude-md-init) before ordinary project-specific work. If Codex mirrors or AGENTS.md are missing/stale, ask the user to run $sync-codex; do not auto-run it.
Situation-based docs:
- Project structure/architecture/tech-stack/deployment/setup (any layer — backend, frontend, or infra):
project-structure-reference.md
- Backend/CQRS/API/domain/entity changes:
backend-patterns-reference.md, domain-entities-reference.md
- Frontend/UI/styling/design-system:
frontend-patterns-reference.md, scss-styling-guide.md, design-system/README.md
- Spec authoring,
docs/specs/ pathing, or TC format: feature-spec-reference.md, spec-system-reference.md, spec-principles.md
- Behavior/public-contract changes or spec-test-code sync:
workflow-spec-test-code-cycle-reference.md plus the spec docs above
- Derived spec indexes/ERDs/reimplementation guides:
spec-system-reference.md and source Feature Specs under docs/specs/
- Integration test implementation/review:
integration-test-reference.md
- E2E test implementation/review:
e2e-test-reference.md
- Code review/audit work:
code-review-rules.md plus domain docs above based on changed files
Do not read all docs blindly. Start from docs-index-reference.md, then open only relevant files for the task.
[BLOCKING] Execute skill steps in declared order. NEVER skip, reorder, or merge steps without explicit user approval.
[BLOCKING] Before each step or sub-skill call, update task tracking: set in_progress when step starts, set completed when step ends.
[BLOCKING] Every completed/skipped step MUST include brief evidence or explicit skip reason.
[BLOCKING] If Task tools are unavailable, create and maintain an equivalent step-by-step plan tracker with the same status transitions.
Quick Summary
Goal: Ensure the review target (changed production code) is covered by tests that protect real business behavior with correct data assertions, infinite repeatability, and spec alignment — verifying every behavior change has test coverage (integration-first, unit fallback) so that specs ↔ tests ↔ code stay aligned (spec-driven development).
Summary:
- Purpose: Review target is the CHANGE (collect BOTH changed production code AND changed test files), never just the test files — Gates 1-6 judge test quality, Gate 7 maps every behavior-changing production file to a covering test (integration-first; unit only with recorded justification) + spec TC. Uncovered changed behavior = HIGH finding minimum.
- The 7 Gates (main review steps): G1 Assertion Value — mutation-score, record the Mutation Probe Ledger (no ledger = FAIL); G2 Data State — assert specific DB fields with async polling; G3 Repeatability — unique IDs, additive-only, 2 consecutive green runs; G4 Domain Logic — read handler, assert ONLY fields it writes; G5 Spec Traceability —
TestSpec annotation → TC in spec docs (1 TC → many tests is correct); G6 Three-Way Sync — feature-docs > test-spec docs > code > test, escalate conflicts; G7 Change Coverage — every behavior-changing file → covering test + non-stale §8 TC.
- The phase pipeline (run ALL, task tracking each): P0 Scope-detect → P1 Collect (split prod vs test files) → P2 Gate Review (Gates 1-6 per file, Gate 7 across set) → P3 Spec Cross-Check (both directions) → P4 Initial Report → P5 Fix ALL Crit/High + WRITE missing tests → P6 Validated-fix + full fresh re-review until 0 Crit/0 High → P7 Build & run ALL tests → P8 Failure Investigation → P9 Why-Review self-validation.
- Read handler/service source (and feature docs) BEFORE judging any assertion. FAIL smoke-only, existence-only (not-null), dead (always-true), copy-paste, and DI-resolution-only tests — why: assertion quality is unknowable without knowing what the handler actually writes.
- Don't just report gaps — fix them. Gate 6: NEVER fix a test to match broken code, NEVER self-resolve a three-way conflict (escalate by asking the user directly). Phase 5 WRITES the missing test (runs
$spec [mode=tests] for SPEC-GAPs); a full fresh re-review runs after every validated fix cycle until a clean pass returns 0 CRITICAL/0 HIGH.
Scope: The FULL change set — changed production code AND changed test files — from uncommitted changes (default), user-specified files, or a user-specified diff (branch/PR). The review target is never "just the test files".
Workflow: Phase 0 Detect → Collect → Coverage Map (Gate 7) → 7-Gate Review → Spec Cross-Check → Report → validate findings → fix validated issues (including writing missing tests) → full re-review after fixes → Build & verify → If fail: investigate + fix plan
Non-negotiable rules:
-
MUST collect BOTH changed production code AND changed test files — coverage of the change is part of the review, not an optional extra
-
MUST verify every behavior-changing production change maps to a covering test — integration test FIRST; unit test ONLY with recorded justification (Gate 7)
-
MUST treat an uncovered changed behavior as a HIGH finding minimum — fix by writing the missing test in Phase 5, not just reporting
-
MUST verify spec↔test↔code alignment for changed code, not only for existing tests — a changed behavior with no TC in spec docs is a spec gap finding
-
MUST read handler/service source BEFORE judging any test assertions
-
MUST flag smoke-only tests (no-exception-only checks) as FAIL
-
MUST flag DI-resolution-only tests (resolve + not-null) as FAIL — NOT integration tests
-
MUST verify tests use unique IDs per run (infinitely repeatable)
-
MUST use async polling/retry for ALL DB assertions — async delays are norm
-
MUST flag repository-created or repository-mutated test data that bypasses real use cases and can leave invalid state
-
MUST require 2 consecutive successful suite/project runs before declaring integration tests verified/idempotent
-
NEVER accept assertions that always pass regardless of handler correctness
-
NO smoke/fake/useless tests — every test MUST execute actual operations and verify data state
-
docs/project-reference/integration-test-reference.md — Integration test patterns, fixture setup, seeder conventions, lessons learned (MUST READ before reviewing)
First Principle — Easy to Change
The success metric of every coding decision is future change cost.
DRY, SRP, abstraction, design patterns, naming, layering, tests — every
technique exists to serve one goal: making the next change cheaper.
When evaluating code, refactor, test, or abstraction, ask:
does this make next change cheaper or more expensive?
- Reject "best practices" raising change cost (premature abstraction,
speculative generality, leaky indirection, ceremony without payoff).
- Name real enemies in findings: coupling, hidden state, duplicated
knowledge, unclear intent, irreversible decisions exposed too early.
- Simpler design easy to change beats sophisticated design that isn't.
Apply this lens before invoking any specific rule, pattern, or checklist
below — if downstream rule would raise change cost, this principle wins.
Phase 0: Scope Detection
Classify BEFORE any gate review. Route wrong → waste all effort.
| Signal | Classification | Action |
|---|
| No user-specified files | Uncommitted changes | Run git diff --name-only (staged + unstaged) to collect scope — BOTH production code AND test files |
| User specifies files/diff (branch, PR, etc.) | Explicit scope | Use provided list/diff directly — still split into production vs test files |
| 10+ test files | Large scope | Parallel sub-agents grouped by module |
| 1-9 test files | Normal scope | Single review pass |
| 0 test files BUT production code changed | Coverage-gap review | Gate 7 IS the review — map every changed behavior to existing tests; uncovered behavior = finding. Do NOT exit |
| 0 changes at all | Empty target | Ask user for explicit scope by asking the user directly |
The review target is the CHANGE, not the test files. Changed test files are reviewed for quality (Gates 1-6); changed production files are checked for coverage and spec alignment (Gate 7). Both halves are mandatory.
Search for test reference docs — NEVER hardcode paths. Grep for integration-test-reference, test-patterns, integration-test-guide near changed test files to discover project-specific conventions before starting gate review.
The 7 Quality Gates
Gates 1-6 apply per changed/target TEST file. Gate 7 applies to the CHANGE SET — every behavior-changing production file must map to a covering test and a spec TC.
Gate 1: Assertion Value — "Would this catch the bug?" (MUTATION-SCORE gate)
Think: If a single line of the handler's core logic were changed (a > flipped to >=, a field assignment removed, a boolean negated), would at least one assertion FAIL? If NONE → FAIL. This is the mutation-testing question, made automatic.
#1 AI failure: hallucination assertions — look real, verify nothing.
Operationalized — run the project's mutation tool (PRIMARY). "Would this catch the bug?" is exactly the mutation-score question. Mechanize it instead of eyeballing it:
- Discover the configured mutation tool from
docs/project-config.json, dependency manifests, and CI config. Common per stack: Stryker (JS/TS, .NET — StrykerNet), PITest (Java/JVM), mutmut or cosmic-ray (Python). Cite the local config or command if one exists.
- Run it scoped to the CHANGED handler/service (mutate only the production files in the review target — never the whole repo) against the covering tests from Gate 7.
- Read the surviving-mutant report. Each surviving mutant = a missing invariant = an assertion gap → a HIGH finding minimum (CRITICAL when the mutated line touches authorization, money, or data integrity).
- Fix in Phase 5 by WRITING the killing test — the assertion or property that fails on that mutant. Re-run until the changed code's mutants are killed (or each survivor has a recorded justification, e.g. equivalent mutant). Raising the line-coverage number is NOT a fix — coverage that executes a line without asserting its effect leaves the mutant alive.
Manual fallback (when NO mutation tool is configured or addable). Apply the single-mutation thought experiment by hand: read the handler source, then for each core-logic line ask "if I deleted or inverted this, which assertion fails?" If the answer is NONE for any business-critical line → FAIL. Prefer recommending the stack-appropriate mutation tool as a harness add so the gate becomes automatic next time.
PASS: Every changed core-logic line is killed by ≥1 assertion — no surviving mutant on the changed code (mutation tool), or the manual single-mutation check finds a failing assertion for each (fallback) — AND the Mutation Probe Ledger (below) is recorded in the report with a KILLED/SURVIVOR verdict per changed core-logic line. No ledger → no PASS, regardless of how clean the eyeball check felt.
FAIL:
- A surviving mutant on a changed core-logic line with no killing test (or no recorded equivalent-mutant justification)
- No-exception as ONLY assertion
- Not-null without content check
- Assertions on fields handler doesn't modify
- Dead assertions:
x >= 0 where x always >= 0, count >= 0, string not-empty on required fields
Verify: Run the mutation tool on the changed handler → list surviving mutants → each survivor is a missing assertion to write. When no tool is available: read handler source → list fields/branches it changes → check at least one assertion would fail if each were mutated.
Recorded artifact — Mutation Probe Ledger (REQUIRED, non-skippable, BOTH paths). "I checked it mentally" is not evidence. Gate 1 cannot be marked PASS without this ledger written into the review report — it is the proof the probe ran, identical in obligation whether a tool ran or the manual fallback did:
Changed core-logic line (file:line, abstract) | Mutation applied (>→>=, assignment dropped, boolean negated, branch removed) | Killing assertion / test (TC-… + file:line) | Verdict |
|---|
| {the line} | {the mutant} | {the assertion that fails on it} | KILLED |
| {the line} | {the mutant} | — none — | SURVIVOR → finding (or recorded equivalent-mutant justification) |
Rules: (1) every changed core-logic line gets a row — no sampling, no "representative subset". (2) Tool path: rows come from the surviving-mutant report; manual fallback: rows come from the line-by-line thought experiment — same table, same columns. (3) A SURVIVOR row with no killing assertion is a HIGH finding minimum (CRITICAL on auth/money/data-integrity lines) UNLESS it carries a written equivalent-mutant justification. (4) An empty or absent ledger = Gate 1 FAIL (not "skipped") — the gate is unproven, so it cannot pass.
Gate 2: Data State — "Does it check the database?"
Think: Does this test prove the database changed, or just that no exception occurred?
PASS: After command, test queries DB and asserts specific entity field values.
FAIL:
- Only checks return value, never verifies DB state
- Checks existence (not-null) without field values
- Missing async polling on side-effect assertions
Exception: Smoke-only ONLY when side effect truly unobservable. MUST include explicit justification comment.
ALWAYS use async polling/retry for data assertions. Event handlers, bus consumers, background jobs run async — data may not be immediately available.
Gate 3: Repeatability — "Can I run this 100 times?"
Think: If this test runs N times in a shared database, does it get noisier each run? Would run #2 fail?
FAIL: Hardcoded IDs, hardcoded business keys without unique suffix, teardown/cleanup, ordering dependency, seeders without existence check, or direct repository setup that creates state users could not create through real use cases.
FAIL (not parallel-safe — see SYNC:test-data-isolation): Assertions hung off a shared mutable entity another test can change, OR off a parent a bulk re-sync/recompute/rebuild/cascade consumer can wipe — even when THIS test never mutates it. Each test MUST own fresh per-test data; only immutable lookup data may be shared. Verify by grepping every other test on that shared data AND every cross-cutting consumer over it.
Verify: Repeatability is only proven when the relevant suite/project passes 2 consecutive runs without resetting data. One green run is not enough.
Gate 4: Domain Logic — "Does test match handler?"
Think: Did I read the handler source? Do I know which exact fields it writes? Do assertions check those fields — and ONLY those fields?
PASS: Assertions match what handler ACTUALLY does (verified by reading source). Covers primary business rule. Validation paths tested.
FAIL: Assertions on untouched fields (copy-paste), missing primary side-effect assertion, event handler tests that never trigger the event.
Verify: Grep handler class → read it → list what it does → compare with assertions.
Also check:
- Authorization: test verifies both authorized AND unauthorized access paths?
- Coverage: happy path + validation failure + DB state check (3 tests minimum)
Gate 5: Spec Traceability — "Is this tracked?"
Think: Can I trace TC-XXX-NNN from test annotation → spec docs → feature docs in one unbroken chain?
PASS: Business test has a TestSpec annotation linking to a TC ID that exists in spec docs. Technical-only test has a TechnicalSpec annotation and does not claim §8 business coverage. The test method name need NOT match the TC, and many test methods may legitimately carry the same TC (one business TC → many tests across components/services — the join key is the test-spec annotation, not the method name; see tc-format.md → TC ↔ Test Code Cardinality).
FAIL (WARN, not BLOCK): Missing annotation, orphaned TC ID (business TestSpec points to a TC absent from spec docs), technical-only test still carrying a business TestSpec, or spec says "Planned" but test exists. NOT a finding: several tests sharing one TC, a method name that differs from the TC, or a technical-only test carrying TechnicalSpec instead of TestSpec.
Gate 6: Three-Way Sync — "Do test, code, and docs agree?"
Think: Have I read ALL 3 sources? Where exactly do they disagree? Does evidence support a verdict, or must I escalate?
Hardest gate. Identify discrepancy, classify using source-of-truth hierarchy — NEVER silently pick winner. Always state the resolved source with file:line evidence — why: a winner picked without evidence hides bugs.
Source of Truth Hierarchy (highest → lowest)
| Priority | Source | Why |
|---|
| 1 (Highest) | Feature docs (docs/specs/…/Section 8 TCs) | Business intent — defines WHAT must happen |
| 2 | Test-spec docs (docs/specs/) | TC scenarios derived from feature docs — defines HOW to verify |
| 3 | Implementation code (handler/entity/service) | What WAS built — may reflect intentional evolution not yet in docs |
| 4 (Lowest) | Integration test code | What IS being tested — most likely to be wrong or stale |
Rule: Docs win over code. Code wins over tests. Feature docs win over test-spec docs.
Conflict Classification
| Pattern | Feature Doc | Impl Code | Test Code | Verdict | Action |
|---|
| All agree | ✓ | ✓ | ✓ | PASS | None |
| Stale docs | — | ✓ | ✓ | Docs lag code | Flag docs for $docs-update; test is correct |
| Wrong test | ✓ | ✓ | ✗ | Test wrong | Fix test assertions to match code + docs |
| Code bug | ✓ | ✗ | ✓ | Code has bug | Report as BUG — do NOT fix test to match code |
| Test + code diverge from docs | ✓ | ✗ | ✗ | Code bug + wrong test | Fix test to match docs; report code bug |
| Three-way conflict | ✗ | ✗ | ✗ | ESCALATE | Cannot self-resolve — ask the user directly |
CRITICAL rules:
- NEVER fix a test to match broken code — that hides bugs
- NEVER assume docs are wrong without evidence they were intentionally superseded
- NEVER self-resolve a three-way conflict — always escalate by asking the user directly
- "Stale docs" verdict requires BOTH code AND test to agree — one source never enough
- When escalating, include: TC ID, what each source says, evidence found
Verify Each Source
- Feature doc: Read Section 8 — scenario title, preconditions, steps, expected results
- Test-spec doc: Find same TC — Planned/Implemented status and described scenario
- Implementation code: Read handler/entity/service — fields written, events fired, validation rules
- Test code: Read test method — arrange, execute, assert
Compare each pair with file:line evidence for each source.
PASS: All three agree. WARN: Minor wording, same semantic. FAIL: Semantic disagreement on field/rule/outcome. ESCALATE: All three differ and evidence cannot resolve.
Gate 7: Change Coverage — "Is every changed behavior tested AND specced?"
Think: For each behavior-changing production file in the review target, which test would FAIL if this change were broken? If NONE → coverage gap. Which spec TC describes this behavior? If NONE → spec gap.
This gate makes the skill verify the REVIEW TARGET has coverage — not merely review tests that happen to exist.
Protocol:
- Collect changed production files from the review target (Phase 0 scope): commands, queries, handlers, entities, services, event handlers, consumers, controllers, frontend services/stores with business logic.
- Filter to behavior-changing files. Exclude: migrations (one-time execution paths), generated code, pure renames/formatting, config-only, DI registration-only changes. Record each exclusion with reason.
- Find covering tests per changed behavior — use graph (
query tests_for <fn>, trace <file> --direction both) plus grep for the handler/class name under test directories. A test COVERS a change only if it exercises the changed path and asserts the changed outcome — read the test; name match alone is NOT coverage.
- Apply test-type priority: integration test FIRST (subcutaneous CQRS through real DI, data-state assertions). Unit test is an acceptable fallback ONLY when integration coverage is infeasible (pure function/calculation logic, no observable data state, no DI path) — record the justification per fallback.
- Check spec alignment for the change — existence AND correctness. Each changed behavior must map to a TC in spec docs (feature doc Section 8 / test-spec docs). Finding a TC is NOT enough: READ the mapped TC and confirm it describes the CURRENT behavior. New behavior with no TC, or a TC that exists but still describes the OLD/superseded behavior → spec gap (spec-driven development violation). A behavior is only fully covered when a covering test exercises it AND a non-stale §8 TC documents it — so this correctness re-check applies to COVERED rows too, never just to GAP rows.
Coverage Mapping Table (MANDATORY output):
Rows are keyed by changed production behavior, not by TC. A behavior is COVERED when ≥1 covering test exercises it — list ALL covering tests in the column when several apply. One Spec TC may legitimately appear across multiple rows and be covered by many tests (one TC → many tests, 1:N). Do NOT expect or require one test per TC, and do NOT flag a TC reused across rows as a duplicate (see tc-format.md → TC ↔ Test Code Cardinality).
| Changed File / Behavior | Spec TC | Covering Test(s) | Test Type | Verdict |
|---|
| {file:line — behavior} | TC-X-NNN / MISSING | {test file:method}[, …one or more] / NONE | integration / unit (justified) / — | COVERED / COVERED-UNIT / GAP / SPEC-GAP |
Verdicts:
- COVERED — integration test exercises the changed path with data-state assertions AND the mapped §8 TC describes the CURRENT behavior. A covering test whose mapped TC is stale is NOT COVERED — record it as SPEC-GAP.
- COVERED-UNIT — unit test covers it, integration infeasible, justification recorded, and the mapped §8 TC is current (same stale-TC rule applies).
- GAP (FAIL) — no test would fail if the change broke. Severity: HIGH minimum; CRITICAL when the change touches authorization, money, or data integrity. Fix in Phase 5 by WRITING the missing test (integration-first) — reporting alone does not clear this gate
- SPEC-GAP (FAIL) — behavior has no TC, OR a covering test exists but its mapped TC still describes OLD/superseded behavior (stale TC ≠ covered). Both the missing-TC and the stale-but-covered case are SPEC-GAP. Fix via
$spec [mode=tests] UPDATE (and $spec when business rules changed)
FAIL:
- Changed handler/command/entity rule with zero covering test
- Unit test substituted where an integration test is feasible, with no justification
- Test exists but does not assert the changed outcome (stale coverage counted as coverage)
- New/changed behavior absent from spec docs, or TC describing superseded behavior
- A COVERED row marked COVERED without reading its mapped §8 TC — TC existence assumed instead of its CURRENT-behavior correctness verified (a stale-but-covered TC silently passes as covered)
Explicit user waiver (recorded verbatim in the report with the user's reason) is the ONLY alternative to closing a GAP.
Review Protocol (9 Phases)
Use task tracking for EACH phase before starting.
Phase 1 — Collect: Split the change set: production files (Gate 7 coverage targets) vs test files. Categorize test files: new (full review), modified (changed methods only), new projects (infra + samples). Categorize production files: behavior-changing vs excluded (with reason).
Phase 2 — Gate Review: Per test file, apply Gates 1-6. Apply Gate 7 once across the change set and produce the Coverage Mapping Table. Record per-file verdict table:
| Gate | Verdict | Evidence |
|---|
| 1. Assertion Value | PASS/FAIL | {file:line} |
| 2. Data State | PASS/FAIL | {file:line} |
| 3. Repeatability | PASS/FAIL | {file:line} |
| 4. Domain Logic | PASS/FAIL | {file:line} |
| 5. Traceability | PASS/WARN | {file:line} |
| 6. Three-Way Sync | PASS/WARN/FAIL/ESCALATE | {file:line} |
| 7. Change Coverage (per change set) | COVERED/COVERED-UNIT/GAP/SPEC-GAP | {coverage mapping table} |
Phase 3 — Spec Cross-Check + Three-Way Diff: Two directions — from tests AND from changed code.
For each TC ID in code:
- Verify TC entry exists in both
docs/specs/ (Section 8) and docs/specs/
- Read what TC describes in each doc
- Read what implementation code actually does
- Read what test asserts
- Classify conflict pattern (Gate 6 table) and record action
- Flag gaps both directions: TC in code but not in docs, or "Implemented" TC in docs but no test found
For each behavior-changing production file in the review target (reverse direction — spec-driven development check):
- Verify a TC exists describing the changed behavior AND read it to confirm it describes the CURRENT behavior — run this even when a covering test was already found and the row is otherwise COVERED. If the TC still describes the OLD behavior, downgrade the row from COVERED to SPEC-GAP and flag it stale (route to
$spec [mode=tests] UPDATE). Finding a covering test never excuses re-checking the TC's correctness.
- New behavior with no TC anywhere → SPEC-GAP finding (Gate 7); recommend
$spec [mode=tests] (and $spec when business rules changed)
Phase 4 — Initial Report: Write to plans/reports/integration-test-review-{date}-{slug}.md
Phase 5 — Fix All Issues (MANDATORY — fix ONLY findings already validated per the embedded double-round-trip-review validate-before-fix contract): Fix every CRITICAL and HIGH issue. MEDIUM: fix if straightforward, document as tech debt otherwise.
- Prioritize: CRITICAL → HIGH → MEDIUM
- Per fix: read handler source, understand domain logic, write/fix assertion
- Gate 7 GAP fixes: WRITE the missing test — integration test first (route through
$integration-test patterns); unit test only with recorded justification. SPEC-GAP fixes: run $spec [mode=tests] UPDATE to add/correct the TC before or alongside writing the test
- NEVER weaken assertions to make tests pass — fix root cause (timing, data, setup) instead
- Re-read changed files to verify fix correctness
- Record each fix with
file:line under ## Fixes Applied
Phase 6 — Validated Fix + Full Re-Review (MANDATORY when fixes are applied):
Do not spawn a fresh reviewer to re-review the same findings before validation/fix. After Phase 5 applies validated fixes, run a full fresh review over the current test scope. When that review uses sub-agents, spawn fresh integration-tester sub-agents (parallel by module for 10+ files; single agent otherwise) using canonical Agent template from SYNC:review-protocol-injection. Each sub-agent re-reads ALL target test files from scratch with ZERO memory of Phase 2/5. When constructing Agent call prompt:
- Copy Agent call shape from
SYNC:review-protocol-injection template verbatim
- Set
agent_type: "integration-tester"
- Embed full verbatim body of 9 SYNC blocks (all present inline in this skill file):
SYNC:evidence-based-reasoning, SYNC:bug-detection, SYNC:design-patterns-quality, SYNC:logic-and-intention-review, SYNC:test-spec-verification, SYNC:fix-layer-accountability, SYNC:rationalization-prevention, SYNC:graph-assisted-investigation, SYNC:understand-code-first
- Task field:
"Run a full fresh integration-test review pass over {file-list} after validated fixes were applied. Review against 7 quality gates: assertion value, data state, infinite repeatability, domain logic, test-spec traceability, three-way sync, change coverage. Read handler source AND feature docs before judging assertions. Flag smoke-only, existence-only, dead assertions, and repository-created invalid test data as FAIL. Gate 3 also flags tests that are not parallel-safe: assertions hung off a shared mutable entity another test can change, or off a parent a bulk cross-cutting consumer (re-sync/recompute/rebuild/cascade) can wipe even without this test mutating it — each test must own fresh per-test data; prove by grepping other tests on that shared data and every consumer over it. Gate 7: map every behavior-changing production file in {changed-production-file-list} to a covering test (integration-first; unit fallback requires justification) AND a spec TC — uncovered behavior is a HIGH finding minimum, missing/stale TC is a SPEC-GAP finding. Source-of-truth hierarchy: feature docs > test-spec docs > implementation code > test code. Classify every disagreement as: wrong test, code bug, stale docs, or escalate (three-way conflict)."
- Target Files: explicit file list (never pass inline contents)
- Reference Docs: include
docs/project-reference/integration-test-reference.md
- Report path:
plans/reports/integration-test-review-rerun{N}-{date}.md
After sub-agents return:
- Read each sub-agent's report
- Integrate findings as
## Re-Review {N} Findings — DO NOT filter or override
- If new CRITICAL/HIGH: validate the new finding set before any additional fixes
- Repeat only after another fix cycle: restart the full review again after validated fixes are applied; if the same blocker repeats across 3 full invocations with no progress, escalate by asking the user directly
- Exit criteria: A complete full review returns 0 CRITICAL and 0 HIGH issues
Phase 7 — Build & Run Tests (MANDATORY): Build and run ALL changed/reviewed test files.
- Build test project
- Run changed tests (filter by reviewed test classes)
- NEVER mark review complete until all tests pass — unverified reviews have zero value
- Record results under
## Test Execution Results
Phase 8 — Failure Investigation (if Phase 7 fails): Investigate systematically (classify → root-cause → fix plan), never just retry.
- Classify failure: Test bug (assertion/setup wrong) vs Service bug (handler broken) vs Environment (service not running, DB timeout)
- Root cause: Read failing output, trace handler source, identify exact mismatch
- Fix plan per failure: failing test (
file:line, TC-ID), error summary, root cause + confidence %, proposed fix
- Apply and rerun — loop until pass or environment blockers identified
- Environment blockers: Document as
BLOCKED — requires running system; do NOT mark as test failures
- Append under
## Failure Investigation
10+ files: Parallel sub-agents grouped by module. Each gets file list + 7 gates + handler paths + feature doc paths + the changed-production-file list for its module (Gate 7). Consolidate into single report — the orchestrator merges per-module coverage tables into ONE Coverage Mapping Table covering the whole change set.
Common Anti-Patterns
| Anti-Pattern | Why It's Bad |
|---|
| Smoke-only (no-exception alone) | Proves no crash, not correctness |
| Existence-only (not-null) | Proves data exists, not handler set it correctly |
Dead assertion (count >= 0, always true) | Tests nothing |
| Framework testing (assert auto-set fields) | Tests framework, not handler |
| Copy-paste assertions (wrong entity fields) | Assertions don't match handler |
Hardcoded ID (Id = "test-001") | Fails on second run |
Cleanup dependency (finally { Delete(); }) | Fragile, hides pollution |
| Order dependency (test B needs A first) | Parallel execution breaks |
| Shared mutable entity (assertions on data another test can change) | Not parallel-safe — another test corrupts the shared state; own fresh per-test data |
| Cross-cutting consumer blind spot (shared parent wiped by bulk re-sync/recompute/rebuild/cascade) | A consumer empties your data without this test touching the parent — sharing is unsafe even without direct mutation |
| Repository data hacks (direct create/update bypassing use cases) | Leaves impossible state and hides real workflow bugs |
| Missing await (unchecked async exception) | Exception swallowed silently |
| Event not triggered (query, never fire) | Tests seeder, not handler |
| Test fixed to match broken code | Hides the bug — docs still say it's wrong |
| Self-resolved three-way conflict | AI picked winner without evidence — silent lie |
| Stale docs assumed without two-source proof | Docs may be right; code may be the bug |
| Test-files-only scope (production changes ignored) | Reviews tests that exist, misses behavior with none |
| Name-match counted as coverage (test never reads changed path) | Stale coverage — test passes while change is broken |
| Unjustified unit-test substitution | Skips DI/data-state verification integration gives |
| Spec-less change (no TC for new/changed behavior) | Breaks spec-driven development — specs drift silently |
| Stale-TC counted as covered (covering test found, mapped TC never re-read) | TC documents OLD behavior — coverage path passes a spec gap silently; must downgrade to SPEC-GAP |
| Surviving mutant left unkilled (Gate 1 mutation tool not run, or survivor ignored) | A changed line whose mutation no assertion catches = a fakeable, over-fitted test that protects no invariant |
| 1:1 TC↔test demanded (one test per TC, method-name=TC, or many-tests-per-TC flagged as duplicate) | Forces splitting/technicalizing business TCs — breaks §8's business/user-story orientation (M1/M5). One TC → many tests is correct |
Workflow Recommendation
MANDATORY — NO EXCEPTIONS: If NOT already in a workflow, MUST use ask the user directly to ask user:
- Activate
workflow-write-integration-test workflow (Recommended) — scout → investigate → spec [mode=tests] → artifact-review --type=spec-tests → integration-test → integration-test-review → integration-test-verify → spec [mode=sync] → docs-update → workflow-end → watzup
- Execute
$integration-test-review directly — run standalone
Phase 9: Why-Review Self-Validation Gate (MANDATORY when findings exist)
Purpose: Adversarial validation of own findings BEFORE handoff. Catches over-flagged Highs, false positives, and severity inflation at the source rather than letting them propagate downstream.
Trigger: Any finding produced (Critical, High, Medium, OR Low). Skip ONLY when the report's verdict is unconditional PASS with literally zero findings.
Protocol:
- Read own finalized report from
plans/reports/{skill}-{date}-{slug}.md
- Invoke
$why-review skill with arg: validate findings in plans/reports/{skill}-{date}-{slug}.md — verify each finding has file:line proof, steel-man each rejected interpretation, and stress-test severity classifications
- Read the validation verdict path returned by why-review, expected as
plans/reports/why-review-validate-{date}.md
- If why-review demotes/removes any finding: UPDATE own finalized report with revised severities, remove false positives, and add a
## Why-Review Validation Notes section citing what changed and why
- If why-review confirms all findings: Append
## Why-Review Validation line to own report stating "All N findings re-validated against actual code; no severity changes."
Skip conditions (record explicit reason if skipping):
- Verdict is unconditional PASS with zero findings → log "Skipped — no findings to validate"
- Why-review skill itself is the active context (avoid recursion)
Why this exists: AI sub-agent reports inherit confirmation bias — the orchestrator absorbs severity claims as ground truth. The 2026-05-09 review incident produced 5 Highs; adversarial validation demoted 3 of them. Codify this as standard practice.
Next Steps
MANDATORY — NO EXCEPTIONS after completing, MUST use ask the user directly:
- "$integration-test-verify (Recommended)" — Run integration tests to verify all pass
- "$workflow-review-changes" — Review all changes before committing
- "Skip, continue manually" — user decides
Related Skills
| Skill | Relationship | When to Call |
|---|
$integration-test | Producer — generates tests this skill reviews | Always preceded by $integration-test |
$integration-test-verify | Successor — runs tests after review clears | Call after review passes all 7 gates |
$spec [mode=tests] | TC source — Gate 5 checks TCs exist in feature doc Section 8 | If Gate 5 fails (orphaned test) → run $spec [mode=tests] UPDATE |
$spec-index | Spec authority — Gate 6 compares test code vs spec bundle | If Gate 6 finds conflict: spec is authority |
$spec | Business doc — Gate 6 compares tests vs feature doc business rules | If Gate 6 finds conflict: check spec vs spec-index alignment first |
$docs-update | Orchestrator — includes spec [mode=sync] | Call when Gate 6 reveals doc staleness |
Standalone Chain
When called outside a workflow, follow this chain after running integration-test-review.
integration-test-review (you are here)
│
├─ SCOPE: the full change set — changed production code AND changed test files
│ Tests may NOT exist yet for changed code — that is a Gate 7 finding, not an exit condition
│
├─ Gate 1-5 findings → fix tests (re-run integration-test if test code needs regeneration)
│
├─ Gate 7 (Change Coverage) gap resolution:
│ │
│ ├─ GAP (changed behavior, no covering test):
│ │ → Write the missing test — integration-first via $integration-test
│ │ → Unit test fallback ONLY when integration infeasible — record justification
│ │ → User waiver (verbatim, with reason) is the only alternative
│ │
│ └─ SPEC-GAP (changed behavior, no/stale TC in spec docs):
│ → $spec [mode=tests] UPDATE to add or correct the TC
│ → $spec [update] when business rules changed
│ → Then link the new/updated TC to the covering test (Gate 5)
│
├─ Gate 6 (Three-Way Sync) conflict resolution:
│ │
│ ├─ Test code ≠ spec (feature doc says behavior A, test asserts behavior B):
│ │ → Determine: spec authoritative or test authoritative?
│ │ → If SPEC is correct: fix test → re-run $integration-test
│ │ → If TEST reflects correct new behavior (spec stale): $spec [update] → $spec [mode=tests] [UPDATE] → update test
│ │
│ ├─ Test code ≠ implementation (test asserts X, code does Y):
│ │ → If CODE is correct: fix test → $spec [mode=tests] UPDATE (update TC to match code's correct behavior)
│ │ → If TEST is correct (code bug): do NOT update test → fix code → $prove-fix → re-run tests
│ │
│ └─ Derived index ≠ Feature Spec (the bucket INDEX.md / ERD disagrees with the canonical §1-8):
│ → The Feature Spec is canonical; the index is regenerable, never authoritative
│ → Run $spec-index to re-derive the index from the specs
│ → Do NOT self-resolve — escalate to user if ambiguous
│
├─ [REQUIRED] → $integration-test-verify
│ After all fixes, run actual tests to confirm all gates pass.
│
├─ [REQUIRED] → $spec [mode=sync]
│ If TCs were updated (Gate 5/6 fix), reconcile §8 TCs ↔ executing test code.
│
└─ [RECOMMENDED] → $docs-update
If Gate 6 revealed doc staleness, $docs-update runs full chain to update all layers.
[IMPORTANT] Use task tracking to break ALL work into small tasks BEFORE starting.
A test that cannot fail is not a test — it is decoration. Every test MUST earn existence by proving it would FAIL if the protected business rule/invariant changed or the bug it guards were reintroduced.
Every finding requires file:line proof with confidence >80%.
Fresh Context Re-Review — Eliminate orchestrator confirmation bias after fixes by restarting the full review with isolated sub-agents where applicable.
Why: The main agent knows what it (or $feature-implement) just fixed and rationalizes findings accordingly. A fresh sub-agent has ZERO memory, re-reads from scratch, and catches what the main agent dismissed. Sub-agent bias is mitigated by (1) fresh context, (2) verbatim protocol injection, (3) main agent not filtering the report.
When: ONLY after a validated-finding fix cycle. A review round that finds zero issues ENDS the loop — do NOT spawn a confirmation sub-agent. A review round that finds issues triggers: validate findings → fix → full review restart from the first phase.
How:
- Start a NEW full review invocation/task breakdown; when that protocol calls for agents, spawn NEW
spawn_agent tool calls — use integration-tester agent_type (integration-test reviews ALWAYS spawn integration-tester, NOT code-reviewer)
- Inject ALL required review protocols VERBATIM into the prompt — see
SYNC:review-protocol-injection for the full list and template. Never reference protocols by file path; AI compliance drops behind file-read indirection (see SYNC:shared-protocol-duplication-policy)
- Sub-agent re-reads ALL target files from scratch via its own tool calls — never pass file contents inline in the prompt
- Sub-agent writes structured report to
plans/reports/{review-type}-round{N}-{date}.md
- Main agent reads the report, integrates findings into its own report, DOES NOT override or filter
Rules:
- SKIP fresh sub-agent when the prior full review found zero issues (no fixes = nothing new to verify)
- NEVER skip the full review restart after a fix cycle — every fix invalidates the prior verdict
- NEVER reuse a sub-agent across rounds — every fresh round spawns a NEW
spawn_agent call
- Continue until a complete full review pass has zero findings; if the same blocker repeats 3 times with no progress, escalate by asking the user directly
- Track iteration count and repeated blockers in conversation context (session-scoped, no persistent files)
Sub-Agent Type Override
MANDATORY: Integration-test reviews spawn the integration-tester sub-agent, NOT code-reviewer.
Keep agent_type: "integration-tester" from the canonical template below; NEVER revert to code-reviewer.
Rationale: integration-tester specializes in test-spec generation, TC traceability, CQRS test patterns, async-polling / eventual-consistency assertion correctness, and cross-service integration context — areas code-reviewer does not cover at depth.
Review Protocol Injection — Every fresh sub-agent review prompt MUST embed 11 protocol blocks VERBATIM. The template below has ALL 11 bodies already expanded inline. Copy the template wholesale into the Agent call's prompt field at runtime, replacing only the {placeholders} in Task / Round / Reference Docs / Target Files / Output sections with context-specific values. Do NOT touch the embedded protocol sections.
Why inline expansion: Placeholder markers would force file-read indirection at runtime. AI compliance drops significantly behind indirection (see SYNC:shared-protocol-duplication-policy). Therefore the template carries all 11 protocol bodies pre-embedded.
Subagent Type Selection
integration-tester — ALWAYS for integration-test reviews (test files, TC traceability, CQRS/async assertion correctness)
code-reviewer — for general code-quality reviews only (NOT integration tests)
Canonical Agent Call Template (Copy Verbatim)
spawn_agent({
description: "Fresh Round {N} review",
agent_type: "integration-tester",
prompt: `
## Task
{review-specific task — e.g., "Review all uncommitted changes for code quality" | "Review plan files under {plan-dir}" | "Review integration tests in {path}"}
## Round
Round {N}. You have ZERO memory of prior rounds. Re-read all target files from scratch via your own tool calls. Do NOT trust anything from the main agent beyond this prompt.
## Protocols (follow VERBATIM — these are non-negotiable)
### Spec ↔ Tests ↔ Code Triangulation
DO THIS FIRST — before any per-protocol check below. The review target is the WHOLE PACKAGE, not the diff alone: load the behavior's spec (§3 ACs / §4 BRs / §8 TCs), its tests, and the changed code TOGETHER, and reason about their mutual consistency BEFORE judging any one in isolation.
1. Locate all three faces: the Feature Spec section(s) governing the changed behavior, the tests that guard it, and the production code that implements it. A missing face is itself a finding (SPEC-GAP / TEST-GAP / DEAD-SPEC).
2. Triangulate pairwise — every disagreement is a finding; classify which face is wrong:
- code vs spec: behavior the code does that no §3/§4/§8 rule describes → CODE-EXTRA or SPEC-STALE; a [HARD] §4 rule or §5 invariant with no enforcing code path → CODE-WRONG.
- tests vs spec: a §8 TC with no test, or a test asserting behavior no TC/rule names → TEST-GAP or SPEC-SILENT.
- tests vs code: a changed code path with no covering test → TEST-GAP; a test that still passes against a deliberately broken invariant → WEAK-TEST (apply the mutation thinking in Bug Detection).
3. Hidden-rule capture: any invariant the code enforces but the spec never states (SPEC-SILENT) MUST be surfaced as a finding to add into §3/§4/§8 AND guarded with a test — the enrichment loop, never a silent pass.
4. Only after the three faces agree — or every disagreement is logged as a finding — proceed to the per-protocol checks below; when enrichment adds spec/test content, re-review the package against the enriched spec.
NEVER mark review PASS while any spec/test/code face disagrees without a logged finding. The diff is the entry point; the package is the unit of judgment.
### Evidence-Based Reasoning
Speculation is FORBIDDEN. Every claim needs proof.
1. Cite file:line, grep results, or framework docs for EVERY claim
2. Declare confidence: >80% act freely, 60-80% verify first, <60% DO NOT recommend
3. Cross-service validation required for architectural changes
4. "I don't have enough evidence" is valid and expected output
BLOCKED until: Evidence file path (file:line) provided; Grep search performed; 3+ similar patterns found; Confidence level stated.
Forbidden without proof: "obviously", "I think", "should be", "probably", "this is because".
If incomplete → output: "Insufficient evidence. Verified: [...]. Not verified: [...]."
### Bug Detection
MUST check categories 1-4 for EVERY review. Never skip.
1. Null Safety: Can params/returns be null? Are they guarded? Optional chaining gaps? .find() returns checked?
2. Boundary Conditions: Off-by-one (< vs <=)? Empty collections handled? Zero/negative values? Max limits?
3. Error Handling: Try-catch scope correct? Silent swallowed exceptions? Error types specific? Cleanup in finally?
4. Resource Management: Connections/streams closed? Subscriptions unsubscribed on destroy? Timers cleared? Memory bounded?
5. Concurrency (if async): Missing await? Race conditions on shared state? Stale closures? Retry storms?
6. Stack-Specific: Check the configured language/runtime pitfalls and framework-specific failure modes discovered from local code.
Classify: CRITICAL (crash/corrupt) → FAIL | HIGH (incorrect behavior) → FAIL | MEDIUM (edge case) → WARN | LOW (defensive) → INFO.
### Design Patterns Quality
Priority checks for every code change:
1. DRY via OOP: Same-suffix classes (*Entity, *Dto, *Service) MUST share base class. 3+ similar patterns → extract to shared abstraction.
2. Right Responsibility: Logic in LOWEST layer (Entity > Domain Service > Application Service > Controller). Never business logic in controllers.
3. SOLID: Single responsibility (one reason to change). Open-closed (extend, don't modify). Liskov (subtypes substitutable). Interface segregation (small interfaces). Dependency inversion (depend on abstractions).
4. After extraction/move/rename: Grep ENTIRE scope for dangling references. Zero tolerance.
5. YAGNI gate: NEVER recommend patterns unless 3+ occurrences exist. Don't extract for hypothetical future use.
Anti-patterns to flag: God Object, Copy-Paste inheritance, Circular Dependency, Leaky Abstraction.
### Logic & Intention Review
Verify WHAT code does matches WHY it was changed.
1. Change Intention Check: Every changed file MUST serve the stated purpose. Flag unrelated changes as scope creep.
2. Happy Path Trace: Walk through one complete success scenario through changed code.
3. Error Path Trace: Walk through one failure/edge case scenario through changed code.
4. Acceptance Mapping: If plan context available, map every acceptance criterion to a code change.
5. Tests Verify Intent: For test/spec changes, verify tests name the protected business rule or invariant and would fail if that intent breaks.
6. Migration Test Exclusion: Do not write tests for migration code. Schema/data migrations are one-time execution paths, not core application logic.
NEVER mark review PASS without completing both traces (happy + error path).
### Test Spec Verification
Map changed code to test specifications.
1. Identify the project's test/spec format from existing docs, test-case files, BDD feature files, or spec folders.
2. Every changed code path MUST map to a corresponding test case/spec (or flag as "needs test case").
3. New functions/endpoints/handlers → flag for test spec creation.
4. Migration files are excluded from test/spec creation; schema/data migrations are one-time execution paths, not core application logic.
5. If spec evidence fields exist, verify they point to actual code (file:line, not stale references).
6. Verify each meaningful test case names the business intent/invariant; flag behavior-only cases that only mirror implementation details.
7. Auth/data changes → verify corresponding authorization and data-state test cases exist.
8. If no specs exist for a changed path → log the gap and recommend the project's test-spec workflow.
NEVER skip test mapping. Untested code paths are the #1 source of production bugs.
### Behavioral Delta Matrix
MANDATORY for any bugfix review. Produce input-state × pre-fix × post-fix × delta table BEFORE writing verdict.
- Minimum 3 rows; include at least one row OUTSIDE the original bug report.
- Any "REGRESSION" delta → review returns FAIL until a preservation test is added.
- Narrative descriptions do NOT substitute for the matrix.
Example rows (external-record sync fix):
| Input | Pre-fix | Post-fix | Delta |
| --------------------- | ------- | ------------------------- | ---------- |
| Record exists (valid) | Reused | Always recreated → orphan | REGRESSION |
| Record missing (404) | Error | Recreated | Fixed |
### Fix-Layer Accountability
NEVER fix at the crash site. Trace the full flow, fix at the owning layer. The crash site is a SYMPTOM, not the cause.
MANDATORY before ANY fix:
1. Trace full data flow — Map the complete path from data origin to crash site across ALL layers (storage → backend → API → frontend → UI). Identify where bad state ENTERS, not where it CRASHES.
2. Identify the invariant owner — Which layer's contract guarantees this value is valid? Fix at the LOWEST layer that owns the invariant, not the highest layer that consumes it.
3. One fix, maximum protection — If fix requires touching 3+ files with defensive checks, you are at the wrong layer — go lower.
4. Verify no bypass paths — Confirm all data flows through the fix point. Check for direct construction skipping factories, clone/spread without re-validation, raw data not wrapped in domain models, mutations outside the model layer.
BLOCKED until: Full data flow traced (origin → crash); Invariant owner identified with file:line evidence; All access sites audited (grep count); Fix layer justified (lowest layer that protects most consumers).
Anti-patterns (REJECT): "Fix it where it crashes" (crash site ≠ cause site, trace upstream); "Add defensive checks at every consumer" (scattered defense = wrong layer); "Both fix is safer" (pick ONE authoritative layer).
### Rationalization Prevention
AI skips steps via these evasions. Recognize and reject:
- "Too simple for a plan" → Simple + wrong assumptions = wasted time. Plan anyway.
- "I'll test after" → RED before GREEN. Write/verify test first.
- "Already searched" → Show grep evidence with file:line. No proof = no search.
- "Just do it" → Still need task tracking. Skip depth, never skip tracking.
- "Just a small fix" → Small fix in wrong location cascades. Verify file:line first.
- "Code is self-explanatory" → Future readers need evidence trail. Document anyway.
- "Combine steps to save time" → Combined steps dilute focus. Each step has distinct purpose.
### Graph-Assisted Investigation
MANDATORY when .code-graph/graph.db exists.
HARD-GATE: MUST run at least ONE graph command on key files before concluding any investigation.
Pattern: Grep finds files → trace --direction both reveals full system flow → Grep verifies details.
- Investigation/Scout: trace --direction both on 2-3 entry files
- Fix/Debug: callers_of on buggy function + tests_for
- Feature/Enhancement: connections on files to be modified
- Code Review: tests_for on changed functions
- Blast Radius: trace --direction downstream
CLI: python .claude/scripts/code_graph {command} --json. Use --node-mode file first (10-30x less noise), then --node-mode function for detail.
### Understand Code First
HARD-GATE: Do NOT write, plan, or fix until you READ existing code.
1. Search 3+ similar patterns (grep/glob) — cite file:line evidence.
2. Read existing files in target area — understand structure, base classes, conventions.
3. Run python .claude/scripts/code_graph trace <file> --direction both --json when .code-graph/graph.db exists.
4. Map dependencies via connections or callers_of — know what depends on your target.
5. Write investigation to .ai/workspace/analysis/ for non-trivial tasks (3+ files).
6. Re-read analysis file before implementing — never work from memory alone.
7. NEVER invent new patterns when existing ones work — match exactly or document deviation.
BLOCKED until: Read target files; Grep 3+ patterns; Graph trace (if graph.db exists); Assumptions verified with evidence.
## Reference Docs (READ before reviewing)
- docs/project-reference/code-review-rules.md
- {skill-specific reference docs — e.g., integration-test-reference.md for integration-test-review; backend-patterns-reference.md for backend reviews; frontend-patterns-reference.md for frontend reviews}
## Target Files
{explicit file list OR "run git diff to see uncommitted changes" OR "read all files under {plan-dir}"}
## Output
Write a structured report to plans/reports/{review-type}-round{N}-{date}.md with sections:
- Status: PASS | FAIL
- Issue Count: {number}
- Critical Issues (with file:line evidence)
- High Priority Issues (with file:line evidence)
- Medium / Low Issues
- Cross-cutting findings
Return the report path and status to the main agent.
Every finding MUST have file:line evidence. Speculation is forbidden.
`
})
Rules
- DO copy the template wholesale — including all 11 embedded protocol sections
- DO replace only the
{placeholders} in Task / Round / Reference Docs / Target Files / Output sections with context-specific content
- DO choose
integration-tester agent_type — integration-test reviews ALWAYS use integration-tester, never code-reviewer
- DO NOT paraphrase, summarize, or skip any protocol section
- DO NOT pass file contents inline — the sub-agent reads via its own tool calls so it has a fresh context
- DO NOT reference protocols by file path or tag name — the bodies are already embedded above
- DO NOT introduce placeholder markers for the protocols — they must stay literally expanded
Critical Thinking Mindset — Apply critical thinking, sequential thinking. Every claim needs traced proof, confidence >80% to act.
Anti-hallucination: Never present guess as fact — cite sources for every claim, admit uncertainty freely, self-check output for errors, cross-reference independently, stay skeptical of own confidence — certainty without evidence root of all hallucination.
Evidence-Based Reasoning — Speculation is FORBIDDEN. Every claim needs proof.
- Cite
file:line, grep results, or framework docs for EVERY claim
- Declare confidence: >80% act freely, 60-80% verify first, <60% DO NOT recommend
- Cross-service validation required for architectural changes
- "I don't have enough evidence" is valid and expected output
BLOCKED until: - [ ] Evidence file path (file:line) - [ ] Grep search performed - [ ] 3+ similar patterns found - [ ] Confidence level stated
Forbidden without proof: "obviously", "I think", "should be", "probably", "this is because"
If incomplete → output: "Insufficient evidence. Verified: [...]. Not verified: [...]."
Validated-Finding Fix + Full Re-Review Loop — Re-review is triggered by a validated finding fix cycle, not by a round number. Review purpose: review → validate findings → fix validated findings → full re-review until a complete review pass finds no issues. A clean review ENDS the loop — no further rounds required.
aka Self-Review Convergence Loop. The name is historical — there is NO 2-round cap; "double-round-trip" only means a validated-finding fix cycle forces at least one fresh re-review. It runs until a clean pass, bounded by the 5-round ceiling below.
Round cap — 5 rounds MAX (a ceiling, NEVER a target). A clean pass ENDS the loop immediately at ANY round — round 1 included; the cap never obliges you to keep spinning. Hitting round 5 with validated findings still open → STOP and escalate by asking the user directly with the still-open findings listed; NEVER emit a silent "good enough" PASS on cap exhaustion, and NEVER let the cap substitute for the clean-review requirement. The 3-repeated-no-progress blocker rule stays an EARLIER exit — escalate at whichever trips first.
Universal scope (any new output/judgment): any newly produced output or judgment gets ≥1 self-review; any new judgment gets ≥1 $why-review --validate-findings pass; anything flagged to re-check is re-checked ≥1 time — before that output is treated as final. This loop is the default convergence contract for ANY work-producing skill, not review skills only.
Routing invariant (author-facing): a skill that validates findings MUST route them through $why-review --validate-findings (the terminal validator) — NEVER fork an inline finding-validation. Routing through why-review is what makes the finding-survival bar and this loop apply; the verify-review-validate-coverage sensor enforces this exact route mechanically.
Round 1: Main-session review. Read target files, build understanding, note issues. Output findings + verdict (PASS / FAIL).
Decision after Round 1:
- No issues found (PASS, zero findings) → review ENDS. Do NOT spawn a fresh sub-agent for confirmation.
- Issues found (FAIL, or any non-zero findings) → run the active review skill's findings-validation gate first; for review skills the default gate is
$why-review --validate-findings <report-path>. Fix only validated findings, then restart the full review protocol from the beginning with a fresh task breakdown.
Fresh full re-review after every fix cycle: Re-run the whole review protocol over the current full target. When sub-agents are part of that protocol, spawn NEW spawn_agent calls — never reuse prior agents. Reviewers re-read ALL files from scratch with ZERO memory of prior rounds. See SYNC:fresh-context-review for the spawn mechanism and SYNC:review-protocol-injection for the canonical Agent prompt template. Each fresh full review must catch:
- Cross-cutting concerns missed in the prior round
- Interaction bugs between changed files
- Convention drift (new code vs existing patterns)
- Missing pieces that should exist but don't
- Subtle edge cases the prior round rationalized away
- Regressions introduced by the fixes themselves
Loop termination: After each full re-review, repeat the same decision: clean → END; issues → validate findings → fix → restart from the first review phase. Continue until a complete review pass finds zero issues, capped at 5 rounds. Escalate by asking the user directly at whichever comes first: the same validated finding repeats for 3 full invocations with no progress · a fix requires product/owner input · round 5 completes with validated findings still open. NEVER loop past 5 rounds, and NEVER convert cap exhaustion into a PASS.
Rules:
- A clean Round 1 ENDS the review — no mandatory Round 2
- NEVER fix unvalidated findings; validate first using the caller's validation gate
- Every surviving finding must additionally clear the finding-survival bar defined in why-review's Findings Validation Routine (a deliberately higher bar than the generic act-gate — "keep this finding?" is a stricter question than "act on this evidence?"); a finding below the bar is demoted or dropped, not kept
- NEVER skip the full re-review after a fix cycle (every fix invalidates the prior verdict)
- NEVER reuse a sub-agent across rounds — every iteration that uses sub-agents spawns NEW Agent calls
- Main agent READS sub-agent reports but MUST NOT filter, reinterpret, or override findings
- The 5-round cap NEVER replaces the clean-review requirement — it bounds runaway looping, it does not authorize shipping an un-clean review; a clean pass ends the loop early at any round, and cap exhaustion escalates rather than passes
- Enforce the round cap of 5 alongside the 3 repeated-no-progress blocker rule; both are escalation triggers, neither is a completion criterion
- Track recursive invocation count and repeated blockers in conversation context (session-scoped)
- Final verdict must incorporate ALL rounds executed
Report must include ## Round N Findings (Fresh Sub-Agent) for every round N≥2 that was executed.
Infinitely Repeatable Tests — Tests MUST run N times without failure. Like manual QC — run the suite 100 times, each run just adds more data. Verification is only PASS after the relevant suite/project passes 2 consecutive runs without database reset.
- Unique data per run: Use the project's unique ID generator for ALL entity IDs created in tests. NEVER hardcode IDs.
- Additive only: Tests create data, never delete/reset. Prior test runs MUST NOT interfere with current run.
- No schema rollback dependency: Tests work with current schema only. Never rely on schema rollback or migration reversals.
- Idempotent seeders: Fixture-level seeders use create-if-missing pattern (check existence before insert). Test-level data uses unique IDs per execution.
- No cleanup required: No teardown, no database reset between runs. Each test is isolated by unique seed data, not by cleanup.
- Unique names/codes: When entities require unique names/codes, append a unique suffix using the project's ID generator.
- Migration code excluded: Do not write tests for migration code. Schema/data migrations are one-time execution paths, not core application logic.
Parallel-Safe Test Isolation — Tests MUST run in parallel and still pass; no test's data may be affected by any other test. repeatable-test-principle guards a test against its OWN prior runs; THIS guards it against OTHER concurrent tests, including indirect corruption through a shared parent + a cross-cutting consumer.
- Own fresh data per test: Each test creates its own entities with unique IDs, down to the root it asserts on. NEVER assert against a shared mutable entity another test can change; only immutable reference/lookup data may be shared — why: shared mutable state is the single point another test corrupts.
- Isolate at the highest mutated entity: Own a private instance of the highest-level entity (aggregate root/parent) any test mutates. Sharing is safe only for data no test ever writes — why: a writable shared parent is contended ground two tests fight over.
- Account for cross-cutting consumers: A bulk re-sync, recompute, projection rebuild, or cascade any test triggers over a shared parent can rewrite or wipe every entity beneath it — so sharing that parent is unsafe EVEN WHEN your test never mutates it directly — why: the corruption arrives through a consumer, not the path under test.
- Suspect contamination FIRST on contradiction: When a test fails intermittently, or its result contradicts the traced behavior of the path under test (the path is provably innocent yet state is wrong), rule out cross-test interference BEFORE blaming the code under test — why: the innocent path takes the blame for another test's writes.
- Prove isolation by search, not assumption: Grep every OTHER test touching the same shared data AND every consumer that fans out over it; cite
file:line evidence. Absence of a sharer is a finding to prove, not assume — why: isolation claimed without a search is unverified.
Integration Test Execution Discipline — How the integration-test family (write · review · verify) runs, diagnoses, and clears a suite. Binds $integration-test, $integration-test-review, and $integration-test-verify identically.
- Verify the WHOLE system passes — not a hand-picked subset.
$integration-test-verify must prove the full relevant suite is green (every test in the system the change can touch), not one cherry-picked test. "All pass" is only true with actual runner output (Passed/Failed/Skipped counts + names) and only after 2 consecutive green runs without a DB reset.
- Drive state through real use-case paths — NEVER hack seed data. Set up every precondition exactly as a real user would: real commands, queries, production consumers/messages, or valid idempotent seeders. NEVER create or mutate domain data by direct repository writes — that fabricates states a user could never reach and hides the real workflow bug. Hacking seed data to force a green run is forbidden.
- On ANY failure →
$debug-investigate the root cause BEFORE any fix. Do not guess, do not patch the symptom site. Trace the failure end-to-start and classify whose fault it is: test code (wrong assertion/setup), source/production code (real defect), or environment/infrastructure/data. Then route: test-code fault → $integration-test-review to fix the test at the root (never weaken assertions or add skips); source-code fault → fix the production defect at the owning layer and report it; environment fault → mark BLOCKED and point at the startup script. NEVER change a test to match broken code.
- 60-second runtime cap — a slow test is a RED FLAG, not a tuning knob. Local integration tests run fast. If any single test (or a stalled suite) exceeds ~60s, STOP and treat the slowness itself as a defect signal — deadlock, missing
await, infinite poll/retry, a real network/external call, or an unbounded query. $debug-investigate the cause; NEVER paper over it by raising the timeout or extending the wait.
- Loop until the whole suite is green. After fixing the validated root cause, restart the full 2-run verification from run 1. Done means the entire relevant suite passes repeatably — never green-once, never a subset.
Source/test drift check. For coding, fix, debug, investigation, test, or review work: when source behavior changes, inspect affected unit/integration/E2E tests and decide from evidence whether tests should change to match intended behavior or the source change is an unintended bug to fix. Do not write tests for migration code; schema/data migrations are one-time execution paths, not core application logic.
Test-Failure Fault Adjudication — When a test fails (or you are debugging or fixing a failure), the job is to determine who is at fault — the source code or the test code. Getting that verdict right matters more than turning the suite green. Binds every debug / fix / test skill identically.
- Root-cause first — never guess, never patch the symptom.
$debug-investigate and trace the failure end-to-start to its actual cause before touching either side. A green-again suite is NOT the goal; a correct verdict on what was actually wrong is.
- Triangulate against the spec AND the source. If a governing Feature Spec covers the behavior (e.g.
docs/specs/** — §3 ACs / §4 BRs / §5 invariants / §8 TCs), it is the tiebreaker for intended behavior — compare BOTH the production source and the failing test against it. With no spec, the documented intent / acceptance criteria / caller contract is the reference. Decide from this evidence whether the SOURCE is wrong or the TEST is wrong.
- Classify who is at fault, then fix the wrong side at its root:
- SOURCE-WRONG — production code violates the spec's intended behavior or a clear invariant → fix the source at the owning layer; keep or strengthen the test that caught it.
- TEST-WRONG — the test encodes a stale or incorrect assertion, setup, or expectation that contradicts intended behavior → fix the test at its root. NEVER weaken an assertion, add a skip, or relax a timeout to force green.
- NEVER change a test to match broken source, and NEVER change source to satisfy a broken test. (Migration code excluded — schema/data migrations are one-time execution paths, not core application logic.)
- Ask the user when intended behavior is unclear. If no spec covers the behavior, the spec is silent, or the spec is ambiguous about which side is correct, STOP and ask the user directly (or consult the canonical spec owner) before editing either side — never silently pick source or test just to make the suite pass.
Reconcile to intended behavior, never to whichever side currently passes — green can encode the very bug.
Spec ↔ Tests ↔ Code Triangulation — The unit of review is the WHOLE PACKAGE (spec + tests + code), not the diff alone. Load all three faces together and reason mutual-consistency FIRST, before any isolated per-file check.
- Locate all three faces for the changed behavior: the governing Feature Spec section(s) (§3 ACs / §4 BRs / §8 TCs), the tests that guard it, and the production code. A missing face is a finding (SPEC-GAP / TEST-GAP / DEAD-SPEC).
- Triangulate pairwise — classify which face is wrong on every disagreement:
- code vs spec → CODE-EXTRA / SPEC-STALE / CODE-WRONG (a [HARD] §4 rule or §5 invariant with no enforcing path is CODE-WRONG).
- tests vs spec → TEST-GAP / SPEC-SILENT.
- tests vs code → TEST-GAP / WEAK-TEST (a test that survives a deliberately broken invariant).
- Capture hidden rules — an invariant the code enforces but the spec never states (SPEC-SILENT) is surfaced as a finding, added into §3/§4/§8, and guarded with a test: the enrichment loop, never a silent pass.
- Re-review after enrichment — when triangulation adds spec content or a test, re-review the package against the enriched spec; converge only when a full pass surfaces no new disagreement.
NEVER mark PASS while any face disagrees without a logged finding. The diff is the entry point; the package is the unit of judgment.
Spec drift adjudication (code-wrong vs spec-stale). Whenever changed behavior diverges from a canonical Feature Spec (business rule, acceptance criterion, flow, state transition, or §8 TC under docs/specs/), you MUST NOT silently pick a side. Adjudicate per shared/sdd-artifact-contract.md → Drift Gates:
- Detect — compare the change against the spec's documented intent. No divergence → record
Spec in sync and move on.
- Classify the divergence:
- CODE-WRONG — the spec correctly states intended behavior and the change violates it → BLOCKING finding; fix the code/test against intended behavior (write/adjust a regression TC first).
- SPEC-STALE — the change is the new intended behavior and the spec now documents the old/wrong behavior → update the spec FIRST via
$spec [mode=update], then sync $spec [mode=tests] + $spec [mode=sync].
- AMBIGUOUS — intended behavior is unclear → ask the user directly (or the canonical spec owner) before editing either side.
- SPEC-SILENT — the code correctly enforces an invariant/behavior that NO canonical spec artifact (§3 AC, §4 BR, §5 invariant, §8 TC) states → not drift but an UNWRITTEN rule discovered by review. ENRICH the spec via the Invariant Harvest pass (
$spec [mode=sync] direction=harvest → spec/references/sync.md): prove it is always-true (≥2 enforcement points or a rejecting guard), express it as a universally-quantified property, then add the rule to §4 (or §3/§5) AND a §8 TC via $spec [update] + $spec [mode=tests] and add the guarding test. A discovered invariant left only in code (or only in tests) is INCOMPLETE — this is the highest-value capture (the rule nobody wrote down).