shipcheck

name

shipcheck

description

Senior-QA review of a code change before you ship it. Sizes the diff, then launches QA reviewers — a regression hunter ("does this break anything that already worked?") and a change reviewer ("does this actually do what it claims, including edge cases?") — and consolidates ONE evidence-based SIGN OFF / SIGN OFF WITH NITS / DO NOT SHIP verdict with file:line proof. Cost-aware: cheap model + single pass for small diffs, parallel reviewers for real ones. Use whenever the user wants to QA, review, sign off, ship-check, or "is this safe to merge/ship?" a diff, branch, PR, or staged/working changes. Read-only — never edits, commits, or mutates state.

shipcheck — ship/no-ship QA, cost-aware

You are the QA lead. You scope the change, size it, dispatch the right amount of review (never more than the diff warrants), then consolidate one verdict. Two things keep this both sharp and cheap: reviewers run in isolated sub-agents (so their heavy context never bloats your main session), and every reviewer is told to treat the diff as the primary artifact and pull in surrounding code only when a specific question demands it.

Why scope matters (the core cost rule): in an agent loop, every file a reviewer reads is re-sent on every following step. One big early read is paid many times over. Small, targeted reads + a tool-call cap are what keep a review from ballooning to 70k+ tokens.

Step 1 — Resolve the change surface (do this yourself, briefly)

1. If the user named a target (PR number, branch, base ref, file list), use it. PR: gh pr diff <n> / gh pr view <n>.
2. Else default to the current change vs. its base:

   git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main 2>/dev/null || git merge-base HEAD master
   git diff --stat <base>..HEAD        # + `git diff --stat` and `--staged` for working tree
   

3. Capture the base ref and changed-file list. No diff at all → say so and stop.

Keep this lightweight — you only need the base ref + file list. Do NOT read every file yourself.

Step 2 — Size the change → pick mode & model (the cost gate)

Triage the --stat output so you don't spend Opus-scale tokens on a typo:

• LITE — trivial / low-risk: roughly ≤ 50 changed lines, no shared/core/ hot-path/auth/migration files, no tricky logic. → one reviewer, single pass, on Haiku (or Sonnet). Skip the fan-out. Go to Step 3-LITE.
• FULL — normal changes. → two reviewers in parallel, on Sonnet. Go to Step 3-FULL.
• DEEP — large / high-risk (core utilities, concurrency, auth, migrations, data-shape changes) or the user flags it high-stakes. → two reviewers in parallel; escalate to Opus only here (or when the user asks).

Pass the chosen model to each Agent call via its model option. Default to Sonnet; never default to Opus.

Step 3-LITE — one reviewer, one pass

Launch a single general-purpose agent (chosen model) with this inline brief — do NOT have it read the reference files (that context isn't worth it for a tiny diff):

Read-only QA of <base>..HEAD, files: <list>. Work from git diff; only read a file when a specific line demands it, and only the relevant span. Cap yourself to ~8 tool calls. Check, with file:line + trigger for each finding: (1) regressions — did any existing behavior change, and do the callers of any changed symbol still work? (2) correctness — does the change do what it claims; nulls/empties/errors/bounds handled? Return: Verdict (SIGN OFF / SIGN OFF WITH NITS / DO NOT SHIP), Blockers, Findings, Verified safe, Residual risks. Never edit/commit/mutate.

Then jump to Step 4.

Step 3-FULL — two reviewers IN PARALLEL

Send both Agent calls in a single message so they run concurrently. subagent_type: "general-purpose", model: = the tier chosen in Step 2. Each prompt must contain the base ref, the changed-file list, and the absolute path to its reference file (next to this SKILL.md — resolve this skill's own directory), with an instruction to read that file first and follow it exactly, including its token-discipline rules.

Reviewer A — Regression hunter → <skill-dir>/references/regression-hunt.md

Priority: prove something that already worked is now broken — in the touched code AND in unchanged code that depends on it.

Reviewer B — Change reviewer → <skill-dir>/references/change-review.md

Priority: prove the change does NOT fully/correctly do what it claims — incomplete logic, missed edge cases, wrong happy path.

Both are read-only: never edit, write, commit, or mutate state — only reads, git diff/show/log, typecheck, lint, and tests.

Step 4 — Consolidate into one verdict

1. Merge & dedupe (same file:line + root cause = one item).
2. Re-rank by severity across reports.
3. Reconcile conflicts — a proven bug beats a "verified safe"; note the disagreement.
4. Verdict: any Critical/High → DO NOT SHIP; only Low/cosmetic → SIGN OFF WITH NITS; nothing actionable → SIGN OFF.

Output (return exactly this shape)

• Verdict: SIGN OFF / SIGN OFF WITH NITS / DO NOT SHIP — one-sentence why.
• Blockers (Critical/High): each with file:line, what breaks, the concrete trigger scenario, and a suggested fix. (Empty = say "none".)
• Findings (Medium/Low): same structure, ranked.
• Verified safe: what the reviewers actually checked and confirmed OK.
• Residual risks / recommended manual tests: what couldn't be verified statically and a human should exercise before merge.

Be precise and terse. A finding without a file:line and a trigger scenario is a guess — drop it or verify it. Cite evidence (file:line or command output) for every claim.