bughunt

name

bughunt

description

One iteration of hypothesis-driven bug discovery — audit the codebase against project invariants and risk patterns via parallel agents, adversarially verify each candidate, and file a groomed bug-issue (red-test sketch + verification evidence + proposed root-cause fix) for every finding that clears the gate. Does NOT implement or ship — specialized dev processes pick up the filed issues. Use when invoked as `/bughunt` (runs one iteration); use `/loop bughunt` to run it as an autonomous loop. Project-agnostic; reads risk dimensions from CLAUDE.md key-invariants and project memory.

/bughunt — one hypothesis-driven bug-discovery iteration

You run one iteration: scan the codebase for risk patterns, form specific bug hypotheses, adversarially verify each, and file a groomed bug-issue for every finding that clears the gate — then declare the outcome. You do not fix anything or open a PR: bughunt is a find-and-file producer; the specialized dev processes (the shovel-ready queue / autodev / a human) implement the filed issues. /loop bughunt runs this on an autonomous loop.

This skill is a thin specialization over the shared file-cycle skeleton. Read dev-skills:file-cycle for the shared iteration machinery (labeling by validation, autonomous multi-output filing, dup-check, declaring the outcome). Looping — cadence, idle escalation, re-arming — is owned by /loop. This file owns:

• Phase 1 — target acquisition (parallel-agent risk-pattern audit + adversarial verification).
• Phase 1.5 — the bughunt-specific quality gate (provability + reachability + impact).
• Phase 4′ body — the bug-issue template file-cycle files.

Specialization-specific invariants

(See file-cycle for the shared invariants — multi-output, autonomous filing, quality-gate-beats-throughput, label-don't-block. These are additional.)

• Verify, don't prove-by-fixing. bughunt does not write or run a test and does not implement a fix. Each candidate is instead adversarially verified by reading the real code (Phase 1), and filed with a concrete red-test sketch — the exact test a dev process would write to reproduce it — plus the verification evidence. If you can't sketch that test, the hypothesis is too vague; drop it.
• Skip unverified hypotheses silently. If adversarial verification can't confirm the bug is real and reachable, drop it and move on — a wrong or unsubstantiated bug erodes trust in the queue. (A verified bug you're confident in still gets filed even if the fix approach is uncertain — label it needs-design; see file-cycle.)
• Propose a root-cause fix, don't hand-wave. The filed issue names the underlying cause and the smallest fix at that site — not a symptom-suppressing guard. If the only fix you can think of papers over the symptom, the hypothesis is incomplete; keep verifying or drop it.
• Respect the project's invariants. Read CLAUDE.md (especially "Key invariants" / "How to approach changes") and project memory before scoping risk patterns. Project-stated invariants are usually the highest-yield audit dimensions because the project has already named them.

Phase 1 — Parallel-agent audit + adversarial verification

Dispatch multiple agents in parallel (single message, multiple Agent tool calls), each scanning a distinct risk dimension. Each returns ranked candidate hypotheses; you synthesize, then adversarially verify the survivors before filing.

Read the project's stated invariants first

Read CLAUDE.md for "Key invariants", "Invariants", "Correctness rules", or similar. Read project memory for feedback_* and project_* files mentioning past bugs (e.g., "phase 2 lessons", "incident postmortem"). Past bugs cluster — the same shape often recurs in adjacent code. Pass these excerpts into each agent's prompt.

Dispatch agents (4–6 dimensions)

The standard set (pick what fits the codebase):

1. Project-invariant violations — for each invariant in CLAUDE.md, search for code that fails to maintain it at any of its sites.
2. Concurrency — shared mutable state without locks; awaits between read-modify-write of the same row; ordering across async tasks; gather() losing partial results on one failure.
3. Resource lifecycles — connections/files/locks acquired without paired release; try without finally; tasks created without registration in the cleanup registry.
4. Type confusion at Any boundaries — places where mypy infers Any (dict accesses on JSON, raw fetches, untyped third-party returns) and downstream code assumes a specific shape.
5. Error swallowing — except Exception: blocks that don't re-raise; try/except around code where a failure should propagate; broad excepts in projects with a "fail hard" stance.
6. Boundary cases — empty list / single-element / max-int handling; pagination edges; range loops with computed upper bounds; slice indices.
7. Stale derived state — caches not invalidated on write; computed fields not recomputed on dependent change; "echo" patterns drifting from source.
8. Time and timezone — naive vs aware datetimes; DST; now() called inside a transaction returning a different value than elsewhere.
9. Query construction — SQL string interpolation; missing parameterization; case-folding traps.
10. Idempotency — retry paths that assume the operation didn't already succeed; webhook handlers without dedup.
11. Test-coverage gaps — boundary cases not tested; mocks that diverge from real-system behavior (a unit test that mocks the very function whose logic is wrong passes forever and hides the bug — check that the layer asserting a behavior actually exercises the code that implements it).
12. Recently-merged feature code — git log the last ~10–15 feat: commits and audit each newly-added code path for feature-logic errors: wrong field defaults, validation gaps, account-scoping omitted on a new endpoint, request/response-model drift, a partial / "stage N" migration leaving a mixed encode-decode state, a stale assumption a later commit silently falsified. Have the finder git show the actual merge commit, not just the file. On a mature, heavily-swept codebase this is usually the highest-yield dimension — the invariant/risk-pattern dimensions above are the most-audited code in the tree, so the freshest un-audited surface is the recent diff and the peripheral subsystems (connectors, CLI, MCP, sandbox internals) that get less bughunt attention than the core.
13. Sibling-path guard gaps — when a validation, guard, constraint, state-transition helper, or contract is enforced at one entry-point, enumerate the sibling entry-points that reach the same downstream and check whether each enforces it too. The guard's own existence proves the invariant matters; the bug is the path that forgot it. High-recurrence shapes: a request-bound applied to a query param but not the opaque-cursor / forged-token path that carries the same value into the same SQL; a "clean up then fail" helper called on most terminal paths but bypassed by one early-return; a cap / advisory-lock taken by the granular add path but not the bulk-replace path; account-scoping applied on attach but not on a sibling bind; a fix that budgets/guards an operation at one call site but leaves a sibling call site of the same operation on the old path. Method: grep for the guard/helper/constant, then read every caller and every sibling that should route through it but open-codes a partial version instead. This pairs naturally with dimension 12 — a guard added in a recent commit is the prime candidate to have missed a sibling.

Each agent's prompt should include

• The relevant CLAUDE.md invariants and feedback_* memory excerpts.
• A reachability filter: don't surface hypotheses that require pathological inputs no real call path produces.
• A provability filter: each hypothesis must come with a sketch of "what test would prove it" — if the agent can't sketch the test, drop the hypothesis.
• A length cap (under 400 words per response).
• A clear deliverable: ranked list with (reachability × severity × provability) ÷ fix-scope scores, file:line references, and the test sketch per finding.

Use subagent_type: general-purpose. Run all agents parallel via a single message with multiple Agent tool calls. If invoked with --model=<value> (see file-cycle flags), include model: "<value>" on each Agent call so every audit subagent runs on the chosen Claude generation; otherwise omit model: and let them inherit the session's model.

Synthesize

Collect ranked lists. Cross-agent corroboration amplifies score. Provability is a hard filter at this stage — anything below ~0.6 doesn't make the synthesis cut. A hypothesis with no path to a red-test sketch is wasted iteration.

If the invariant/risk-pattern dimensions (1–11) come up empty on a mature codebase, pivot to the recently-merged-feature surface (dimension 12) and the peripheral subsystems before declaring empty. An empty invariant sweep on a heavily-audited core is a true negative about the wrong surface, not about the codebase — the bug, if there is one, is most likely in code merged since the last sweep. Run a second audit round targeted there (have finders surface their single strongest lead even when sub-threshold, so the synthesis sees near-misses rather than a bare empty set) before you conclude there's nothing to file.

Adversarially verify the survivors (the confirm gate)

Because bughunt files instead of proving-by-red-test, adversarial verification is its rigor — the step that keeps plausible-but-wrong bugs off the tracker. Before filing, verify each surviving candidate: spawn independent skeptics — ideally diverse lenses, each prompted to refute —

• a correctness lens that reads the cited file:line and its callers/callees and asks "does this mechanism actually produce the symptom, step by step, or does some guard/type/caller already prevent it?"; and
• a reachability lens that asks "is the triggering input actually producible by a real call path today, or does it need a state the system never constructs?"

A candidate survives only if the lenses confirm it's real AND reachable. Under ultracode, author Phase 1 as a single Workflow (finder fan-out → dedup/rank → diverse-lens refutation pipeline); otherwise dispatch the finders as parallel Agent calls and run verification as a second round. Record each verdict — it becomes the issue's verification evidence.

If the audit — after the pivot — surfaces no candidate that survives verification with provability ≥ 0.6, declare LOOP-OUTCOME: empty (file-cycle Phase 7) and end the iteration.

Phase 1.5 — Bughunt quality gate

Apply file-cycle's quality-gate principle, per candidate — bughunt is multi-output: every finding that clears the gate is filed, not just the top one. A candidate clears only if all are true:

1. Provable — provability ≥ 0.6: a concrete red test is sketchable, and the candidate survived adversarial verification (re-check on the actual code, not just the finder's excerpt).
2. Reachable — the input that triggers the bug is producible by some real call path (user action, scheduled job, API request, crash-recovery). "Theoretically possible if every guard fails" doesn't qualify.
3. Material — the symptom matters. Data corruption, lost messages, security issues, user-visible incorrect behavior, silent failures masking real errors — all material. A typo in a log message no one reads is not.
4. Single-root-cause — the finding is one bug with one underlying cause, not a bundle of "while we're here" items. Adjacent findings are separate issues — file them too.

If a verified candidate is real but the fix approach is genuinely uncertain (crosses architectural lines, could legitimately live in N places, or is "by design" per a comment worth a human call), don't drop it and don't ask — file it needs-design with the fork stated (file-cycle Escalation).

If no candidate clears the gate: report what was found and why each was rejected, then declare LOOP-OUTCOME: gate_killed. Don't file a speculative bug to a theoretically-reachable but practically-impossible path.

Phase 4′ — The bug-issue (file-cycle files it)

For each candidate that clears Phase 1.5, author a bug-issue with the body below; file-cycle (Phase 4′) owns the labeling, dup-check, autonomous filing, and outcome declaration.

• Symptom — the observable failure (what someone notices), and when it fires.
• Root cause — the code-level cause at file:line, traced concretely.
• Reachability / trigger path — the real call path that reaches the buggy line, and any config/state it needs.
• Red-test sketch — the exact failing test a dev process would write: the input that triggers the bug, the assertion, and the precise failure it should show today (the predicted symptom, not an incidental error). Name the test's home (e.g. tests/unit/...).
• Verification evidence — the adversarial-verdict summary (what the correctness + reachability lenses found; why it's real, not theoretical).
• Proposed fix — the smallest root-cause fix and its site; note any sibling call sites that share the cause (dimension 13). Flag if it needs expand/contract or touches a public surface.

Label

• A verified bug with a concrete, converged proposed fix → the project's build-queue label (shovel-ready) + bug.
• A verified bug whose fix has an open design fork → needs-design + bug, fork stated in the issue.

(See file-cycle Phase 4′ for the labeling rule and the file / dup-check / record / report steps.)

Specialization-specific boundaries

(Additive to file-cycle's shared boundaries.)

• Don't file a bug you haven't verified. Adversarial confirmation is the bar; a sketchable-but-unconfirmed hypothesis is dropped, not filed.
• Don't implement or open a PR. bughunt files; the consumer fixes.
• Don't over-specify the red-test sketch. Smallest input, sharpest assertion — enough to reproduce, not a finished suite.

Specialization-specific escalation

file-cycle's "label, don't block" governs — bughunt runs unattended and does not stop to ask. The cases that once were questions become labels on the filed issue:

• Fix approach uncertain / crosses architectural lines / could live in N places → file needs-design, fork stated.
• Bug looks "by design" per a comment or memory entry → still file it needs-design, citing the comment and why it may nonetheless be wrong; let the human adjudicate on the tracker.

A verified bug filed with an honest open fork beats a stalled loop or a dropped finding.