eat-the-broccoli

name	eat-the-broccoli
description	Use when the user says '/eat-the-broccoli', 'eat the broccoli', 'full quality sweep', 'pre-release audit', 'run the deep tests', or asks to hunt for gaps / membrane misses / stubs / dead code / silent failures / the bugs that pass tests but are still broken. A tiered quality-and-pattern audit: deterministic tooling (lint, types, tests, deps, dead-code, silent-failure) PLUS a learned-pattern hunt for the hard, judgment-requiring failure classes that tools can't catch. Works in any repo or stack. Levels: quick / standard / deep. Scope: changed / module / repo.
version	2.6.0

Eat the Broccoli 🥦

The thorough quality sweep you run before a release, after a refactor, or when something just smells off. Two halves:

1. Deterministic tooling — lint, types, tests, dependency CVEs, dead code, silent failures. Catches the known categories.
2. The pattern hunt — a checklist of the hard failure classes that pass every test and still ship broken: races, stale caches, silent fallbacks, null-masking, deploy-staleness. Tools can't catch these; only a trained eye plus a "is this by design?" check can.

Portable by design. The deterministic half is tool-agnostic — wire in your stack's linter / typechecker / scanner. The pattern hunt needs no tooling at all; it's a structured way of looking.

---

Two dials: depth × scope

Depth	digs into	Scope	covers
quick	lint + fast tests + dep-audit	changed	the diff vs your base branch
standard	+ dead-code, silent-failure, contract guards, pattern hunt	module	one subsystem / path
deep	+ full pattern hunt, docs/contract accuracy, multi-pass	repo	everything

Pick any combination — quick × changed is a pre-commit reflex; deep × repo is a pre-release gate. State the level + scope you chose, and say what you skipped — silent truncation reads as "covered everything."

Forks & monorepos: when most of the tree is upstream/vendored, repo scope drowns in code you don't own. Default to changed or your owned packages — here the scope dial is load-bearing, not optional.

---

Phase 1 — Deterministic tooling

Run your stack's equivalent of each. Skip what you don't have; the sweep still works.

Dimension	Tools
Lint / style	ruff, eslint, golangci-lint, clippy
Types	pyright, tsc, mypy, cargo check
Tests	pytest, jest, go test, cargo test
Dependency CVEs	pip-audit, npm audit, cargo-audit
Dead code	vulture, ts-prune, cargo-machete
Silent failures	bare except: / empty catch {} / discarded Result
Contract guards	your CLI / API contract tests
Test freshness	mutation testing (mutmut, Stryker, cargo-mutants); or grep for assertion-free tests, tautological asserts, and hand-built fixtures whose schema has drifted from the real one

On that last row — coverage does not cover it. Coverage asks "was this line executed?"; test freshness asks "could this test have failed?" Those come apart badly in fast iterative work, because tests written against yesterday's contract keep passing against today's code and actively guard the drift: a green assertion that the API returns 422 is indistinguishable from a green assertion that it should. Fixtures are the sharpest edge — a hand-built schema that has fallen behind the real one makes fixture and code agree with each other and disagree with production, so the suite is green precisely where it is blind. If a defect survives a green suite, the suite is a suspect, not an alibi.

Tooling by language

Concrete per-stack tool choices (Python, Rust) live in references/stacks.md — consult it when wiring a stack, not while hunting. The pattern hunt below is language-agnostic and ports verbatim across any stack.

Reading silent failures: a truly-silent swallow (except: pass, empty catch {}, let _ = fallible()) is the dangerous one. A broad catch that logs or re-raises is often deliberate degradation — flag the new ones, and any with no log and no re-raise, not the absolute count.

---

Phase 2 — The pattern hunt 🥦 (the part that earns the name)

Tools find known categories. These are the structural failures that pass tests and still ship broken. Each is a place where broken and by-design look identical — so each row carries the disambiguator that tells them apart.

The one meta-question: "This looks intentional — is it actually?" When you find a smell, check the intent (a comment, a test, a doc, or ask). When it resolves to "yes, by design," record that verdict (an inline annotation or a .broccoli-accept line — see below) so you don't re-litigate it next sweep.

Worked example. You spot a function returning [] on a DB timeout (fallback-masks-primary). Check the intent: does it log or raise? It logs nothing → ❌ broken — callers can't tell "no rows" from "DB down." If instead it logged a warning and returned [] as a documented degraded mode → ✅ by design: record it with the reason, move on.

A. State & timing

Smell	✅ by design if	❌ broken if
Race / shared mutable state — >1 writer to one key/file	single-writer guaranteed, or the key carries the writer's durable id	two writers share a proxy key → last-write-wins clobber
Stale cache — value resolved once, reused	the value is immutable for that lifetime	it's ephemeral/locational but frozen at init (location, clock, "current X")
Idempotency miss — a retried / redelivered op	dedup-key / set-semantics make replay harmless	it increments / appends / re-sends on replay
Off-by-one window — a since/until, threshold, range	inclusive/exclusive matches intent	the event you need lands exactly on the excluded edge
Coupled-lifecycle orphan — two things that must live/die together	bound, or a reaper cleans orphans	orphans pile up silently (a handle without its resource; a record opened, never closed)

B. Failure visibility

Smell	✅ by design if	❌ broken if
Empty/null masking — a field empty because something broke upstream	the empty has a contract (declared nullable)	a count no longer matches its list; a .get(k, default) ate a missing key
Default masks dropped intent — a param with a default, or one the interface accepts and no code reads	the default is a real sensible value, and every accepted param is consumed somewhere	it silently absorbs a value that was supplied but got dropped. Harsher variant: the arg parser advertises a flag the handler never reads — documented, accepted, discarded, no error. An advertised no-op is worse than a missing feature: the missing one fails loudly and teaches
Fallback masks primary failure — a degraded path	it logs/flags that it engaged	"it works" actually means "fallback works, primary silently dead"
Partial success as success — a multi-step op, one rollup status	rollup is AND-of-all + per-item surfaced	a mid-step failure is swallowed by the final OK
Unfalsifiable success — no distinct signal for worked-vs-failed	success and failure produce different legible output	both look the same (empty output, silence). Ask: if this failed, could I tell?
Proxy read as evidence — a status, version, path or env var taken as proof of a different fact	the reading answers the question actually asked — the registry for "did it publish", a digest for "is this content current", the tool's own credential resolution for "can this run"	it answers a narrower neighbouring question and nothing says so. Four in one day: a CI job's success read as published (two channels shipped nothing behind six green ticks); CARGO_REGISTRY_TOKEN/CHOCOLATEY_API_KEY unset read as no credential exists while the token sat in credentials.toml / choco apikey; gh secret list empty read as the secret never existed when it was simply not shared with that repo; a version header read as content. The proxy USUALLY agrees — that is what makes it attractive and the disagreement invisible. The reading is not noisy, it is precise and about the wrong thing

C. Boundaries & contracts

Smell	✅ by design if	❌ broken if
Classifier blind spot / membrane miss — a matcher on crossing data	it sees through wrappers / chains / quotes / encodings	a wrapped or chained variant slips a start-anchored match
Encoding / quoting mangle — data crossing CLI↔shell, JSON↔DB, wire↔local	the boundary escapes / validates explicitly	it assumes clean input (quotes, newlines, unicode, "0.7" vs 0.7)
Schema drift — code assumes a shape the migration didn't produce	the unused shape is intentionally deprecated-but-kept	the code path is silently dead against the real schema
One predicate, two questions — a check written for one question reused for a second that looks identical	both questions genuinely have the same answer for every state, including the edge ones	they diverge on a state neither caller thought about — "what may I delete?" and "what exists?" agree everywhere except on things that are archived, so an existence test borrowed from a deletion allow-list judged every archived referent missing and a routine prune destroyed live links
Trust-the-input — consuming upstream data unvalidated	validated at the boundary	it assumes well-formed and NoneTypes three calls later
Two-sources-of-truth drift — a copy of a load-bearing thing	one is generated from the other	both are hand-maintained and have diverged
Invisible dependency — coupling that doesn't appear in the import list	dependencies are declared where a reviewer looks for them (imports, manifest, injected params)	the coupling is through symbols the logic resolves at runtime, so a portability review by import-list clears it. Three checks were audited for portability by reading imports; only one declared its dependency there, and the other two shipped and produced false findings against a foreign repo. Reviewing coupling by reading imports finds only the coupling that chose to be visible
Else-branch mis-route at N+1 — if A … else B standing in for an enumerable set	the else is a genuine catch-all whose behaviour is right for anything not-A	it's a correct default for the two types that existed and silently mis-files the third. Adding a type to a dispatch map would have written invalidated dead-ends into the store — correct in SQLite, wrong in the canonical log, surfacing only on a rebuild months later.

D. Indistinguishable incompleteness

An incomplete result that is indistinguishable from a complete one. Not wrong answers — incomplete ones that look complete. Every instance passes its own checks; the caller reads a partial truth as the whole one and builds on it. The class only becomes visible when someone re-derives the total through a different surface — so hunt it deliberately:

Smell	✅ by design if	❌ broken if
Silent truncation — a bounded read (limit / page size / cap) over a collection	the response carries the source-side total (total, points_count) next to the page, and the caller checks it or pages to exhaustion	the default cap truncates seamlessly — a 201-chunk source at max=200 joins into a string indistinguishable from complete; a limit=500 scroll against 3 847 points reads as "not found"
Completeness signal computed from the wrong side — has_more / is_last derived after filtering	derived from source cursor exhaustion	derived from the filtered page length — heavy filtering drives it false while whole pages go unread
Abbreviated identity on a consumer surface — an id/hash truncated for display where something downstream keys on it	the abbreviation is display-only, clearly non-canonical, full value adjacent	a consumer parses the rendered form as the identity — exact-match joins silently return 0-of-N. Never abbreviate an identity value where a consumer reads it
Absence asserted from a defaulting read — .get(k, default) / optional accessors make missing key and null value identical	absence claims are made with an explicit key-presence check (k in row), after enumerating every candidate field in the schema you already printed	a .get() default becomes "present but null"; a capability is declared absent while the sibling field that provides it sits in the same output
Absence claimed from a truncated enumerator — an existence/absence conclusion drawn from a capped read (ls … | head, LIMIT n, one unpaginated page)	you counted the enumerator against the cap first, or queried the authoritative registry (list-unit-files, the index) directly	the cap silently hid the item on the next line — an ls ~/.config/… | head reported a systemd unit absent that was there all along, and a whole plan built on "it does not exist." Absence from a bounded read is unfalsifiable by construction; prove it from a count or the registry, never a truncated view

The counter-move for the whole class: verify through the surface the consumer actually reads (not the producer's internals, not your test's mock), and design responses so completeness is checkable from the response itself. A partial read must be self-evidently partial.

E. Environment & control flow

Smell	✅ by design if	❌ broken if
Deploy-staleness — installed/copied artifact vs source	hash/mtime match, or a single source of truth	the box runs old code while source is "fixed" (the #1 recurring root cause)
Metadata version read as running code — a staleness call from recorded metadata (pip dist-info, lockfile, manifest) rather than the live artifact	the metadata provably tracks the artifact (non-editable install, freshly written)	under an editable/develop install the recorded version lags the loaded code — pip show said 1.13.14 while import resolved the newer develop tree, and a daemon was declared "frozen" on a version that was never its running code. Read the import path / process / served endpoint, not the manifest
Config/env divergence — behavior depends on env / version / run-context	the dependency is declared + checked with a clear error	assumed present (works interactively, absent in CI / cron / a peer's box)
Authority on the wrong field — a gate / filter / recipient check	keyed on the source of truth	keyed on a proxy that usually agrees
Gate wrong about whether the channel CAN run — a precondition check that gates on one location for a credential or capability	it consults the same resolution the tool itself uses, or falls through to it	it checks one env var while the tool reads a config file, so a present credential reads as absent and the step skips. Hard-failing does NOT fix this one — it converts a silent non-publish into a confident refusal, because the premise was never examined. Sibling of the row above and strictly worse: that one is keyed on a proxy for the ANSWER, this one on a proxy for the QUESTION
Gate gates its own escape — a guard blocking its own clear-path	the recovery action is always-open before the gate	the verb that would clear the deny is itself denied
Unrecoverable gate — a deny with a "do X first" message	doing X actually satisfies it	the satisfaction window closed before X can run
Dead branch by construction — a path an earlier check already decided	intentional belt-and-suspenders

This table is living. When a new class of issue bites you, add a row with its disambiguator — every incident becomes a permanent future check. Found one we're missing? PR it (see Contributing).

---

Phase 3 — Triage, verdict, record

1. Real issue → log it (an issue, a TODO, your tracker).
2. Confirmed by-design → record the verdict (an inline annotation or a .broccoli-accept line) so the next sweep skips it.
3. Rank by blast radius, then cut. Sort findings by the damage the failure would do, not by how many you found — a sweep with 8 prioritized findings gets acted on; one with 30 gets skimmed and ignored. Calibrate severity to the stated context: a missing backup is a 🟢 note for a scratch script and a 🔴 blocker for a billing path. Never grade an MVP against an enterprise checklist — that's noise wearing a badge.
4. Roll up a verdict: 🟢 GREEN (ship) · 🟡 YELLOW (ship + logged follow-ups) · 🔴 RED (blockers — name them).
5. Re-runs are idempotent: track counts over time. A rising silent-failure / debt count is the signal, not the absolute number.

Output contract

Every sweep ends with these four blocks, in this order. A finding nobody can act on is the same as no finding.

VERDICT   🟢 GREEN | 🟡 YELLOW | 🔴 RED        — one line, blockers named if RED

FINDINGS  ranked by blast radius, not by discovery order
          <pattern-name> · <file:line> · <why it is broken, not just what it is>
          severity calibrated to the STATED context, not to an absolute bar

COUNTS    silent-failures · accepted (.broccoli-accept) · new-since-last-run
          the trend is the signal; the absolute number is not

NOT COVERED   what this sweep could not check, and why

That last block is not optional, and it is the one people drop. A sweep that reports only what it found is indistinguishable from a sweep that found everything — precisely the failure family in §D, aimed at this skill's own output. Tooling you don't have, paths you skipped, a scope of changed when the risk is repo-wide, a language the linter doesn't parse: name it. An audit silent about its own blind spots converts "unknown" into "clean" in the reader's head.

The .broccoli-accept file

One confirmed by-design finding per line — a .gitignore for false alarms, so the next sweep stays quiet on what you've already judged:

# pattern:location — why it's intentional (and what would reverse it)
fallback-masks-primary:db/cache.py:fetch — logged degraded mode; reverse if callers start trusting []
blind-except:sync/git_notes.py — non-fatal best-effort write; errors visible upstream

Keep the why. A verdict without a reason is just a mute button — and the next person (or the next you) can't tell a real judgment from a silenced alarm.

---

Contributing

The pattern hunt is meant to grow across languages and harnesses — that's the whole point. Hit a failure class that isn't here? Open a PR with a row:

| **Name** — the smell | ✅ by design if … | ❌ broken if … |

Real war stories make the best rows. The catalog gets sharper every time someone adds the bug that just bit them.

---

Why "broccoli"?

Because it's the work you know you should do and skip anyway. This makes it a single command, gives the boring-but-load-bearing checks a place to live, and — once a verdict is recorded — means you never chew the same stalk twice. 🥦

---

Want the deterministic checks bundled into one command, and by-design verdicts recorded for you so they compound across a team? See INTEGRATIONS.md.