verify-plan-against-code

name

verify-plan-against-code

description

Use BEFORE dispatching any AI/subagent-drafted implementation plan to a build agent (codex, cursor, droid, a fresh Claude). Triggers on: 'dispatch this plan', 'implement this plan', 'send to codex/cursor', 'phase N of the build', any plan an LLM drafted (including the orchestrating Claude, from memory/CLAUDE.md/a summary) for a multi-file or multi-seam change. Run a citation-enforced ADVERSARIAL verification of every seam claim — file paths, function/flag/command names, routing/dispatch logic, test entry points — against the REAL codebase BEFORE coding begins. A claim with no file:line citation is unverified. Bake corrections into the plan before dispatch. This is the PRE-DISPATCH gate, complementary to a POST-DISPATCH verification pass on the agent's actual output.

verify-plan-against-code

AI-drafted plans cite file paths, function names, flag names, and routing logic that may not exist at the claimed locations. A build subagent reading only the plan implements against hallucinated anchors. Its tests pass (they assert the new code's own behavior, not the spec), and the bug surfaces at integration. This skill is the pre-dispatch gate; verifying the agent's receipts/output/commit after it runs is the post-dispatch one. Both are needed; this runs first.

When to invoke

• Before dispatching any LLM-drafted plan to a build subagent.
• Before implementing a plan that touches ≥2 files or ≥1 routing/dispatch seam.
• When the plan was drafted WITHOUT the drafter reading the actual files (drafted from memory, CLAUDE.md, or a prior summary).
• After a design council / thinktank (which reviews design), before implementation (this verifies code-reality anchors).
• Any time a plan cites a specific file:line / function / flag and the drafter has not shown a Read receipt for it.

When NOT to invoke

• Plans that only create NEW files with no integration seams (nothing existing to verify against).
• Plans drafted right after the drafter Read the relevant files with citations this session.
• Trivial 1-file edits with an already-verified root cause.

The verification protocol

1 — Extract every seam claim

Seam claims are assertions about EXISTING code the plan connects to:

Claim type	Example
File path	"add X to src/.../commands.py"
Function / class / command name	"in the Commands enum, add variant X"
Flag registration site	"add --rank to SEARCH_PYTHON_PASSTHROUGH_FLAGS"
Dispatch / routing logic	"the bootstrap front-door forwards --rank to the Python handler"
Test fixture / harness entry	"test via CliRunner invoking the app"
Config / constant location	"PUBLIC_TOP_LEVEL_COMMANDS in test_routing_parity.py"

Internal details of NEW files the plan will create are not verifiable; the EXTERNAL anchors it connects to are. Verify the seams, not the new internals.

2 — For each claim, grep/Read the REAL repo and produce a citation or a CORRECTION

The durable move is grep-then-cite (line numbers drift; the grep is what you re-run):

Claim:  "add --rank to SEARCH_PYTHON_PASSTHROUGH_FLAGS in rust_core/src/main.rs"
Verify: grep -n 'SEARCH_PYTHON_PASSTHROUGH_FLAGS' rust_core/src/main.rs
Result: const at main.rs:160, members listed below it → VERIFIED (cite the live line)

Claim:  "the bootstrap entry point is bootstrap.py:main"
Verify: grep -n 'def main' src/tensor_grep/cli/bootstrap.py
Result: actual symbol is `main_entry`, not `main` → CORRECTION NEEDED

3 — Hold an ADVERSARIAL stance: default is "this claim is wrong"

Do NOT ask "does this plan look reasonable?" Ask "which claim is false?" A plan that looks reasonable to an LLM reading only context is not verified — only the file:line check is. Routing/dispatch seams are the highest-risk class: plans usually get the what right (add flag X) and the where wrong (wrong registration site).

Real receipt (tensor-grep, 2026-06-26): a verification council on two subagent-drafted (non-council-verified) plans caught 5 blockers before a line was written:

1. A new search flag needed registration in BOTH the native allow-list AND bootstrap._TG_ONLY_SEARCH_FLAGS (the Python front-door) — the plan cited only one, so the flag would have leaked to ripgrep.
2. A new top-level command needed a Commands::X passthrough variant + dispatch arm in rust_core/src/main.rs — the plan omitted it, so the native front-door would have run the command name as a ripgrep search.
3. The plan's insertion line numbers were wrong (mid-function → SyntaxError); the real anchors were elsewhere.
4. A blocker in the sibling plan: an evict() method the plan called did not exist on the cache class.
5. An architecture flaw: appending to a log inside the dispatch path would be silently dropped on response-cache hits — the append had to move to the stream level.

4 — Produce a BLOCKER list; bake corrections INTO the plan before dispatch

## Plan verification report

### BLOCKERS (fix before dispatch)
1. Flag registration gap — plan adds `--rank` to the native allow-list only.
   Verified: it must ALSO go in `bootstrap._TG_ONLY_SEARCH_FLAGS`
   (grep -n _TG_ONLY_SEARCH_FLAGS src/tensor_grep/cli/bootstrap.py). Fix → step 2b.
2. Missing dispatch arm — plan omits `Commands::Orient` in rust_core/src/main.rs
   (template: the existing Commands::Map variant + its `handle_python_passthrough` arm). Fix → step 1a.

### VERIFIED (no correction)
- KNOWN_COMMANDS at src/tensor_grep/cli/commands.py — exists (Read).
- PUBLIC_TOP_LEVEL_COMMANDS at tests/e2e/test_routing_parity.py — exists.

Fold every correction directly into the plan steps — do NOT pass the blocker list as a separate doc. A corrected plan dispatched once beats an incorrect plan dispatched → audited → re-dispatched.

Solo verification vs a verification council

Situation	Approach
≤3 files, 1 seam	Solo: the orchestrator reads the files + verifies claims.
≥4 files or ≥2 independent seams	Council: 2-3 subagents each verify a subset of seams independently, then compare.
Plan drafted without reading any files	Council always — every claim is suspect.
Plan is expensive to execute if wrong (many agent-sessions)	Council always.

Council config: each verifier gets (a) the plan, (b) its assigned seams, (c) repo read access, and returns BLOCKER / VERIFIED per claim with file:line evidence — a claim with no citation is discarded. The orchestrator synthesizes and corrects the plan. (This composes naturally with a dynamic-workflow fan-out: a parallel lens per seam-group, sonnet, schema'd verdicts.)

Hard rules

1. A claim with no file:line citation is UNVERIFIED, not "probably fine." Burden is on the verifier to find the citation, not on the build subagent to discover the discrepancy.

2. Corrections go INTO the plan, not into a follow-up review. Pre-dispatch correction costs ~0; post-dispatch fix costs 1-4 agent-sessions.

3. A CliRunner/TestClient testing strategy for a front-door-routed CLI feature is itself a blocker — flag it (see dogfood-the-shipped-artifact).

4. The verifier finds blockers, it does not approve. Cite VERIFIED only when grep/Read confirms the entity exists at the claimed location with the claimed shape.

5. Verify routing/dispatch seams explicitly — they are where "right what, wrong where" hides.

6. Multi-site registration is a universal silent-failure bug class — enumerate ALL N sites before marking a registration claim VERIFIED. Any entity that must be registered in N places (a CLI flag in 2 front doors; a /v1 route in a cron registration + a test-scope exemption; an Android Activity in a manifest; a DI service in a container + a lifetime-policy list) fails QUIETLY if any one site is missed. Burden: (a) identify ALL registration sites for the entity type, (b) confirm each independently via tg callers <registration-symbol>, grep, or Read, (c) cite a file:line for every site. A plan that cites only one of N sites is a BLOCKER no matter how correct that one site looks. Cross-domain receipts of this class:

   Context	N-site registration	What was missed	Failure mode
   tensor-grep --rank flag	native allow-list + bootstrap._TG_ONLY_SEARCH_FLAGS	the Python front door	flag silently leaked to ripgrep
   billing /v1 route	cron 4-part registration + test_route_scope_coverage exemption	the test-scope exemption	route failed the test gate
   Android Activity	class + AndroidManifest.xml <activity> tag	the manifest	runtime crash on launch
   WordPress hook	listener + invocation

Relationship to other skills

Skill	Relationship
(post-dispatch verification)	Verifies receipts/output/commit AFTER the agent runs. This is PRE-dispatch. Both needed.
standard-dev-workflow	Its council-review phase reviews design; this verifies code-reality anchors. Run this after the design review, before implementation.
dogfood-the-shipped-artifact	The CliRunner-bypass risk this skill flags at plan level is what dogfood catches at runtime.
tensor-grep-code-audit	The tg callers/blast-radius technique for enumerating sites (Rule 6); its P7 documents the call-graph blind spots — string/list/decorator registrations callers can't see.

Anti-patterns

Anti-pattern	Risk
"The plan looks reasonable — dispatch it"	hallucinated paths/signatures → subagent implements against fiction
"The design council approved it"	design review ≠ file:line reality check
"The subagent will figure out the right location"	it inherits the wrong anchor and implements correctly against the wrong seam
"I'll catch it in review"	post-dispatch fix costs multiple follow-up sessions; pre-dispatch costs ~0
"The plan cites files I know exist"	file existence is necessary, not sufficient — the function/flag/constant must also exist there with the claimed shape

History

• 2026-06-26: created from the tensor-grep session. Two subagent-drafted plans (tg orient, tg session) were verified against real code before building; the council caught 5 blockers (front-door flag/command registration, wrong insertion anchors, a missing method, a cache-bypass architecture flaw), all baked into the plans before dispatch → both built clean on the first CI pass. External parallels: seldon (plan verification + codebase inspection), grounded (pre-flight identifier verification), claude-replan (parallel review agents), hallucination-grep (cross-check LLM output vs the real codebase).