tensor-grep-validation-and-qa

name

tensor-grep-validation-and-qa

description

Use when deciding what counts as proof that a tensor-grep (tg) change works — before trusting a subagent's "tests pass", writing a new test, claiming a routing/docs/release fix is done, shipping a doc-drift/ranking/classification heuristic off green fixture tests, proving a red-green baseline on a reverted/pre-fix commit, reviewing a payload-byte/ratio governance test, de-flaking a timing-sensitive test, or running the pre-push gate. Covers TDD-first discipline, the CliRunner-vs-real-binary trap, the fixture-green-vs-real-corpus-dogfood trap for precision/heuristic features, the `capfd`-vs-`result.stdout` capture-surface trap on routing/delegation changes (needs `tests/integration/` run with the native `tg` binary rebuilt, not just `tests/unit/`), the `tests/conftest.py` `sys.path.insert`-outranks-`PYTHONPATH` trap that can falsify a red-green baseline even with `tensor_grep.__file__` verified, a shared-envelope field growth breaking a payload-ratio governance test plus its tmp-path-length platform sensitivity, the self-gate-suite-subset-is-not-full-CI-matrix trap, a new test that proves nothing until seen fail on the pre-fix baseline, a `max(baseline*N, floor)` timing ratio degenerating to its floor below clock resolution plus the profile-before-attributing-a-flake discipline, preferring a structural order-based assertion over any wall-clock form, the certified/golden inventory (routing parity, docs governance, release-asset validation), agent-readiness/`tg dogfood`, benchmark-gated speed claims, acceptance thresholds, and which suite/marker/fixture to use for a new test, plus the `--preview` / `--no-sync` / `-x` gotchas.

tensor-grep validation and QA

This is the evidence-bar runbook: what is allowed to count as proof that a change to tensor-grep (the tg CLI) works, and how to add a test that actually enforces it. tensor-grep describes itself as a "benchmark-governed, contract-heavy codebase" (CONTRIBUTING.md:3) — many behaviors are pinned by tests that fail on drift, and speed claims are gated by measured numbers, not review opinion.

Who this is for

Two readers, written to the lower bound of each:

• A Sonnet-class AI in a cheap autonomous session: copy-pasteable commands and hard gates so you cannot silently skip validation.
• A mid-level human engineer: the why behind each gate, so you extend it correctly to new cases.

When to use this skill vs a sibling

Your task	Use
"Is this proof good enough to claim done?" / adding or picking a test	this skill
The non-negotiable gates (draft-PR-only, registration sites, fail-closed contract, push-race)	tensor-grep-change-control
Picking/reading a benchmarks/*.py script, the noise-floor rule for sub-10ms rows	tensor-grep-benchmark-and-proof-toolkit
Interpreting a tg doctor --json / tg dogfood field — what it does and does NOT prove	tensor-grep-diagnostics-and-tooling
A live bug/red-CI to triage	tensor-grep-debugging-playbook
"Has this already been tried and lost?"	tensor-grep-failure-archaeology
Internals/why the front door is shaped this way	tensor-grep-architecture-contract
Env var / flag reference	tensor-grep-config-and-flags
Day-to-day CLI invocation syntax	tensor-grep-run-and-operate
Writing docs of record (AGENTS.md, README, docs/*.md)	tensor-grep-docs-and-writing
Release mechanics / positioning	tensor-grep-release-and-positioning
Using tg to navigate a codebase	tensor-grep (usage skill) / code-search-and-retrieval-reference

No skill routes around change-control. This skill tells you what evidence a gate needs; it does not relax any gate in tensor-grep-change-control.

---

Part 0 — THE ORACLE FAMILY: when your verification isn't (read this first)

The single most repeated failure mode in this repo. TEN distinct forms, most in ONE session (2026-07-25; forms 7 + 8 added 2026-07-26, form 9 2026-07-27, form 10 2026-07-28). Every form shares one shape: something that looks like verification isn't.

(The count read NINE while ten forms were present, for three days. A header that miscounts the thing below it is the smallest possible instance of this Part's own subject -- and it was found by an audit that COUNTED the forms rather than reading the sentence. Re-derive the number when you add one; do not trust the header, including this one.

Both halves of that two-file edit were still wrong on 2026-08-01, in OPPOSITE directions. AGENTS.md's header kept saying "nine forms" for four more days -- the fix landed here and never crossed over -- while THIS file, which had the count right, misdated forms 8-9 to 2026-07-27 when Form 8's own text reads 2026-07-26. Each doc was half right, and reading either one alone confirmed it. The dates and the count are now derived from the **Form N —** headings themselves, which are the only authority, and tests/unit/test_skill_library_drift.py now fails if the stated count and the enumerated forms disagree in EITHER file -- because a rule that says "re-derive the number" had already been read, agreed with, and half-applied.)

The one question that catches all ten — before trusting any green signal, ask: "what would this check show if the thing it verifies were BROKEN?" If the answer is "the same", it is not verification.

Forms 1-5 assume the setup worked and the comparison was wrong. Form 6 inverts it: the assertion is fine and the SETUP silently no-opped, so the hostile arm was never hostile. Ask both questions.

Two GLOBAL skills carry the general form of this Part — load them, they are not tensor-grep-specific and this file is not a substitute for either (added 2026-08-02; a census found them referenced in ZERO tracked files here while verify-plan-against-code had 23, so nothing routed to them):

• detect-the-false-green (~/.claude/skills/) — when a green result is about to LICENSE a claim ("fixed", "clean", "covered", "safe to delete"), or when the check has never been observed to fail. Covers the shapes this Part's forms are instances of: an early return, a skip nobody reads, a platform-gated test, a confident zero, and an audit that re-implements the gate it audits.
• author-a-probe-that-cannot-lie (~/.claude/skills/) — read it BEFORE writing any probe or benchmark whose number you will act on. Positive control, blind-vs-busy empty results, arm interleaving, max-not-mean, and shared-resource pollution windows — the last one binds here, because the dev box is a shared server.

Adding a Form to the family stays a two-file edit (AGENTS.md + this file). Adding a POINTER, like the two above, does not — those skills own their own content and must not be copied in.

Form	What it looks like	Direction of harm	Receipt
1. Normalize-both-sides	A comparator applies the same lossy transform to BOTH arms	Masks real defects — silent	#262 (CRLF/encoding-blind rg-parity oracles); surviving accepted limit at tests/helpers/rg_parity.py:560, now proven lossy and pinned by PR #748
2. Harness-corrupts-output	Post-processing mangles a byte-correct result before comparison	Manufactures false failures	test_output_golden_contract.py::run_tg did line.replace("\\","/") on the WHOLE line, turning a binary notice's \0 into /0; fixed in #746
3. Test-never-executes	The file exists, looks like proof, and SKIPS	Fakes coverage	test_native_json_byte_fidelity.py skipped in every CI job; fixed #746, class-fixed #749
4. Gate-diagnosis-wrong	A gate's conclusion is right, its root cause is false	Sends the fix at the wrong target	The gate that found form 3 claimed TG_REQUIRE_RG_PARITY was in "zero workflows" — it is at grep -n TG_REQUIRE_RG_PARITY .github/workflows/ci.yml. No line number here on purpose. This anchor has now been re-stamped twice (:706 -> :764) and was wrong again by 2026-08-23 (real hits: 907/918/925; :764 had drifted onto unrelated cargo test --lib commentary). Re-stamping is the failure mode this very table warns about -- run the grep
5. Repro topology deletes the mechanism	Every fixture shares one structural property, and that property is the one that matters	Proves a strict subset of the real defect — defeats even an honest RED	PR #750: repro + all 4 tests used non-git tempdir(), the one topology where the fix's mechanism suffices; inside a git repo the fix is a no-op
6. The FIXTURE never applied	The hostile condition silently failed to take effect, so the "bad" arm is really the good arm

Form 7 applies to benchmarks and scorecards, not just tests. Same question, unchanged: what would this column show if a tool were GOOD at it? A tied-at-floor column is worse than no column, because it looks like data. Every scored dimension needs at least one run where arms differ, or it gets deleted with the reason written down.

Form 8 — the SPLIT ORACLE (2026-07-26). A precondition proved in a DIFFERENT run is not THIS run's precondition. tests/unit/test_trust_benchmark_premise.py pins "rg cannot signal an incomplete scan inside its JSON stream" with two arms: ARM 1 runs rg --json over a tree with an unreadable directory and asserts no incompleteness marker; ARM 2 asserts rg exits 2. ARM 1 never asserted its OWN run exited 2 — so on a tree where the directory is actually readable, rg exits 0, completes, correctly emits no marker, and ARM 1 passes, reporting "rg hides incompleteness" on the evidence of a scan that was never incomplete. What made it feel safe is the shape to learn: a helper DID verify the directory was unreadable to the test process, and ARM 2 DID assert exit 2 — both true, neither load-bearing for ARM 1. Move the premise assertion INTO the run that draws the conclusion. (Canonical text: AGENTS.md "The Verification-Oracle Family", Form 8.)

Form 9 — the REVIEWER'S expected number is the broken half (2026-07-27). Forms 1-8 all assume the checker is wrong about the CODE. This one inverts the subject: a census mismatch is a two-sided hypothesis, and the side that is wrong is often the expectation you brought to it. In one session this fired twice in opposite directions — an envelope seam expected at 2 sites was really 3, and four comments suspected of claiming "observed no walk" turned out to be individually correct. Both would have been filed as product defects on the strength of a number that felt wrong. So: read the breakdown before filing the finding. A count that disagrees with your expectation is a prompt to enumerate the members and look at each, not evidence of a bug. The same applies to an audit handed to you by another agent: the 2026-07-27 skill audit's own corrected line numbers were stale, because they were computed against a worktree 28 commits behind origin/main — the finding was real, the expected value was not. Re-derive before you act on someone else's number.

Form 10 — the oracle's UNIT is the BRANCH, and the defect lives in the MERGE (2026-07-28). Every form above assumes the check is looking at the right CODE. This one is about the right TREE. PRs #835 and #836 were each fully green (48 checks apiece, bidirectional controls, an independent adversarial gate on one) and main went red the moment both were on it: #835 asserted exactly ONE line of --mermaid output mentions INCOMPLETE RESULT, #836 deliberately added a second. Git merged them with NO textual conflict — the collision is semantic and exists only in the union, which CI never evaluated because each PR's checks ran against its own base. Cost: main red and a release lost (Semantic Release skipped, so v1.101.8 was never produced). It recurred immediately: #837, rebased onto the still-broken main, returned the identical assert 2 == 1.

Before pushing, rebase onto the REAL target and run the union. A branch green against a stale base has verified a tree nobody will ship. Merge is semantically live even when git is silent when (a) two PRs touch the same OUTPUT SHAPE — different files and different functions still collide, or (b) one PR adds to a rendering that another PR COUNTS. Grep the whole suite for assertions about the shape you are changing: the file you are editing is not the boundary of the blast radius. (This family is MIRRORED — see AGENTS.md; adding a form is a two-file edit.)

Ambient keys and tracker prose are instruments too (2026-08-06)

• A --sign no-key RED that only pops TG_EVIDENCE_SIGNING_KEY is split from the default key path (AGENTS.md A70) — same family as Form 8 / ambient-fixture pollution. Merge/poller twin (A77): a gh pr checks pipe into a stdin-eating heredoc can empty the checklist and read as ALL_TERMINAL while jobs are still pending — write checks to a file and require heavy lanes present by name. Quota twin (A78): usage-limit seat errors are FAILED seats, not pending Sol SHIP. Status-stamp twin (A79): READY→BLOCKED stamps must retarget tracker pins in the same PR.
• A TASK_BOARD “campaign note” under the canonical index is not documentation; it is a malformed row to the tracker parser (A71).
• Bare-wheel dogfood without semantic extras cannot grade tg find (A73).

Running the probe: the LOCATION trap (2026-07-26)

A perturbation proves nothing if the thing you perturbed survives elsewhere. Verifying the truncation_cause doc ratchet, the first probe removed ONE occurrence of unreadable-path from docs/CONTRACTS.md and the test still passed — which reads as "the ratchet is toothless". It was not: the string appears twice, and the check is a substring scan over the whole file. Removing EVERY occurrence failed the test correctly.

Before concluding a guard is broken, confirm your perturbation actually removed the property it guards — grep -c the string first. This is the setup-not-assertion failure wearing a third face: what looked like "passes in both arms" was really a probe that never created a second arm.

Writing a hostile fixture (Form 6 defence)

Any fixture that simulates something BAD — permission denied, network partition, disk full, killed process, corrupted file, missing binary — is a claim about the world, and claims get verified. Assert the fixture BITES before the probe runs, and abort loudly if it doesn't:

try:
    os.listdir(denied_dir)
    print("entries listed -> STILL VACUOUS")  # abort: the fixture did nothing
    raise SystemExit(1)
except PermissionError:
    pass  # good, the fixture is real

Windows ACL specifics learned the hard way (#281):

• To APPLY a deny ACE, use PowerShell with the SID, not an icacls account string. [System.Security.Principal.WindowsIdentity]::GetCurrent().User → FileSystemAccessRule with Deny, plus SetAccessRuleProtection($true, $false). icacls account-name forms (%USERNAME%, MACHINE\user) failed to map and processed 0 files while looking almost like success.
• To REMOVE it, icacls <dir> /reset — it takes no account name and works unelevated on a directory your own user locked. Get-Acl/Set-Acl can fail with SeSecurityPrivilege because Get-Acl pulls a section you cannot write back.
• A mapping failure is not a privilege failure. If /reset ALSO fails, the DACL belongs to a different SID and genuinely needs elevation — that distinction is what proved #268 operator-gated rather than a tooling quirk.
• Always restore the ACL and delete the fixture when the probe finishes.

The rules that fall out

• SKIPPED IS NOT PASSED. Read the skip count, every time. A green suite can report proof that never ran. If a test needs an env gate or a built binary, grep whether a job actually provides BOTH — tests/unit/test_native_e2e_ci_coverage_contract.py now asserts this as an invariant.

• A golden diff is evidence about the harness+product PAIR, never the product alone, whenever the harness post-processes before comparing. Reading one as a product defect sent an agent hunting an emitter that did not exist.

• A gate's clearance is a hypothesis — and so is its ROOT-CAUSE STORY. Verify the diagnosis, not just the finding. Relaying an unverified root cause nearly produced CI plumbing that already existed.

• Isolation-level evidence is not outcome-level evidence, and this binds PROSE. Measuring a helper in isolation and writing it up as a user-visible failure is the both-arms trap in claim form. In #747 an isolated workspace_root_guard=False became "the guard never fires"; the control arm through real main_entry() showed it fires IDENTICALLY in both arms — the defect was latent. A confidently-wrong comment is worse than none.

• When you cannot observe RED, say so. CPU-SAFE forbids compiling, so a Rust fix often cannot watch its own test fail pre-fix. The correct move is a STRUCTURAL argument from pinned source, stated plainly as an argument — never dressed as an observation. Gates judge whether the chain closes; the disclosure is expected behaviour, not a defect.

• Prefer an invariant to an enumeration. An enumeration is correct when written and silently incomplete on the next addition. Three fixes this session replaced one: _TG_ONLY_SEARCH_FLAG_PREFIXES parity (#272), the CI-coverage invariant (#749), and the rg-grammar differential model (#745).

• A modelled gate must be proven able to FAIL. #745's fuzz gate was validated by reverting one line (72 shapes, exit 1), mutation-killing 6/6, and checking its oracle against real rg --debug (301/301). A green gate that cannot fail is worse than no gate.

• A control that reproduces the failure is SUFFICIENT, not proven OPERATIVE. A two-arm control matched CI's failure byte-for-byte (same exit code, same stdout) and was called "confirmed"; a fix was dispatched on that basis. .github/workflows/ci.yml:688 then showed the failing job never builds the binary the control had forced into the mix -- the reproduction was real, the mechanism was not the one CI runs there, and the agent had to be recalled mid-flight. Say "sufficient" until you have traced that the SAME mechanism fires in the real failing job; a control earns authority only by REPRODUCING the failure in one arm -- "the unmodified control still passes" rules out nothing on its own.

2026-08-12 campaign bullets (dated)

• RED-by-design is valid ONLY for the EXPECTED failure-reason class (A61). A planned-RED test earns its red only when it fails with the expected AssertionError/reason class. A collection error, import failure, fixture/setup crash, or any die-before-the-contract exit is a GATE FAILURE, not a behavioral RED — it proves the harness broke, not that the contract was exercised. Pin the exact expected refusal/reason in the assertion and reject any arm that never reaches it.
• Workflow receipt verification must aggregate EVERY per-node receipt and require union == expected manifest population. One valid node receipt cannot clear a job: the verifier must collect all per-node receipts, take their UNION, and assert that union equals the expected manifest's full node population — a subset match (or a single node's success) reading as "workflow verified" is the false-zero family applied to receipts.
• Live CI evidence binds to the EXACT tested tree and is recorded OUTSIDE it. A CI verdict is evidence about specific SHAs — record head + base + merge-ref explicitly — and the evidence itself lives OUTSIDE the tested tree (a PR comment, a docs PR), never committed to the tested branch as self-covering proof of itself. A receipt that only exists inside the tree it vouches for is circular.
• Shared-filesystem timing wobble must reproduce in ISOLATION before widening any bound. On a shared dev/CI box, a one-off timing failure is a hypothesis about contention, not about the bound: reproduce the wobble in an isolated run first; if it does not reproduce, the correct response is suspect-instrument (the shared filesystem), never a looser assertion — widening a bound on an unreproduced wobble converts a shared-resource artifact into a permanent weakened contract.

A89 — the REAL-ARTIFACT parity arm (2026-08-08/09, M17 wave)

A parity test whose "real" arm is FAKE-BACKED (a stub producer standing in for the shipped binary, a range.byteOffset field never emitted by any real backend) makes three arms agree on the bug itself: the schema test, the fake-backer, and the product all "confirm" the shape that the real artifact would have broken. The M16/M17 receipts (A89, 2026-08-08) are this family applied to oracle INPUTS: when the real producer is cheap to invoke, ADD A REAL ast-grep --json subprocess arm (or equivalent real-binary arm) — a fake-backed arm is a hypothesis about what the producer does, not a measurement. Rule: every parity claim names WHICH producer backed each arm (fake-backed vs real-artifact); a suite whose arms are all fake-backed proves only that its fakes agree with each other.

A87 — static review is not a typecheck (2026-08-08/09, receipts #987/#988)

Two Rust PRs passed codex static audits and failed the FIRST real compile — #988 survived three audit rounds then hit E0599/E0308/E0382; #987's regression surfaced only on the full matrix (its author's self-gate never ran tests/unit/test_backend_bug_fixes.py). Rules: for Rust, "static SHIP" is SHIP-PROVISIONAL until the first CI compile of the head SHA completes (the first compile IS the typecheck gate); a self-gate's suite selection is a hypothesis until the matrix runs — state which suites ran and which were skipped alongside any self-verified result.

• A CHECK-RUN LIST WITH UNEXPANDED MATRIX PLACEHOLDERS IS A SKIPPED JOB, AND IT LOOKS LIKE A PASS (2026-08-19). A green PR whose checks read test-python (${{ matrix.os }}, py${{ matrix.python-version }}) did not run those lanes -- the job was never instantiated, so the matrix was never expanded. Any summary that counts failures == 0 reports this as success. Measured three times in one campaign: a 3,500-line refactor merged having run ZERO tests; the doc/skill governance suites never ran on doc changes; and the FORMATTER never ran on doc changes, so a docs PR reddened main and surfaced days later on an unrelated code PR. The root cause is the same each time -- ci.yml's cost-smart changes filter is a HAND-WRITTEN PATH LIST, and it watched a narrower set than the jobs behind it depend on. Before calling a PR green, require EXPANDED lane names (awk -F'\t' '$1 ~ /test-python \(/' and assert a count), not merely zero failures. None of the three was found by reading ci.yml.

• A CI JOB'S NAME IS NOT ITS FAILURE (2026-08-19). Formatting & Linting runs ruff AND mypy; it failed on mypy while a cycle was spent reproducing ruff. Read the failing STEP (gh api .../jobs/<id> --jq '.steps[] | select(.conclusion=="failure") | .name') before reproducing anything.

• A SPLIT HAS A HARD FLOOR SET BY TEST-PATCH TOPOLOGY, AND mypy ADDS A SPLIT-ONLY FAILURE CLASS (2026-08-19). Python resolves bare names through the DEFINING module's globals, so every function referencing a monkeypatched name by bare identifier must stay co-located with wherever the test's setattr lands. Measured on benchmarks/run_gpu_native_benchmarks.py: that call-graph closure is 1,752 lines / 17 functions, so the file CANNOT reach a 1,500-line limit by splitting. Derive the closure before scoping a wave -- a line count is not a split plan, and lowering a ratchet pin is the honest outcome where the floor binds. Separately, after any split, from .impl import X in the facade is a PRIVATE binding under mypy's implicit_reexport = false (runtime resolves it; mypy fails attr-defined). It cannot appear pre-split, because the symbol was locally defined and nothing was re-exported. Use from .impl import X as X, then re-check that ruff --fix did not merge the import blocks back and drop names -- its organiser silently dropped six in this same campaign.

Related global skill: measure-what-it-claims (same family, generalised beyond this repo).

Part 1 — What counts as evidence here (in order of trust)

Ranked by how hard each is to fake, cheapest-to-check first:

1. A failing test written before the fix (TDD-first). CONTRIBUTING.md "Performance Discipline": "Start with a failing test when behavior changes." Repeated in AGENTS.md Operating Rules #1 (AGENTS.md:389). If you cannot point to the test that failed before your diff, the fix is unverified — see superpowers:test-driven-development.

2. A contract test, not just a behavior test. This repo names them test_*_contract*.py / test_*_contracts.py (e.g. tests/e2e/test_backend_contracts.py, tests/e2e/test_io_contracts.py, tests/unit/test_main_cli_contracts.py, tests/unit/test_rg_contract.py). A contract test asserts an invariant that must hold for every implementation of a protocol (every ComputeBackend must expose .matches / .total_matches / .is_empty — tests/e2e/test_backend_contracts.py:8-12), not one code path's happy case.

3. Dogfood on the real binary, not CliRunner. tests/unit/ uses Typer's CliRunner 400+ times (grep -rc CliRunner tests/unit/*.py) — CliRunner calls the Typer app object directly and skips tensor_grep.cli.bootstrap:main_entry entirely, so a routing bug in the bootstrap front door (the layer that intercepts plain-text searches and forwards them to rg before Typer ever sees argv) is invisible to it. This is not hypothetical: the tg search --rank flag shipped broken to real users while every CliRunner test stayed green, because the flag was missing from one of the two search-flag front doors (CONTRIBUTING.md:73, AGENTS.md:411-418). After any command/flag/routing change, run the real binary: python scripts/dogfood/dogfood_features.py (installed tg on PATH) or the clean-room Docker path in scripts/dogfood/README.md. See dogfood-the-shipped-artifact (global skill) and tensor-grep-change-control Part 5.

   A dogfood/ run mostly exercises a WARM, cached path — repo-map/AST-parse state already populated from a prior call — so it can misjudge a change whose effect is COLD-path-only. Receipt: a warm end-to-end dogfood read the walk-merge ( — re-derive with: grep -n '_python_imports_and_symbols' src/tensor_grep/cli/repo_map.py) as ; an isolated cold microbench of the same function (fresh process, single pass over distinct inputs) showed it is actually (961ms→446ms) — the warm run never exercised the changed code path. To validate a cold-path optimization, microbench the target function directly or clear the cache between reps; never trust a single warm end-to-end dogfood run as the sole evidence for a performance change (pair with ; see Part 1 point 15 below for the same warm/cold discipline applied to a whole-campaign verdict pass).

---

Part 2 — Required local validation (run before push)

From CONTRIBUTING.md:5-14 and AGENTS.md:654-698:

uv run ruff check .
uv run ruff format --check --preview .
uv run mypy src/tensor_grep
uv run pytest -q

For release/workflow/package-manager changes, also:

uv run python scripts/validate_release_assets.py

Gotchas that each cost a real CI cycle when missed:

• ruff format needs --preview; ruff check must NOT get it. CI runs ruff format --check --preview . but ruff check . with no --preview. Running ruff format without --preview locally is an active revert — it rewrites preview-style lines back to non-preview style on disk, so the next CI ruff format --check --preview fails on lines you never touched. Passing --preview to ruff check produces false failures instead (preview lint rules like RUF056 don't match the CI lint gate). (CONTRIBUTING.md:22)
• Windows CRLF false-alarms a bare ruff format --check. .gitattributes pins *.py/*.rs to eol=lf; run ruff format --preview <files> (which normalizes) before trusting a local check. Audit real on-disk endings with git ls-files --eol — git show/git cat-file -p smudge output and can report false CR. (CONTRIBUTING.md:24)
• Markdown ruff format --check --preview on Windows disk is not a blob defect. *.md is not in .gitattributes eol=lf. A working-tree FAIL with core.autocrlf=true can be CRLF-only; pipe the git blob (git show origin/main:path | ruff format --check --preview --stdin-filename path -) before opening a format PR. HYGIENE-FORMAT 2026-08-30 retired on this: 15/15 blobs passed, disk failed. (detect-the-false-green)
• mypy runs in strict = true mode targeting python_version = "3.11" syntax even though the repo's CI-tested floor is 3.11-3.12 (pyproject.toml:559, requires-python = ">=3.11") — new functions need full type annotations (disallow_untyped_defs = true); do not rely on inference alone.
• uv run alone re-syncs the environment to default deps and silently drops optional extras (e.g. [dev]'s tree-sitter). If a prior step installed extras deliberately, use to keep them — this is exactly what CI's job does before running the readiness gate (). Forgetting after an extras install is how a "clean" local run diverges from what CI actually validated.

---

Part 3 — The certified/golden inventory

Three test surfaces are explicitly named in CONTRIBUTING.md "Important surfaces" (:75-79) as the ones that must stay in sync with any workflow/docs/release-asset change. A fourth (routing parity) is the load-bearing contract behind the "Adding a Command or Flag" rule. Treat all four as CI-blocking certified truth, not advisory tests.

1. Routing parity — Python launchers + native golden output

• tests/e2e/test_routing_parity.py runs the same argv through three launchers — python -m tensor_grep, the compiled native tg binary, and bootstrap.py — and asserts matching exit code / stdout / stderr (run_command, LAUNCHERS = ["python-m", "native", "bootstrap"], test_routing_parity.py:146-160,163,404-489). It also pins PUBLIC_TOP_LEVEL_COMMANDS (test_routing_parity.py:18-69) against both Python's and native's visible --help command lists (test_top_level_help_visible_commands_match_public_contract, :554-564) and pins PUBLIC_SEARCH_HELP_FLAGS (from src/tensor_grep/cli/rg_contract.py:388) against both search --help outputs (:525-537).
• rust_core/tests/test_search_golden.rs is a Windows-only (#![cfg(windows)]) Rust integration test that runs the built native tg binary against fixture data in tests/golden/fixture_data/ and diffs the output against committed golden files (tests/golden/*.txt, e.g. simple_string_match.txt, case_insensitive_match.txt, regex_match.txt).
• CI wires this as the search-golden-parity (windows-latest) job, which runs cargo test --test test_search_golden (.github/workflows/ci.yml:522-547), and separately the cross-platform test-python matrix job runs the full tests/ tree including tests/e2e/test_routing_parity.py (uv run pytest tests -v --tb=short -m "not eval", .github/workflows/ci.yml:406-413). Both are required by the Semantic Release job (needs: [..., search-golden-parity, ...], .github/workflows/ci.yml:942-943) — a routing-parity regression blocks the release, not just the PR.
• This is the concrete enforcement mechanism behind the "4 registration sites for a command / 2 front doors for a search flag" rule in tensor-grep-change-control Part 3 — when you add a site, add it here too, or the CI registration-completeness gate (blocking since v1.17.1, #282) fails the run.

2. Docs governance — content-pinned assertions on docs of record

Several tests/unit/test_*_docs_governance.py / test_*_docs.py files assert that specific strings still appear in specific docs, so a docs edit that silently drops a load-bearing claim fails CI instead of drifting unnoticed:

• tests/unit/test_public_docs_governance.py — pins README pointers to canonical docs (docs/benchmarks.md, docs/tool_comparison.md, docs/gpu_crossover.md, docs/routing_policy.md, docs/harness_api.md, docs/harness_cookbook.md), capability phrases ("tg calibrate", "tg mcp", "native CPU engine", "benchmark-governed"), and per-release verified-commit/tag markers (test_public_docs_governance.py:1-56).
• tests/unit/test_enterprise_docs_governance.py — pins README links to docs/CI_PIPELINE.md, docs/SUPPORT_MATRIX.md, docs/CONTRACTS.md, docs/HOTFIX_PROCEDURE.md, docs/EXPERIMENTAL.md, a ## Future Work heading, the CI-tested-vs-best-effort Python version matrix, and that docs/CONTRACTS.md explicitly excludes experimental surfaces (tg worker, TG_RESIDENT_AST) from stability guarantees.
• Sibling governance files worth knowing exist: test_benchmark_docs.py, test_benchmark_governance.py, test_harness_api_docs.py, test_issue_intake_governance.py, test_routing_policy_docs.py, test_stamp_release_assets.py.
• Full authoring rules (which doc owns which contract, the two governance layers) live in tensor-grep-docs-and-writing — use that skill when editing a governed doc; use this skill to know the check exists and is CI-blocking.

3. Release-asset validation

• scripts/validate_release_assets.py — a standalone validator (validate_all(), CLI entry main(); locate both with grep -n '^def validate_all\|^def main' scripts/validate_release_assets.py). Since the 2026-08-19 size-campaign split it is a thin FACADE over scripts/_release_assets_checks/ (ci_workflow, release_workflow, workflow_checks, docs_and_manifest_checks); the primitives the tests patch — _read, _version_from_* — deliberately stayed in the facade, because a test patching a module attribute that production no longer reads passes while the code under it is unchanged. Invoke and import it exactly as before. It checks release/package-manager asset consistency: README canonical-doc links and release markers, uv.lock editable version parity with pyproject.toml/rust_core/Cargo.toml/npm/package.json, and more. Run it directly: uv run python scripts/validate_release_assets.py — exit 0 and "Release/package assets validation passed." on success, exit 1 with one ERROR: line per failure otherwise.
• tests/unit/test_release_assets_validation_*.py (themed siblings split from the former monolith; still one of the largest release-governance suites) exercises validate_release_assets.py module functions directly via importlib.util rather than shelling out, including test_should_validate_release_and_package_assets_consistency which just calls validate_all() and asserts errors == [] against the real repo state — i.e. it fails the instant any of the other release-asset invariants regress.
• Related validators worth knowing exist for release proof (not just static asset shape): scripts/verify_github_release_assets.py(→test_verify_github_release_assets.py), scripts/validate_pypi_artifacts.py, scripts/validate_release_binary_artifacts.py, scripts/validate_release_version_parity.py, scripts/validate_pr_title_semver.py, scripts/stamp_release_assets.py.
• CI enforces this via the release-readiness job (a strict docs build plus workflow/package-manager validator checks, docs/CI_PIPELINE.md:16) — also a needs: dependency of Semantic Release. Deep release-mechanics coverage (push-race, PR-title→bump schema) lives in ; this skill only anchors it as a certified test surface.

Golden/snapshot output tests (a fourth, smaller certified surface)

• tests/e2e/test_output_golden_contract.py — 21 GOLDEN_CASES (derived 2026-08-12; this spot was stamped "20" — recount the GOLDEN_CASES = [ list entries, which live at :56-89 now, was cited :28-60; locate with grep -n "GOLDEN_CASES = \[" tests/e2e/test_output_golden_contract.py). default/--cpu/-o/-c/-r/-n/binary/--json/--ndjson combinations run through both python-m and native launchers and compared for output parity.
• tests/e2e/test_output_snapshots.py uses the pytest-snapshot plugin's snapshot.assert_match fixture (pyproject.toml:637, dev dependency) to pin exact JSON-formatter output, with file-path normalization to <FILE> so the snapshot stays host-independent (test_output_snapshots.py:5-46). Marker: pytest.mark.snapshot (registered in pyproject.toml:43).

Per-task-pinned agent-accuracy gate (a fifth certified surface, tests/eval/, new directory)

tests/eval/test_agent_accuracy.py is its own top-level test directory, distinct from unit/e2e/ integration — a capability-regression gate, not a code-contract test. test_agent_accuracy_gate asserts not misses over a golden set of agent-capsule tasks (#690/#696/#693): any single task regressing fails the gate, with no aggregate-score floor to absorb it. This is the loop-4 hill-climbing instrument for this repo (see Part 1 point 13 above for the full discipline and the #250 receipt). All 16 golden tasks live inside src/tensor_grep itself, which is a known self-referential-corpus risk (a visible answer key, a Goodhart/contamination surface) — the standing mitigation is that every real tg prepare/tg agent misroute found in the wild becomes a NEW permanent pinned task rather than a one-off patch, generalizing the fix instead of just closing the symptom.

---

Part 4 — Agent-readiness / tg dogfood

scripts/agent_readiness.py is a fast (3-5 minute) CI-blocking dogfood gate for agent-critical surfaces — separate from, and complementary to, the full local-validation gate (AGENTS.md:684). tg dogfood (dogfood() in src/tensor_grep/cli/main.py, deliberately with NO line number — the one that used to sit here drifted past the end of the file when main.py was split on 2026-08-20; find it with grep -n "^def dogfood" src/tensor_grep/cli/main.py) wraps the same check plan with a one-page verdict and an optional --timeout-s (default 170.0) around the nested readiness process.

Run it directly:

python scripts/agent_readiness.py --output artifacts/agent_readiness.json
tg dogfood --output artifacts/dogfood_readiness.json

Useful flags on scripts/agent_readiness.py (main(), :1258 (re-derive with: grep -n '^def main' scripts/agent_readiness.py)): --json (machine-readable report to stdout), --no-shell-probes (skip public shell version probes — used by CI's Linux agent-readiness job), --only-shell-probes (Windows-only shell probes, mutually exclusive with --no-shell-probes — used by CI's windows-agent-readiness job), --no-wsl-probe.

Acceptance semantics: the script's exit code is 1 if report["summary"]["failed"] else 0 (:1336 — re-derive with: grep -n 'summary.*failed' scripts/agent_readiness.py) — any failed check fails the whole gate; there is no partial-credit threshold. CI wires two blocking jobs off it — agent-readiness (Ubuntu, --no-shell-probes --no-wsl-probe, .github/workflows/ci.yml:121-157) and windows-agent-readiness (Windows, --only-shell-probes, :159-193) — and both are needs: of Semantic Release (release: job at ci.yml:1121, needs: at :1123 — re-derive with: grep -n '^ release:' .github/workflows/ci.yml), so a readiness regression blocks the release the same as a routing-parity regression.

Checks currently in the plan (build_check_plan, names verified at scripts/agent_readiness.py:698-1009): public-version-{powershell,cmd,pwsh-noprofile,git-bash,wsl, python-subprocess}, public-doctor-{cmd,pwsh-noprofile}, public-windows-launcher-quoted-patterns, public-search-advertised-flag-sweep, repo-cli-build-warmup, repo-doctor, context-render-trust (the context_consistency agent-trust check — AGENTS.md:821,848), rg-parity-edges, broad-generated-scan-guard, ast-info-json, ast-run-smoke, mcp-context-render-smoke, mcp-stdio-protocol-smoke, agent-capsule, agent-capsule-mixed-language, agent-capsule-hardcases, docs-claim-check. This list drifts with each release — re-verify with the grep in Provenance below rather than trusting this snapshot.

For what a tg doctor --json field actually proves (vs merely install evidence), see tensor-grep-diagnostics-and-tooling — this skill only covers the readiness gate as a pass/fail CI evidence surface, not field-by-field diagnostic interpretation.

---

Part 5 — Benchmark-gated speed claims (summary; depth lives in the sibling)

Never claim a speedup without a measured line vs the accepted baseline (AGENTS.md:702, CONTRIBUTING.md:37-42). The which-script decision table, the fair-baseline rule, and the launcher-attribution/stale-binary-refusal rules live in tensor-grep-benchmark-and-proof-toolkit — load that skill before running or reviewing a benchmark. This skill records only the acceptance thresholds, which are QA-gate facts, not benchmark methodology:

Gate	Default threshold	Where
benchmarks/check_regression.py CLI	--max-regression-pct default 5.0% slowdown fails	check_regression.py:64,66 (CLI arg)
perf_guard.check_regressions() (library default, used when no CLI override)	max_regression_pct 10.0%	src/tensor_grep/perf_guard.py:48-53
Noise-floor filter	rows with baseline_time_s < min_baseline_time_s (CLI default 0.1s, library default 0.2s) are skipped entirely — avoids false regressions from scheduler jitter on tiny durations	check_regression.py:70,72, perf_guard.py:52,76-77
Sub-10ms hot-query rows	use an absolute jitter tolerance in addition to the ratio check (a 5% ratio on a 2ms row is noise)	AGENTS.md:731
CI blocking gate	benchmark-regression job runs a same-runner base-vs-head comparison on every PR and every push to main, and is a blocking gate before Semantic Release, not advisory	docs/CI_PIPELINE.md:23,42-43

If a candidate is correct but slower: revert it and record the attempt in docs/PAPER.md so no future agent (human or model) retries the losing idea — see tensor-grep-research-methodology.

---

Part 6 — How to add a test

Step 1 — pick the directory (what each one means here)

Directory	What lives there	Run cost
tests/unit/ (**re-run `ls tests/unit/*.py	wc -l`** -- 291 on 2026-07-27; do not cite the stamp)	Fast, isolated; heavy CliRunner usage (400+ call sites) — good for flag-parsing/formatter/validator logic, not sufficient alone for routing changes (Part 1 point 3)