| name | tensor-grep-large-repo-scale-campaign |
| description | Use when tg hangs, stalls, or runs for minutes on a large/unscoped repo; when `--deadline` "seems ignored" and a symbol query still overruns its budget; when working task #52 (CLOSED, end-to-end deadline ineffective on a ~1800-file TS repo) or #390 (CLOSED, the daemon-path deadline gap) for historical context, caller-scan / build_repo_map latency, the unscoped-`tg search` hang, the generic >1500-file implicit-root fast-refuse (A9/#702), or the exit-2 partial-result semantics. The decision-gated campaign to finish agent-native SCALE HONESTY: tg must never hang and never silently lie (return an empty/partial result as if it were complete) on a customer-scale repo. Gives the reproduce -> phase-instrument (cProfile) -> ranked solution menu -> fail-closed build -> change-control promotion runbook with exact commands, expected numbers, and branch-on-mismatch forks. Verified against v1.110.14 (2026-08-11) on 2026-07-24 (#400/#413/#428/#702 fully shipped; #52 and #390 both CLOSED; the exit-code contract is exit-2-regardless-of-found, per #401; every S1 `file:line` anchor plus the `CALLER_SCAN_FILE_CEILING`/`DEFAULT_AGENT_REPO_MAP_LIMIT` unification re-verified against `origin/main`, correcting drift from the Java/C#/PHP language-support campaign, #719-#726); `tg find` re-verified in the same pass (previously spot-checked 2026-07-16 at v1.78.1).
|
tensor-grep — Large-Repo Scale-Honesty Campaign
A decision-gated runbook for the project's hardest live problem: making tg
bounded and honest at customer scale. Two failure shapes, one contract:
- Never hang. Every scan path has a wall-clock bound; when the bound trips it
returns a partial result, never spins forever.
- Never silently lie. A truncated/partial result is flagged (
result_incomplete /
partial in JSON, an exit-2 signal for symbol commands, a stderr warning) so an
agent can tell "complete zero" from "gave up early." A quietly-empty result that
reads as "no matches / no callers / dead code" is the one bug a context tool cannot
ship.
This skill is the campaign map: what already shipped, what is still open, the exact
commands + expected numbers at each gate, the wrong paths that are fenced off, and how
promotion routes through change-control. You ship nothing user-visible from this
skill without beating a measurable gate and a conscious flag-flip (Phase 4).
Why this is the live frontier (receipts)
--deadline is now a hard end-to-end wall-clock bound on the graph commands
(task #52, CLOSED by PR #478 / 67f9779, shipped v1.54.3).
Receipt (2026-07-05, on a real ~1884-file TypeScript repo, pre-#396):
tg callers QueryEngine --deadline 10 took ~25 s (not ~10 s), and a direct
deadline_seconds=8 call took >90 s. Root cause: build_symbol_callers_from_map
(the caller-scan) re-reads/re-parses candidate files in an any()-loop per
definition, so "each stage is bounded" did not make the pipeline bounded. #478
closed the residual gap a verify-plan-against-code pass found after #396/#440: four
loops still had no wall-clock bound -- (A) _iter_repo_files' file-tree walk (count-only
bound, no deadline), (B) _relevant_tests_for_symbol's two unguarded any() loops (the
dominant cause on a high-fan-out symbol like "main", confirmed a 3x-recurring P0), (C)
build_symbol_impact_from_map had no deadline parameter at all, and (D) the string_refs
second pass in build_symbol_refs_from_map. All four now fold into the existing
_DeadlineBreakFlag/partial/deadline_limit machinery, guarded if deadline_monotonic is not None so an undeadlined caller sees byte-identical behavior. See S2 for what this
does and does not close (the #390 daemon-path gap is explicitly separate).
- #396 (v1.39.1) shipped a caller-scan re-parse cache +
Path.resolve() memoization
— the code comment measures ~18 s of ~22 s wall time was redundant resolve()
churn (src/tensor_grep/cli/repo_map.py:76-80), claims 7.9x on central symbols.
So the pre-#396 receipt numbers are STALE — Phase 0 re-measures at HEAD.
- The unscoped-
tg search hang was real and is now fixed, not in-flight. AGENTS.md
(grep -n "hangs ~600 s then errors" AGENTS.md; was :378, now :480 — drifted +102 lines this
pass, same-size drift as the other AGENTS.md cites below, consistent with one insertion earlier
in that file)
still narrates the pre-fix symptom ("hangs ~600 s then errors" because tg's own index dirs +
a vendored tree were not auto-excluded) — that doc lags. The fix, , shipped in
() and was hardened further by (v1.42.0) and ; see
§1 below. Do not present this as an open bug or an unmerged PR.
0. When to use this skill — and when to use a sibling instead
Use this skill when the task is bounding/honesty at scale: a hang, a --deadline
overrun, task #52, the #390 daemon gap, the exit-code partial contract, or a
latency profile of a graph command on a big repo.
| If you actually need to… | Use this sibling instead |
|---|
| A hang/slowness whose CAUSE is unknown — systematic bisection first | tensor-grep-debugging-playbook (then return here) |
| Single-file / small-repo micro-latency, or make a number claim-quality | tensor-grep-benchmark-and-proof-toolkit |
| The front door / routing / registration / backend contract | tensor-grep-architecture-contract |
| Register a new flag/command (2 front doors, 4 sites) | tensor-grep-config-and-flags |
| Merge/release/experimental-flag gates + the incidents behind them | tensor-grep-change-control |
| Learn a settled battle (FFI reverts, dep caps, mock-vs-real, golden-sensitivity) | tensor-grep-failure-archaeology |
| Build/run the toolchain (uv, maturin, cargo) | tensor-grep-build-and-env |
| Update README/AGENTS/docs after shipping | tensor-grep-docs-and-writing |
| Position externally (never "faster grep") | tensor-grep-release-and-positioning |
This skill never routes around change-control. It produces the evidence; the
flip is a tensor-grep-change-control decision (Phase 4).
1. What already shipped (verify each before you build on it)
The P0-6 "moat" deadline program (#384-#401) + the now-shipped unscoped-hang fix (#400,
#413, #428, #702). Every anchor below is at HEAD on 2026-07-24 (v1.96.0); re-verify line numbers,
they drift — main.py is 17032 lines and repo_map.py is 18752 lines as of this pass, do not
trust an older total. Both files grew substantially (repo_map.py by ~1850 lines) across the
Java/C#/PHP language-support campaign (#719-#726) landed since the last full citation pass at
v1.93.2 — several anchors below drifted by 1000+ lines and have been re-pinned in this pass.
| Ship | What it does | Verify |
|---|
| #384-#388 deadline threading | deadline_seconds -> _deadline_monotonic_from_seconds -> build_repo_map(deadline_monotonic=...); converted once to an absolute time.monotonic() stamp so the scan can self-bound and return partial. | grep -n deadline_seconds src/tensor_grep/cli/repo_map.py (e.g. _deadline_monotonic_from_seconds at grep -n "^def _deadline_monotonic_from_seconds" src/tensor_grep/cli/repo_map.py (:768 as of 2026-08-14), was :752; build_repo_map's own deadline_monotonic param at grep -n "^def build_repo_map" src/tensor_grep/cli/repo_map.py (def :7669 as of 2026-08-14), was :7427 — no longer 1-line churn; re-grep rather than trust either number) |
#389/#393 graph-command CLI --deadline | tg callers / refs / impact / blast-radius gained --deadline FLOAT; #393 bounds the caller-scan traversal itself. | tg callers --help shows --deadline FLOAT RANGE; source: grep -n "^def callers|^def refs|^def impact|^def blast_radius" src/tensor_grep/cli/main.py → callers:12467, refs:12360, impact:12155, blast_radius:12853 (was main.py:11629/11645/11440/12083 — all four drifted +715/+715/+715/+770 lines; re-grep, do not trust these numbers either) |
#394 payload result_incomplete | Truncation stamped at the payload layer so MCP/_json consumers see it, not just the CLI. | grep -n result_incomplete src/tensor_grep/cli/repo_map.py |
#395 tg inventory --deadline | inventory walk wall-clock bounded. | tg inventory --help shows --deadline; source grep -n "^def inventory" src/tensor_grep/cli/main.py → :8823 (was :8414, drift +409), build_inventory(..., deadline_seconds=...) at (was , drift +2) |
| tg find (v1.77.0, #189, main.py:4728-4837, was :4574-4676, drift +147 lines) — whole-repo hybrid NL search, a NEW command that reuses this campaign's bounding shape from day one rather than retrofitting it later | Takes --deadline/--max-repo-files plus an internal corpus-wide chunk cap (_FIND_CORPUS_CHUNK_CAP); a truncated scan sets result_incomplete=true and exits 2 (found-but-truncated prints results THEN exits 2 -- same "truncation trumps found" rule as #401 below, not the found-exits-0 shape #399 walked back). Unlike tg search, tg find does NOT get the instant vendored/workspace-root refusal (#400) -- it always attempts the bounded scan rather than refusing outright, because ranking the whole repo (not raw-text matching it) is the command's entire point. | tensor-grep-run-and-operate §11c has the full exit-contract prose; grep -n "^def find" src/tensor_grep/cli/main.py → :4721 |
| #702 (v1.92.3, A9) -- a 4th unscoped front-door fix alongside #400/#413/#428, closing a gap those three never covered | bootstrap._run_rg_passthrough (bootstrap.py:1337, was :1088, drift +249) -- the plain flag-less search front door that runs before main.py's Typer app is ever reached -- had NO walk ceiling at all until this shipped; the three existing main.py-side refusal guards (#400/#413) never ran for this path. Fixed by adding bootstrap._search_paths_include_oversized_implicit_root (bootstrap.py:1053, was :804, drift +249, the same-size shift as _run_rg_passthrough above -- consistent with one insertion earlier in the file), gated on paths_defaulted, sharing the SAME IMPLICIT_SEARCH_WALK_FILE_CEILING = 1500 constant (io/scan_limits.py:106, re-verified exact this pass, re-exported via io/directory_scanner.py) the other 3 doors already used -- natively reproduced, not a WSL artifact. An over-ceiling implicit root now refuses in ~1.7s instead of silently walking up to the 60s TG_RG_TIMEOUT_SECONDS backstop. | grep -n _search_paths_include_oversized_implicit_root src/tensor_grep/cli/bootstrap.py; grep -n IMPLICIT_SEARCH_WALK_FILE_CEILING src/tensor_grep/io/scan_limits.py |
Merge/release state to stamp every session: #400/#413/#428/#702 are all in the installed
binary as of v1.96.0 — this is not a source-only or in-flight fix. If a future session finds
a NEW in-flight PR referenced by this skill, do not describe it as shipped until
git log --oneline origin/main | head shows a chore(release) commit above it (see the
project's merge-gate guardrail: an open PR is guidance, not a receipt, until it lands on main).
2. Still open (the campaign's actual work)
- #52 — end-to-end deadline ineffective on a large TS repo — CLOSED by PR #478
(
67f9779, shipped v1.54.3). See the shipped-table row in S1 for the four loops it
bounded (A: file-tree walk, B: _relevant_tests_for_symbol, C: build_symbol_impact_from_map,
D: string_refs second pass). This supersedes the likely-closed framing this skill
previously carried after #396/#440/the CALLER_SCAN_FILE_CEILING chokepoint alone — those
three narrowed the gap but a verify-plan-against-code pass still found the four loops above
live and unbounded; #478 is the commit that actually closes it. Still do a one-time fresh
Phase-0 re-measure on a real large TS repo before citing a specific wall-clock number in a
benchmark claim or PR -- "the mechanism is closed" and "I have re-confirmed the number on my
reference repo" are different claims; S3 Phase 0 gives the exact command.
- #390 — daemon-path deadline gap — CLOSED (task #203, shipped #647/#652/#653, v1.81.2-.4).
This was framed OPEN in earlier passes of this skill — that framing is now STALE and risks a
future session re-implementing already-shipped code. Verify directly rather than trusting
either framing:
session_store.py (the daemon's request-dispatch module) now threads a real
deadline_monotonic value into its build_symbol_impact_from_map (session_store.py:1449, was
:1276, drift +173) and
build_symbol_callers_from_map (session_store.py:1490, was :1317, drift +173) call sites — the code comment at
session_store.py:1484-1488 (was :1311-1315, drift +173) states explicitly: "the exact #390 daemon-path shape this task
(#203) closes." The same threading was extended to refs/blast-radius/file_importers
(search session_store.py for WARM_DAEMON_DEFAULT_DEADLINE_SECONDS to see every site). A
daemon-served graph query on a cached session repo_map is now bounded the same way a fresh
CLI invocation is — the #478 design doc's explicit "session/daemon territory out of scope" carve-out
from Candidate (c) below has since been executed as its own follow-up, not left open.
Candidate (c) below is retained as the historical solution-menu entry that predicted this fix's
shape, not as an open task — do not re-propose closing #390 as new work.
- The native-walk bound reuses ,
which now defaults to (, confirmed exact this pass, was 600 s).
(was , drift +102 -- )
still narrates the pre-#400 "600 s" symptom — that doc lags; the resolver is the source of
truth.
3. The phased runbook (decision-gated)
Run in order. Each gate states the expected observation and where to branch.
PowerShell is the dev-box shell; uv run is cross-platform. Use uv run --no-sync
so a bare uv run does not re-sync away the [dev] tree (tensor-grep-build-and-env).
Phase 0 — Reproduce the baseline on a REAL large repo (NEVER skip)
You cannot claim "bounded" without the unbounded number, and you cannot claim a fix
without the pre-fix number. Use a public large TS repo so this is reproducible
(never a private customer path).
# thousands of .ts files, deep import graph — a public customer-scale proxy
# (any large TS repo works; substitute one with >~1500 .ts files if you prefer)
git clone --depth 1 https://github.com/microsoft/TypeScript C:\tmp\ts-ref
$repo = "C:\tmp\ts-ref\src"
# Wall-clock the bounded command against its own budget (source or the installed >=v1.40.4
# binary both have #400/#440; either works, source keeps you at HEAD for a just-landed fix):
Measure-Command { uv run --no-sync python -m tensor_grep callers $repo Node --deadline 10 --json | Out-Null }
Expected + gate:
- GATE 0 (the #52 test): total elapsed should be <= deadline + ~10% (i.e.
~11 s for
--deadline 10). Read the JSON: partial/result_incomplete must be
true iff the scan was actually truncated, and a truncated result must carry a
non-empty incomplete_reason/caveat.
- If elapsed >> deadline (e.g. 25 s for a 10 s budget) -> a NEW regression, not #52
reopened, and not #390 either -- #52 is CLOSED at HEAD (#478/
67f9779, S1/§2), built
specifically to prevent exactly this shape across all four loops (A/B/C/D), and #390 (the
daemon-path gap) is ALSO CLOSED (task #203, S1/§2 above) -- a session-served query is bounded
the same way a fresh CLI one is now, so do not reach for "still-open #390" as the explanation.
A live overrun here means a new unbounded loop was introduced since #478/#203; if you can
reproduce it specifically on a session-served/daemon query and NOT on a fresh CLI one, that is a
fresh, narrower finding (a regression in the #203 threading), not a reopening of #390 itself.
Proceed to Phase 1 to find where the budget leaks.
- If elapsed is bounded AND truncation is honestly flagged -> matches the current
expectation (§2: #52 closed). Do NOT just declare victory: re-run on a second
large repo and a central symbol (highest fan-in), confirm the exit code matches
docs/CONTRACTS.md:157 (was :156, drift +1; exit 2 fires on ANY truncation, found or not — §5), then route
promotion of the "closed" claim through Phase 4 / change-control.
- If it HANGS (no return, no error) on an unscoped
tg search on a root with a
vendored dir -> this is a NEW bug, not the old #400 shape: #400's instant refusal
(_should_refuse_unbounded_vendored_root_scan, exit 2, <1s) plus the native-walk
deadline are shipped in the installed tg binary (>=v1.40.4) as well as source, so a
hang here on EITHER means the fix regressed -> tensor-grep-debugging-playbook.
Phase 1 — Phase-instrument the ACTUAL slow command (do NOT guess)
Profile the command Phase 0 flagged, on the same repo. cProfile is the oracle here
(the graph commands do not all expose --profile; only context-render and
blast-radius-render do — verified against tg --help). This invocation is verified
to run and print stats:
# tottime = internal (self) time -> finds the true hot function.
# tg's own output prints first; the profile table is APPENDED at the end -> tail it.
uv run --no-sync python -m cProfile -s tottime -m tensor_grep callers $repo Node --deadline 30 2>&1 | Select-Object -Last 30
Expected + gate (verified shape on a real repo): the top tottime rows are the
per-file parse — on Python targets {built-in method builtins.compile}, ast.walk,
and repo_map.py:2127(_python_imports_and_symbols) (was :1921, drift +206); on TS targets the analog is the
tree-sitter parse via repo_map.py:2271(_typescript_parser) (was :2065, drift +206) — invoked many times,
because build_symbol_callers_from_map re-parses candidate files in its any()-loop
(repo_map.py:3639 documents the "N definitions -> N re-reads/re-parses" hazard, was :3407-3409,
drift +231 — grep -n "N definitions means N re-reads" src/tensor_grep/cli/repo_map.py).
- GATE 1a — caller-scan re-parse dominates (many parse calls, high
ncalls on the
parse/resolve functions): the leak is the re-parse loop, not the one-shot map
build -> Solution menu candidate (a) or (b) in Phase 2.
- GATE 1b — a single
build_repo_map pass dominates (parse called ~once per file,
not per definition): the map build itself is the cost -> the fix is bounding/caching
build_repo_map, which #384-#388 already partly did via deadline_monotonic;
re-measure whether the deadline is honored INSIDE that pass -> different branch --
a NEW gap, since #390 (the daemon-path deadline gap) is CLOSED (S1/§2 above), not an
open "#390 gap" to attribute this to.
- GATE 1c —
resolve() / path work dominates despite #396: the cache is being
defeated (e.g. an uncached resolve() before the cache lookup — repo_map.py:2712
warns about exactly this, was :2506-2507, drift +206 — grep -n "this resolve() runs BEFORE the re_export_cache lookup" src/tensor_grep/cli/repo_map.py) -> candidate (b), fix the cache ordering.
Redundancy measurement before you design a cache (candidate b obligation):
monkeypatch the parse function with a collections.Counter keyed by path and count
how many times each file is re-parsed for one callers call. This is the technique
that overturned a code-review guess in PR #345 (see fenced paths). Numbers first,
then design the key.
Phase 2 — Solution menu (RANKED), each with a proof obligation
Pick top-down. Each candidate carries an obligation you must verify/measure, not
assume, before building.
Candidate (a) — bound the caller-scan re-parse loop (preferred)
Apply the same per-file deadline check #400 used for the native search walk to the
caller-scan any()-loop: check native_walk_deadline_exceeded (or an equivalent
deadline_monotonic) once per file, and on expiry break to a partial.
- Obligation: the partial must be fail-closed and flagged — set
result_incomplete = True + a concrete incomplete_reason, and let
_emit_symbol_command_result (grep -n "^def _emit_symbol_command_result" src/tensor_grep/cli/main.py -- :11722 as of 2026-08-14, was :11762 at the 2026-07-24 pass, was :11298 before that) apply the exit contract: per §5, ANY
truncated/partial result exits 2, regardless of whether it's empty or non-empty
(#401 — do not build toward the old #399 "found-but-capped exits 0" shape). Mirror
#400's shape (main.py:8137-8270, was :7979-7992): break, never return a clean empty.
- Why preferred: it directly closes #52 with a mechanism already proven in-tree; low
blast radius; deterministic. This is what #478 (
67f9779) actually shipped -- the same
per-file-deadline-check shape, applied to all four residual loops (S1/S2). This menu entry
stays useful as the template for the next #52-shaped finding, not just a historical proposal.
Candidate (b) — extend the #396 caching to the residual re-parse
If Phase 1 shows re-parses that #396's caches miss, widen coverage.
- Obligation — cache-key correctness: a repo-map cache MUST key on file
mtime+size (use the existing
_mtime_aware_cache (grep -n "^def _mtime_aware_cache" src/tensor_grep/cli/repo_map.py -- :108 as of 2026-08-14, was :100), not a plain
lru_cache keyed on path alone — a plain cache returns stale results in the
long-lived daemon (the code comments at repo_map.py:30-34 + the
_MTIME_CACHE_CLEAR_REGISTRY declaration (grep -n "^_MTIME_CACHE_CLEAR_REGISTRY" src/tensor_grep/cli/repo_map.py -- :64 as of 2026-08-14, was :55) document this trap
and the sweep). Prove the redundancy with the
Counter-monkeypatch (Phase 1) BEFORE adding the cache, and prove correctness with a
mutate-file-then-re-query test.
- Caching bounds the common case but does not bound a pathological single file —
ship it WITH candidate (a), not instead of it.
Candidate (c) — close the #390 daemon-path deadline gap — SHIPPED (task #203, #647/#652/#653, v1.81.2-.4); retained here as historical record, not an open candidate
This entry correctly predicted the shape of the eventual fix and is kept for that reason, but the
work itself is DONE — do not re-propose it as new. What actually shipped: session_store.py's
daemon call sites (build_symbol_impact_from_map at session_store.py:1449, was :1276, drift +173,
build_symbol_callers_from_map at session_store.py:1490, was :1317, drift +173, plus refs/blast-radius/
file_importers siblings) now pass a real deadline_monotonic value
(WARM_DAEMON_DEFAULT_DEADLINE_SECONDS-derived) into every builder call — exactly the call-site
wiring this candidate originally scoped as the remaining work after #478's builder-side prerequisite.
See S2 above for the closing citation (session_store.py:1484-1488's own code comment, was :1311-1315, names task
#203 as the closer). If a NEW daemon-served command is added in the future, use this shipped pattern
as the template — thread deadline_monotonic at the call site, mirroring the sibling commands in
the same file, rather than assuming the daemon path is unbounded by default going forward.
Candidate (d) — replace the regex/slow TS parse with tree-sitter (HIGHEST RISK — fenced)
Only if (a)+(b)+(c) leave the parse itself as the irreducible hotspot.
- Obligation (mandatory, before any swap): golden parity across the parser corpus.
A parser change alters symbol/import extraction for every downstream command; prove
byte-identical (or explicitly-diffed-and-accepted) output on the AST parity corpus
first (
benchmarks/run_ast_parity_check.py). Do this in a fenced branch; do not mix
it with a bounding fix.
Phase 3 — Build behind the fail-closed contract + measure the gate
- Write the failing test first (
tests/unit/test_repo_map_targets.py,
tests/unit/test_cli_modes.py are the patterns #400 used). TDD, then the smallest fix.
- Honor the Backend Fail-Closed / partial contract (§4): a bound that trips
produces a flagged partial, never a silent empty, never a raw crash.
- Re-run Phase 0 + Phase 1 on the reference repo after the fix.
- A6 — anti-hang test discipline (a hang-class regression test can itself hang): when the
red test is "this used to hang/overrun," wrap it so a still-broken fix fails FAST instead of
wedging the test run. (1) layer an OUTER shell timeout (
--kill-after=Ns) around an INNER
per-test timeout implemented via a thread/watchdog, not signal (signal-based timeouts are a
no-op on Windows and under the GIL for CPU-bound native calls — this repo runs CI and dev on
Windows). (2) either land the fix before the red test lands, or land the red test already
wrapped in both timeouts. (3) never leave an unbounded loop/subprocess-spawn/backtracking
regex in the TEST ITSELF without its own bound — the test must fail the same way the product
bug does (bounded, flagged), not hang the CI runner. (4) distinguish "slow but protected" from
"hung" by EXIT CODE (124/137 = killed) not elapsed wall-clock, which is noisy on a shared
runner. (5) apply the same two-layer timeout in CI as locally — a hang that only reproduces in
CI because the outer timeout was dev-box-only is the worst kind to debug.
Promotion gate (all must hold, measured on the reference repo, same symbol):
| Metric | Requirement |
|---|
| Wall-clock vs budget | elapsed <= --deadline + ~10% (the #52 bar) |
| Truncation honesty | partial/result_incomplete set iff truncated; non-empty incomplete_reason |
| Exit code | matches docs/CONTRACTS.md:157 (was :156, drift +1) (0 complete / 1 complete-not-found / 2 incomplete — regardless of whether anything was found, per #401) |
| No regression | benchmark-regression CI gate green (benchmarks/check_regression.py); correctness identical on the AST parity corpus |
| Real binary | dogfood via scripts/dogfood/ on the REAL artifact, not CliRunner |
- GATE 3: if the wall-clock still overruns the budget OR a partial isn't flagged,
do not ship — the honesty contract is the whole point. Iterate or record the
negative result.
Phase 4 — Promote through change-control (never here)
This skill produces evidence; tensor-grep-change-control owns the flip.
- One release-bearing PR per tick (respect the push-race / one-merge-per-tick rule,
tensor-grep-release-and-positioning).
- Attach Phase 3 evidence (before/after wall-clock table + exit-code proof + parity).
- Re-dogfood on the REAL large repo before declaring the contract done — this is
the #399 lesson: #398 shipped an exit-code rule that had to be walked back one release
later because the real-repo behavior (every large-repo query exiting 2) was wrong.
- Update
docs/CONTRACTS.md / AGENTS.md / SESSION_HANDOFF.md if the contract
changed (tensor-grep-docs-and-writing).
4. Fenced-off wrong paths (do NOT do these)
| Forbidden | Why | Do instead |
|---|
| Raise the default timeout to "fix" a hang | Masks, doesn't bound. A bigger number still hangs on a bigger repo; it just moves the wall. | Add a real per-file wall-clock bound that returns a flagged partial (#400 pattern). |
| Return a silent empty / clean 0-result on timeout | Violates the Backend Fail-Closed Contract (AGENTS.md, grep -n "^## Backend Fail-Closed Contract" AGENTS.md → :2090; was :438-440, drift +1233 — a cross-file citation this far stale is worth a grep, not a re-stamped number). A partial that reads as "no matches / no callers / dead code" is the exact bug this campaign exists to kill. | result_incomplete = True + incomplete_reason + stderr warning + the exit-2-regardless-of-found contract (§5). |
| Assume "each stage bounded => pipeline bounded" | The #52 lesson: build_repo_map and caller-scan were each bounded in isolation, yet the end-to-end command overran because the caller-scan re-parses ~all files. | Measure the WHOLE command wall-clock (Phase 0), then profile it (Phase 1). |
| Guess the hotspot from code review | PR #345 receipt: a review council guessed a 3.6%-of-runtime path; the live profile found the real one (_module_aliases_for_path called ~1.4M times) -> lru_cache+frozenset cut the run 61.7 s -> 12.8 s (4.8x). | Profile the ACTUAL slow command at scale (Phase 1); the profiler is the oracle. |
| "Fix" a golden to match your dev box | #363 receipt: a "stale" golden was edited to include rg submatches, then reverted — the golden is backend-sensitive and CI has no rg. CI is the oracle. | Read the failing CI job's -/+ diff first; force a deterministic backend in the test. tensor-grep-failure-archaeology. |
A plain lru_cache keyed on path in the daemon | Returns stale results across a long-lived session when a file changes (repo_map.py:30-34, 55). | _mtime_aware_cache (mtime+size in key) + register its cache_clear in the sweep registry. |
|
5. The contract you are defending (exit codes + fail-closed)
Symbol-command exit codes are a 3-state agent contract (docs/CONTRACTS.md:157, was :156, drift +1,
enforced at main.py:11639-11832, was :11054-11099, drift +708 — grep -n "^def _emit_symbol_command_result" src/tensor_grep/cli/main.py). This is the current, FINAL shape — #401 reverted #399,
so exit 2 fires on ANY truncated/partial result, whether or not it found something:
| Exit | Meaning | Agent action |
|---|
0 | complete result — the scan was NOT truncated | trust the findings as exhaustive |
1 | genuine not-found on a complete scan | the symbol truly is absent |
2 | INCOMPLETE — truncated by --deadline (partial: true) or a --max-repo-files/scan cap (result_incomplete: true) — REGARDLESS of whether anything was found | do NOT treat as exhaustive even if it has results (a truncated caller-set is not a safe blast-radius); parse the JSON and retry with a larger budget or a narrower scope |
Do not describe a "found-but-scan-capped result exits 0" behavior anywhere — that was #399,
walked back by #401 after a unanimous design council concluded truncation must always trump
"found something," so a caller/blast-radius consumer can never mistake a capped result for a
complete one. The native-walk bound (#400) is the search-side analog: on expiry it sets
all_results.result_incomplete = True + incomplete_reason, writes a stderr warning,
and breaks (main.py:8137-8270, was :7979-7992) — a flagged partial, never a silent empty. Any new
bound you add MUST follow this shape (see tensor-grep-architecture-contract for the
full BackendExecutionError contract).
tg find (v1.77.0, #189) followed this shape from its first shipped version — a genuine
example of a new command adopting this campaign's contract instead of retrofitting it: any
--deadline/--max-repo-files/internal chunk-cap truncation sets result_incomplete=true and
exits 2, whether or not ranked matches were found. It is bounded but NOT refusal-gated the way
tg search is (#400's instant vendored/workspace-root refusal does not apply) — see the §1
shipped table.
6. When NOT to use this skill
- A hang/slowness whose cause you don't yet know — do the systematic bisection in
tensor-grep-debugging-playbook first; come back here once it's a scale/bounding
problem.
- Single-file or small-repo micro-latency, or turning one measurement into a
claim-quality number ->
tensor-grep-benchmark-and-proof-toolkit (noise-floor,
fair-baseline rules).
- Registering the flag/command mechanics for a new bound ->
tensor-grep-config-and-flags
(2 front doors, 4 sites) + tensor-grep-architecture-contract.
- The merge/release flip itself ->
tensor-grep-change-control.
Operator practice (dated, CEO-enforced 2026-07)
- Profile-at-scale, don't council-guess a hotspot (PR #345). For a latency fix the
profiler is the oracle; a diff-review council correctly killed a wrong theory but
then guessed the wrong hot path.
- Dogfood the REAL large repo before declaring a contract done (#399 walked back
#398 one release later on real-repo behavior). Fixture/self-repo tests give false green
for scale honesty.
- Model tiering for any fan-out: set the model explicitly per seat — haiku for
scans/discovery, sonnet for the bulk (profiling readers, fix implementers, verifiers),
opus for planning/synthesis/hard debugging. State the split before the fan-out runs.
- Windows FS reality: worktree "tests pass" is a hypothesis — re-run in a real venv;
hammer concurrency/timeout tests 15-20x (Linux-reasoning agents miss Windows FS
semantics; delete-pending
PermissionError, os.replace WinError5).
- Git hygiene: relocate uncommitted work with
git checkout -b X origin/main (carries
it), never a bare git stash pop; never broad git checkout -- . with edits you need.
Provenance and maintenance
Every claim above is verifiable from the repo at HEAD on 2026-07-08 (v1.49.3), with the
#52/#390/#478 facts specifically re-verified against HEAD on 2026-07-14 (v1.75.4), the
tg find §1 row + §5 addendum verified 2026-07-16 (v1.78.1), and every file:line anchor
in this skill (S1 shipped table, §5 contract, Phase 1/2 citations) re-verified against HEAD on
2026-07-24 (v1.96.0) -- that last pass found and corrected drift of 1000+ lines on several
repo_map.py/main.py anchors, accumulated silently across the Java/C#/PHP language-support
campaign (#719-#726) which grew repo_map.py by ~1850 lines; it also found main.py:7894 /
main.py:8263-8384 had already drifted BEFORE the v1.93.2 stamp, i.e. a prior provenance bump did
not fully re-verify every citation -- re-verify everything independently before trusting it rather
than assuming a recent-looking stamp means every anchor was checked. The
unscoped-hang fix #400 = e7f18b7 is fully shipped (v1.40.4, bb14abe,
plus follow-ons #413/#428) — do not re-check "is #400 released yet" as if it were still in
question. Re-run these when a claim may have drifted; date-stamp any change.
- Deadline threading:
grep -n deadline_seconds src/tensor_grep/cli/repo_map.py | head
and tg callers --help | grep -i deadline.
- #52 closed check:
git show --stat 67f9779 (PR #478) confirms the four-loop fix landed;
re-run Phase 0 on a large repo to get a fresh wall-clock number before citing one in a claim;
also grep -n "CALLER_SCAN_FILE_CEILING\|DEFAULT_AGENT_REPO_MAP_LIMIT" src/tensor_grep/cli/repo_map.py
to confirm the current values — do NOT describe them as deliberately decoupled at
"2000 map limit / 512 caller-scan ceiling"; that framing is STALE. Backlog #57
(2026-07-09, repo_map.py:167-177) UNIFIED them: CALLER_SCAN_FILE_CEILING was raised
512 -> 2000 (matching DEFAULT_AGENT_REPO_MAP_LIMIT) now that #478's per-loop
--deadline bound removed the unbounded-hang risk that originally kept the caller-scan
ceiling frozen below the map default. 2000 is a deliberate knee, not a removal of the
cap — it stays a hard backstop for a --max-repo-files-raised mega-repo and the
flag-less default path (repo_map.py:167-176 documents the full rationale).
- #390 daemon gap — CLOSED, verify the daemon call sites directly, not just the builder
signatures:
grep -n "deadline_monotonic" src/tensor_grep/cli/session_store.py should show
WARM_DAEMON_DEFAULT_DEADLINE_SECONDS-derived values threaded into build_symbol_impact_from_map
(session_store.py:1449, was :1276, drift +173), build_symbol_callers_from_map (session_store.py:1490, was :1317, drift +173), and the
refs/blast-radius/file_importers siblings — read the code comment directly above each call
site, which names task #203 as the closing fix.
- Native-walk bound + default budget:
grep -n "native_walk_deadline\|compute_native_walk_deadline" src/tensor_grep/backends/cpu_backend.py;
grep -n TG_RG_TIMEOUT_SECONDS src/tensor_grep/cli/subprocess_policy.py (default 60 s, confirmed exact at subprocess_policy.py:75 this pass, not the
pre-#400 600 s AGENTS.md:480, was :378, drift +102, still narrates as the historical symptom).
- Vendored-root refusal (two front doors): .
Open / candidate (not settled — do not present as done):
- #390 (daemon gap) is CLOSED, not open — task #203 (#647/#652/#653, v1.81.2-.4) threaded
deadline_monotonic through every session_store.py daemon call site that #478 had explicitly
left out of scope. Do not re-list this as open work; if you find a NEW daemon-served command
missing the same threading, that is a fresh, narrower finding, not a reopening of #390 itself.
- #52 is CLOSED — PR #478 (
67f9779, shipped v1.54.3) bounded the four residual unbounded
loops (S1/S2) that #396/#440/the CALLER_SCAN_FILE_CEILING chokepoint alone left live. Still
do a one-time fresh Phase-0 re-measurement on a real large TS repo before citing a specific
wall-clock number in a new claim (the mechanism being closed and a specific number being
re-confirmed are different statements) -- do not re-open "#52 closed" as a question, only
re-confirm the number.
- The parser swap (candidate d) remains an unbuilt candidate, gated on AST parity and may
never be worth it. The caller-scan re-parse bound (candidate a) shipped as #478 (see S3 Phase 2
candidate (a) note) -- it is no longer an open candidate for the #52 shape specifically, though
the pattern remains the template for the next similar finding.
- Doc-of-record narrative lags reality:
AGENTS.md:480 (was :378, drift +102 — grep -n "hangs ~600 s then errors" AGENTS.md; still narrates the pre-#400 "600 s"
hang symptom as if unfixed), SESSION_HANDOFF.md — trust the code + docs/CONTRACTS.md,
note the doc lags.
- Merge-gate discipline: if a future in-flight PR (not yet on
origin/main) looks relevant
to this campaign, capture the PATTERN as guidance only — do not write "#NNN shipped" until
git log --oneline origin/main | head shows a chore(release) commit above it.
Drift-gate pass (against local tg 1.101.24 / installed; PyPI 1.101.27): nearly every numeric
file:line citation in this skill had drifted again since the 2026-07-24 (v1.96.0) re-verification
above — the SAME class of drift that pass itself found and fixed, recurring on the identical
anchors. Representative drifts, all confirmed by re-deriving the symbol's current definition line
(not by incrementing the old number): main.py's callers/refs/impact command defs +715 lines,
blast_radius +770; _run_rg_passthrough and _search_paths_include_oversized_implicit_root in
bootstrap.py +249 each; _should_refuse_unbounded_vendored_root_scan +111;
_emit_symbol_command_result +708; _module_aliases_for_path +512; tg find's def find +147;
repo_map.py's _python_imports_and_symbols/_typescript_parser/the re-parse-hazard comment/the
uncached-resolve() comment all +206 to +231 (one shared insertion window); session_store.py's
three #390 call-site citations +173 each; and four separate AGENTS.md cites (the 600s-hang
symptom, the callers-blind-spot note in the sibling tensor-grep-backlog-campaign skill, and the
Backend Fail-Closed Contract heading) drifted +102 to +1233 lines. Two constants held EXACT
(UNBOUNDED_VENDORED_ROOT_DIR_NAMES and IMPLICIT_SEARCH_WALK_FILE_CEILING, both in
io/scan_limits.py) and two were substantively correct (subprocess_policy.py:75's 60s default,
and the repo_map.py:1220-1221 per-bucket deadline check) — proving the drift is not uniform
file-wide churn but concentrated wherever an unrelated feature (the Java/C#/PHP language campaign,
further AGENTS.md growth) inserted lines above a citation. Per the "cite the SYMBOL, not the line"
rule this section already carries, every citation above was rewritten as a grep -n "<symbol-or- distinctive-phrase>" <file> instruction with the was→now drift kept as the receipt, rather than
re-stamped with a fifth hardcoded number. This pass ALSO found and corrected two live self-
contradictions this skill's own drift had produced: Phase 0's GATE 0 branch and Phase 1's GATE 1b
branch both still told a reader to attribute an overrun to "the still-open #390 daemon-path gap"
several sections after this same skill's own S1/§2 documents #390 as CLOSED (task #203) — a stale
number is silent, but a stale CAUSAL claim actively misdirects debugging. Do not re-stamp any of the
citations above with a sixth hardcoded number; re-run the grep.