| name | tensor-grep-config-and-flags |
| description | Use when adding, changing, or auditing a tg environment variable, CLI flag, or provider mode (native/lsp/hybrid); when a search flag silently leaks to ripgrep or a command misroutes; when deciding whether a config axis (GPU, LSP, classify, semantic) is production or EXPERIMENTAL default-OFF; when adding a new `SearchConfig` field and needing to know whether it must be forwarded/refused/KNOWN_GAP'd for native delegation; or before registering a new `tg search --flag` or `tg COMMAND` (including `tg inventory`'s `--max-repo-files`/`--deadline`). Catalogs the load-bearing TG_*/TENSOR_GREP_* env vars (routing, timeouts, GPU, classify, session, MCP security, LSP) with their default and guard, the 2-front-door / 4-site registration checklist, and the native-delegation field-coverage ratchet. |
tensor-grep config and flags
A ground-truthed catalog of every tg config axis — env vars, CLI flags, provider modes — plus the
registration checklist for adding a new one. Verified against source as of 2026-07-23, v1.95.0
(pyproject.toml). Re-verify commands are in Provenance and maintenance
because these drift with every release.
When to use this skill
- You are adding, renaming, or removing an env var or CLI flag.
- A search flag is reaching ripgrep raw (
rg: unrecognized flag at runtime) or a command 404s.
- You need to know whether a knob (
--gpu-device-ids, --provider lsp, TENSOR_GREP_CLASSIFY_PROVIDER=cybert)
is production-safe to recommend to a user, or still experimental/default-off.
- You need the authoritative default value or guard condition for a
TG_* / TENSOR_GREP_* variable
before writing docs, a benchmark harness, or an agent prompt that references it.
When NOT to use this skill (use the sibling instead)
| If you need... | Use instead |
|---|
| The why behind the front-door/routing architecture, not just the flag list | tensor-grep-architecture-contract |
| The process gates for shipping a flag/command change (PR, CI, one-merge-per-tick) | tensor-grep-change-control |
| Reproducing/debugging a routing bug once you already know which flag is involved | tensor-grep-debugging-playbook |
tg doctor output fields, dogfood harness, benchmark scripts | tensor-grep-diagnostics-and-tooling / tensor-grep-benchmark-and-proof-toolkit |
How to actually use tg commands day to day (not configure them) | .claude/skills/tensor-grep/SKILL.md |
| Build/toolchain setup (uv, maturin, cargo) | tensor-grep-build-and-env |
| Release mechanics / positioning claims | tensor-grep-release-and-positioning |
The two front doors, and why config is split across them
tg has two CLI entry points that both parse flags, and a config change that only lands in one
is a silent bug, not a crash:
- Python bootstrap (
src/tensor_grep/cli/bootstrap.py) — tensor_grep.cli.bootstrap:main_entry
intercepts plain-text searches before the Typer app loads and forwards them straight to rg.
- Rust native front door (
rust_core/src/main.rs) — the standalone tg binary, which re-implements
flag parsing with clap.
The canonical, always-current documentation of every env var and the full search flag surface lives
in two places that are meant to stay in sync — read these first when you need ground truth fast:
tg --help epilog: src/tensor_grep/cli/main.py:187-200 (the app = typer.Typer(help="""...""") block).- Native
tg --help epilog: ENVIRONMENT_OVERRIDES_HELP const, rust_core/src/main.rs:67.
If those two drift from each other or from this file, trust the source, not this document — see
Provenance and maintenance.
Environment variable catalog
Boolean env vars in tg follow one convention almost everywhere (env_flag_enabled,
grep -n "def env_flag_enabled" src/tensor_grep/cli/runtime_paths.py — was :13-15, now :20-22):
the raw value is lower-cased and stripped, and it is "on" only if it is exactly 1, true, yes,
or on — anything else (including unset) is "off". Two STRICT exceptions compare the raw value
against the literal "1" only, so true/yes/on do NOT enable them: TG_DOCTOR_OFFLINE is
== "1" on the Python side (grep -n "TG_DOCTOR_OFFLINE" src/tensor_grep/cli/main.py — :492),
and TG_RESIDENT_AST is != "1" on the Rust side (grep -n "TG_RESIDENT_AST" rust_core/src/main.rs — :7581).
Routing / launcher
| Var | Default | Effect | Source |
|---|
TG_SIDECAR_PYTHON | sys.executable | Python executable used for sidecar-backed commands (classify, GPU sidecar). | main.py:188, main.py:533 |
TG_NATIVE_TG_BINARY (alias TG_MCP_TG_BINARY) | auto-resolved | Path to the native tg binary front door used by Python-backed commands. Priority 1 override; stale in-tree dev builds (rust_core/target/{debug,release}/tg.exe) are otherwise skipped unless pinned here. | main.py:189, runtime_paths.py:238-248 |
TENSOR_GREP_NATIVE_FRONTDOOR_FLAVOR (alias TG_NATIVE_FRONTDOOR_REQUESTED_FLAVOR) | cpu | nvidia/cuda prefers the NVIDIA release-native front-door asset (tg-*-nvidia.exe), with CPU fallback; anything else normalizes to cpu. | main.py (grep -n "def _normalize_native_frontdoor_flavor" src/tensor_grep/cli/main.py -- :550 as of 2026-08-14; the prior :7276/:454-473 cites were stale) |
TG_RG_PATH | auto-resolved | Path to the rg executable used for text-search passthrough. | main.py:191, runtime_paths.py:281 |
TG_FORCE_CPU | off | Force CPU routing for search commands (boolean convention). | main.py:192, main.py:2772 |
TG_RUST_FIRST_SEARCH | off | Opt-in: prefer the Rust native front door before Python bootstrap logic for search dispatch. | bootstrap.py:242 |
TG_RUST_EARLY_RG, TG_RUST_EARLY_POSITIONAL_RG | off | Internal early-dispatch toggles surfaced by tg doctor --json; not documented in the public epilogs. |
Timeouts
| Var | Default | Effect | Source |
|---|
TG_RG_TIMEOUT_SECONDS | 60.0s (lowered from 600s in #288) | Ripgrep-passthrough search timeout. Fails fast with a stderr hint to scope the search or raise the timeout, instead of hanging. Overridden by TG_SIDECAR_TIMEOUT_MS when that is set to a positive value. | grep -n "TG_RG_TIMEOUT_SECONDS" src/tensor_grep/cli/subprocess_policy.py (was :32-44, now :75) |
TG_SIDECAR_TIMEOUT_MS | unset | Milliseconds; if set and > 0, takes precedence over TG_RG_TIMEOUT_SECONDS for the ripgrep-passthrough timeout (ms / 1000.0). Also documented as the general sidecar-command timeout. | subprocess_policy.py:32-40, main.py:193 |
TG_SUBPROCESS_TIMEOUT_SECONDS | 600.0s | Default timeout for the generic run_subprocess() helper (git ops, MCP validation commands, etc.) unless a call site overrides timeout_env_var. | subprocess_policy.py:82-87 |
TG_GIT_TIMEOUT_SECONDS | 120.0s | Timeout for git subprocess calls (checkpoint/session git operations). | subprocess_policy.py:28-29 |
TENSOR_GREP_TRITON_TIMEOUT_SECONDS | 5.0s | Timeout for Triton-backed NLP (CyBERT) probes. | cybert_backend.py:18-19, main.py:196 |
TENSOR_GREP_LSP_OPERATION_BUDGET_SECONDS | 2.0s | Total per-command budget for optional external LSP provider requests before falling back to native evidence. | grep -n "TENSOR_GREP_LSP_OPERATION_BUDGET_SECONDS" src/tensor_grep/cli/repo_map.py (was :95-96, now :193-194), main.py:198 |
TENSOR_GREP_LSP_REQUEST_TIMEOUT_SECONDS, TENSOR_GREP_LSP_INITIALIZE_TIMEOUT_SECONDS | implementation defaults | Per-request / per-initialize LSP timeouts; reported (not overridden) by . |
Every non-positive or unparseable value for the float-typed timeout vars above silently falls back to
the compiled-in default (_configured_positive_float, subprocess_policy.py:9-17) — a bad value does
not crash, it just gets ignored. Do not assume "I set it, therefore it changed" without checking
tg doctor --json's env block (see Discovering effective config).
GPU
| Var / flag | Default | Effect | Source |
|---|
--gpu-device-ids IDS (CLI flag, e.g. tg search --gpu-device-ids 0,1) | unset (no GPU routing) | Explicit, user-intent GPU pin for search / tg agent / benchmark evidence probes. Comma-separated non-negative ints; parse errors raise typer.BadParameter immediately (main.py:4061-4074). | main.py:5725-5762, main.py:6890-6969 |
TENSOR_GREP_DEVICE_IDS | unset (all detected devices visible) | Lower-level env allow-list of GPU IDs available to tensor-grep at all (like CUDA_VISIBLE_DEVICES), consulted by device detection/memory-manager code, not just the CLI flag. | device_detect.py:30-55, main.py:194 |
--gpu-timeout-s (flag, tg agent only) | 5.0s | Max seconds for each opt-in agent GPU evidence subcommand. | grep -n "gpu_timeout_s: float = typer.Option" src/tensor_grep/cli/main.py (was :6970-6975, now :9987-9991) |
Fail-loud contract: an explicit --gpu-device-ids request that cannot be honored raises
ConfigurationError (a RuntimeError subclass, pipeline.py:20-21) — it never silently falls back to
CPU. Example message shape (pipeline.py:26-39):
GPU acceleration is experimental. Explicit GPU device selection [0, 1] could not initialize a
GPU backend: fixed-string (-F) search has no GPU backend
This is deliberate: -F/--fixed-strings GPU search has no kernel yet, so pairing it with an explicit
--gpu-device-ids must error, not silently drop to CPU and report a clean result (see the Backend
Fail-Closed Contract in AGENTS.md:214-224, and tensor-grep-architecture-contract for the general
principle). By contrast, the heuristic (non-explicit) auto-GPU path degrades to CPU with a visible
warnings.warn(...) + fallback_reason, not an exception — only explicit user intent gets fail-loud
treatment (pipeline.py:174-176, _should_honor_explicit_gpu_ids).
GPU remains EXPERIMENTAL end-to-end. GPU Phase-0 SHIPPED (v1.75.0-v1.75.4, PRs #593-#597):
NVIDIA native assets are built and locally correctness-proven (RTX 4070 sm_89 / RTX 5070 sm_120 --
docs/gpu_crossover.md), but gated OFF the public release by the CI Actions var
TENSOR_GREP_RELEASE_NATIVE_ASSET_PROFILE (default native-frontdoor, CPU-only; GPU asset
publishing needs the non-default native-frontdoor-gpu) -- Phase 1 is now a reversible flag-flip, not
a multi-week rebuild. That flip publishes assets only: no speed crossover is proven vs rg/tg_cpu,
GPU auto-recommendation stays false, and the reviewer-gated public-gpu-proof.yml speed-crossover
gate remains unmet (docs/CONTRACTS.md:80-82). Do not market or default-enable it.
Classify provider
| Var | Default | Effect | Source |
|---|
TENSOR_GREP_CLASSIFY_PROVIDER | heuristic (local, deterministic) | Set to cybert or triton to opt into the CyBERT/Triton NLP classifier for tg classify FILE. Any other value (including unset) uses the local regex-heuristic classifier. | sidecar.py:16-17, sidecar.py:119-125 |
tg classify always reports provenance in its JSON output (classification_backend /
provider_requested / provider_used / provider_status / fallback_reason / cache,
sidecar.py:57-71) so a caller can distinguish "asked for cybert, got heuristic because it failed"
from "asked for heuristic". Never read a classify result as model-backed without checking
provider_used.
Session / daemon
| Var | Default | Effect | Source |
|---|
TG_SESSION_MAX | 64 | Max on-disk cached sessions retained per root; oldest are pruned past this. | grep -n "_SESSION_MAX_ENV|_configured_session_max" src/tensor_grep/cli/session_store.py (was :49-51, :120-121, now :88-89 (env+default), :159 (usage)) |
TG_SESSION_NEARBY_LOOKUP | off | By default session discovery is confined to the explicit root; set to opt into parent/sibling-directory session discovery. | grep -n "_SESSION_NEARBY_LOOKUP_ENV" src/tensor_grep/cli/session_store.py (was :52-54, :124-130, now :90-92) |
TG_SESSION_DAEMON_IDLE_SECONDS | 900.0s | Idle stretch (no requests) after which the warm tg session daemon self-shuts-down. Non-positive disables the idle limit. | grep -n "_DAEMON_IDLE_SHUTDOWN_SECONDS_ENV|_DEFAULT_DAEMON_IDLE_SHUTDOWN_SECONDS" src/tensor_grep/cli/session_daemon.py (was :67-72, now env-name :93, default :95) |
TG_SESSION_DAEMON_MAX_UPTIME_SECONDS | 86400.0s (24h) | Hard max daemon lifetime regardless of activity. Non-positive disables the uptime limit. | grep -n "_DAEMON_MAX_UPTIME_SECONDS_ENV|_DEFAULT_DAEMON_MAX_UPTIME_SECONDS" src/tensor_grep/cli/session_daemon.py (was :67-73, now env-name :94, default :96) |
TG_SESSION_DAEMON_RESPONSE_TIMEOUT_SECONDS | 60.0s | Client-side socket read timeout for a daemon response (#390, moat P0-6 step 5). Env-configurable so a large repo whose warm-daemon graph query legitimately needs >60s isn't killed by a hard cap that returns a bare "timed out"/exit 1/zero JSON. Does not by itself bound the daemon's own traversal — the served graph commands run on a cached map and are not covered by the scan-side --deadline; see the #390 daemon-path gap in tensor-grep-large-repo-scale-campaign. | grep -n "_DAEMON_RESPONSE_TIMEOUT_ENV|_DAEMON_RESPONSE_TIMEOUT_SECONDS =" src/tensor_grep/cli/session_daemon.py (was , now env-name , default ) |
The daemon binds to 127.0.0.1 only (session_daemon.py:46) — it is not exposed off-host. Operational
detail (starting/stopping the daemon, tg session daemon start|status|stop) lives in
.claude/skills/tensor-grep/REFERENCE.md, not here.
Agent capsule
| Var | Default | Effect | Source |
|---|
TG_CAPSULE_INLINE_CALLERS | off (env_flag_enabled-style on-values: 1/true/yes/on) | When on, tg agent/tg prepare prepend # tg: callers=N (top: a, b) to the PRIMARY snippet's source, reusing already-collected blast-radius evidence (no new scan). Off by default for a stronger reason than most flags here: it mutates snippets[i].source/line_map/token_estimate on the primary snippet (an inserted line shifts both the displayed source and its line-number mapping, and raises the token estimate ~+2.8%), rather than only adding a new field — a consumer that diffs/re-parses source byte-for-byte will see it change. An additive snippets[i].inline_structural_annotation field is also added. py/js/ts/rs comment syntax only; fails closed (no annotation) for any other language. callers=N is only ever emitted on a verified count; token-budget truncation is fail-closed (never silently drops the annotation without accounting for its cost). | agent_capsule_constants.py (find it: grep -n "_CAPSULE_INLINE_CALLER_ANNOTATION_ENV = " src/tensor_grep/cli/agent_capsule_constants.py) (_CAPSULE_INLINE_CALLER_ANNOTATION_ENV) |
MCP security gate (default-OFF)
| Var | Default | Effect | Source |
|---|
TG_MCP_ALLOW_VALIDATION_COMMANDS | off | Gates whether the tg mcp server's tg_rewrite_apply tool may accept and shell-execute lint_cmd / test_cmd (from either the direct call arguments or a loaded apply-policy JSON file — both paths are gated, not just the direct one). Off by default because these commands can be steered by untrusted repo content / prompt injection; the agent-safe edit loop does not require them. Rejected requests return code="unsupported_option". | mcp_server.py:249-254, apply_policy.py:41-45,226-230 |
This is an Enablement Discipline case: default-OFF, opt-in only, and it is the kind of knob you should
never flip on in a shared/CI MCP server config without an explicit operator decision — see
AGENTS.md "Enablement Discipline (autonomous behaviors)" (referenced from the workspace root
CLAUDE.md) and tensor-grep-change-control for the graduation gate (council-verify → dry-run →
conscious flag-flip).
Evidence signing (tg evidence emit --sign / tg evidence verify)
| Var | Default | Effect | Source |
|---|
TG_EVIDENCE_SIGNING_KEY | unset → ambient default key | Ed25519 private key path for --sign. Precedence: --signing-key flag > this env var > the ambient per-USER default key ~/.tensor-grep/keys/evidence_ed25519.key. Consequence (A70): clearing/unsetting the env var does NOT disable signing when the ambient default key exists — resolution falls through to the default path and --sign still signs. To force a true no-key fail-closed arm, isolate HOME/USERPROFILE (or remove the default key). With no resolvable key file, --sign fails closed: non-zero exit, no receipt written — never a silent unsigned fallback. | grep -n "def resolve_signing_key_path" src/tensor_grep/cli/evidence_signing.py — :133-140 (precedence), _default_signing_key_path :127-130, _DEFAULT_KEY_FILENAME = "evidence_ed25519.key" :60 |
TG_EVIDENCE_TRUSTED_KEYS | unset | The trust pin: comma-separated base64 Ed25519 public keys, merged with repeatable --trusted-key flag values. verify always reports the signer's fingerprint recomputed from the actual key bytes, but only upgrades key_trusted to true against this out-of-band pinned set (hmac.compare_digest); --require-trusted fails valid closed on an unpinned key. An embedded public key proves internal consistency, never authenticity. | grep -n "_TRUSTED_KEYS_ENV|def resolve_trusted_public_keys" src/tensor_grep/cli/evidence_signing.py — :57, :143-152 |
Ledger (tg ledger claims / findings)
| Var | Default | Effect | Source |
|---|
TG_LEDGER_CLAIM_TTL_SECONDS | 900 | Claim TTL in seconds; expired claims are pruned. | grep -n "_TTL_ENV|_DEFAULT_TTL_SECONDS" src/tensor_grep/cli/ledger_store.py — :124-125 |
TG_LEDGER_AGENT_ID | anonymous sentinel (after fallback) | Agent identity for claim/release, recorded verbatim (never inferred from process/user identity; do not put secrets in it — it lands in a plaintext, multi-agent-readable per-repo JSON). Precedence: --agent-id flag > this env > TG_EVIDENCE_AGENT_ID > the anonymous sentinel. The sentinel is DELIBERATE: two zero-config agents must both file as anonymous so _find_overlaps shows them each other's overlaps (#845) — do not auto-derive a per-checkout id. | grep -n "_AGENT_ID_ENV|_FALLBACK_AGENT_ID_ENV|_DEFAULT_AGENT_ID|def resolve_agent_id" src/tensor_grep/cli/ledger_store.py — :127-129, :302 |
TG_LEDGER_FINDING_TTL_SECONDS | 86400 (24h) | Wall-clock backstop TTL for findings; revision match, not this TTL, is the primary freshness signal. | grep -n "_FINDING_TTL_ENV|_DEFAULT_FINDING_TTL_SECONDS" src/tensor_grep/cli/ledger_store.py — :168-169 |
TG_LEDGER_MAX_BLOB_BYTES | 256 MiB | Total on-disk bytes across all DISTINCT (content-addressed, dedup'd) finding blobs for one root — independent of the live-findings count cap, so a flood of small findings cannot accumulate unbounded disk under the count cap. | grep -n "_MAX_BLOB_BYTES_ENV|_DEFAULT_MAX_TOTAL_BLOB_BYTES" src/tensor_grep/cli/ledger_store.py — :178-179 |
In-process caches (bound long-lived agent-loop state)
These exist so a long-lived tg session daemon / MCP server process doesn't grow unbounded caches.
All are documented together in main.py:199-200; defaults are implementation-internal (read the
cited module if you need the exact number) — this skill's job is to tell you that they exist and
where, not to duplicate the numeric defaults, which drift independently of flags/commands.
TENSOR_GREP_CPU_LITERAL_INDEX_CACHE_MAX_ENTRIESTENSOR_GREP_STRING_INDEX_CACHE_MAX_ENTRIESTENSOR_GREP_AST_QUERY_CACHE_MAX_ENTRIESTENSOR_GREP_AST_NODE_INDEX_CACHE_MAX_ENTRIESTENSOR_GREP_REPO_CONTEXT_CACHE_MAX_ROOTSTENSOR_GREP_LSP_PROVIDER_CLIENT_CACHE_MAX_ENTRIESTENSOR_GREP_LSP_PROVIDER_OPEN_DOCUMENT_MAX_ENTRIES
Internal constants (not env-configurable)
Not every load-bearing bound in tg is an environment variable — some are deliberately hardcoded
constants, single-sourced so multiple call sites cannot drift apart. Distinguish these from the
env-configurable knobs above before assuming a behavior can be tuned at runtime:
IMPLICIT_SEARCH_WALK_FILE_CEILING = 1500 (DEFINED at src/tensor_grep/io/scan_limits.py:106;
io/directory_scanner.py only RE-EXPORTS it. An earlier revision cited directory_scanner, which
would send someone changing the ceiling to edit a re-export and wonder why nothing moved —
matches the sibling tensor-grep-architecture-contract A9 wording) — the
fast-refuse ceiling for an unscoped/defaulted-path search or tg find walk (A9, v1.92.3/#702). It is
imported by both src/tensor_grep/cli/main.py's _LARGE_ROOT_SCAN_FILE_CEILING and
src/tensor_grep/cli/bootstrap.py's _search_paths_include_oversized_implicit_root — one constant,
two Python call sites, so a future change to the ceiling cannot silently desync the Typer-app path
from the flag-less bootstrap-passthrough path. The Rust rust_core/src/rg_passthrough.rs keeps its
own copy of the same numeral, synced by convention (not a shared build-time constant across the
Python/Rust boundary) — if you ever change the Python value, grep rg_passthrough.rs for the
matching literal and update it in the same PR, or the two front doors will silently disagree on
where the ceiling sits.- This is a distinct axis from
TG_DIR_SCAN_MAX_ENTRIES (env-configurable, a different directory-
scan bound) — do not conflate the two when reading a scan-refusal report; check which constant/env
var actually produced the observed refusal before describing the mechanism.
LSP provider
| Var | Default | Effect | Source |
|---|
TG_LSP_PROVIDER | native | Overrides the LSP semantic-provider mode for editor/MCP clients; same value space as --provider (native/lsp/hybrid). Set by tg lsp --provider ... before calling run_lsp(). | main.py:9649-9799, main.rs:51 |
TG_ALLOW_UNVERIFIED_TOOLCHAIN | off | Security opt-out: skips checksum verification of downloaded LSP-toolchain archives/binaries (rust-analyzer, etc.) for air-gapped/offline installs — same default-secure/opt-out-to-weaken pattern as TG_MCP_ALLOW_VALIDATION_COMMANDS below. Off by default; fails closed (refuses the unverified binary) unless set. | lsp_provider_setup.py:229-265,465-480 |
Provider modes: native / lsp / hybrid
The --provider flag appears on every symbol/navigation command (defs, refs, source, impact,
callers, blast-radius*, context-render, edit-plan, agent, lsp) with the same three-way
contract everywhere, default native:
tg defs REPO_PATH SYMBOL --provider lsp
tg blast-radius REPO_PATH SYMBOL --provider hybrid
tg lsp --provider hybrid
native — tg's own tree-sitter/AST-derived symbol graph. Production default.lsp — routes through an external language server (ExternalLSPProviderManager). EXPERIMENTAL.hybrid — combines native with LSP evidence when available.
tg lsp validates the value explicitly and exits 2 on anything else
({"native", "lsp", "hybrid"} check, main.py:9656-9785):
Unsupported LSP provider mode; expected one of: native, lsp, hybrid
LSP-availability is not LSP-proof — this is a load-bearing distinction from AGENTS.md:163:
"Treat tg lsp-setup / tg doctor --with-lsp availability as install evidence only; provider-backed
navigation must report health_status, health_check, lsp_proof, lsp_evidence_status, and
not_lsp_proof_reason when it falls back to native evidence. A navigation row counts as LSP proof only
when it carries lsp_provider_response = true from a completed provider request." Do not tell a user
"LSP is working" because tg doctor --with-lsp found a binary on PATH.
Production vs EXPERIMENTAL (default-OFF), and the guard that keeps it off
| Axis | Status | Guard | Why |
|---|
Native CPU/rg search, AST search (tg run), symbol nav (native provider) | Production | none — default path | Backbone of the tool. |
tg agent / Actionable Context Capsule | Production, opt-in by design | explicit tg agent invocation | Not a default search mode; it's a distinct command surface, but it is a shipped, supported feature. |
classify local heuristic | Production | default | Deterministic, no model download. |
--gpu-device-ids / GPU backends | EXPERIMENTAL | must be explicitly requested; heuristic auto-GPU only fires when rg is unavailable | Slower than CPU today; no promotion-ready path (AGENTS.md:226-234). |
--provider lsp / --provider hybrid, TG_LSP_PROVIDER | EXPERIMENTAL | explicit --provider value or TG_LSP_PROVIDER env | Availability ≠ working navigation; see LSP-proof contract above. |
TENSOR_GREP_CLASSIFY_PROVIDER=cybert/triton | EXPERIMENTAL | explicit env opt-in | Requires a Triton/CyBERT model deployment; falls back before expensive model load if unavailable. |
TG_MCP_ALLOW_VALIDATION_COMMANDS=1 | Off by design (security), not "not ready yet" | explicit env opt-in on the MCP server process | Shell-executes lint_cmd/test_cmd, a prompt-injection surface. |
| Local hybrid semantic search (BM25 + CPU dense embeddings + RRF) | SHIPPED, EXPERIMENTAL default-OFF — tg search --semantic (grep -n '"--semantic"' src/tensor_grep/cli/main.py — was :6619, now :7403; core/retrieval_dense.py + ) |
tg inventory: walk-only repo manifest (v1.19.0, #343)
tg inventory PATH [--json] [--max-repo-files N] [--deadline SECONDS] (src/tensor_grep/cli/inventory.py,
registered main.py:8292-8404) emits a single-pass file/byte/language/category manifest by
reusing the same gitignore-aware walker (repo_map._iter_repo_files) that orient/callers/
blast-radius trust — so counts stay truth-consistent with every other tg command and inherit
its .tensor-grep/.git/vendor exclusions for free.
--max-repo-files defaults to 50_000, still well above the AST map limit — this is a
deliberate, documented divergence, not an oversight. The AST-side number changed underneath this
divergence (backlog #1, 2026-07-06): DEFAULT_AGENT_REPO_MAP_LIMIT was raised from 512 to
2000 (repo_map.py:157), and the CLI-side mirror _DEFAULT_AGENT_REPO_SCAN_LIMIT (main.py:82)
was raised to match — do not describe the AST cap as 512 anymore.
DEFAULT_MAX_INVENTORY_FILES = 50_000 (inventory.py:40), passed to the CLI option as a
literal 50_000 (grep -n "50_000" src/tensor_grep/cli/main.py -- no line range: the old :8406-8413 pin sat INSIDE a --deadline option block that the 2026-08-23 de-duplication deleted outright, so it has no successor line to re-stamp to) rather than importing the constant, so the (heavy)
repo_map import stays lazy. A nearby code comment still says "matching map's 512 pattern" —
that comment is about the STYLE (keep-literal, don't import), not the current live number; map's
own limit is 2000 now, not 512. A guard test pins the 50_000 literals together; re-verify with
grep -rn "50_000" src/tensor_grep/cli/inventory.py src/tensor_grep/cli/main.py.DEFAULT_AGENT_REPO_MAP_LIMIT = 2000 (repo_map.py:157) budgets a full AST parse per file
for tg map/orient/context/edit-plan/session repo-map defaults — reusing it for
inventory would silently truncate any repo over ~2000 files and defeat the "whole-repo
manifest" purpose (inventory.py:36-39 states this explicitly in a code comment).
inventory is walk-only (stat() + an 8KB read for binary-sniffing per file), orders of
magnitude cheaper than an AST parse, so a much higher cap (50_000) is still safe even after the
AST-side raise.CALLER_SCAN_FILE_CEILING was ALSO raised, 512→2000 (repo_map.py:167-177; backlog #57,
2026-07-09) — the "DIFFERENT constant that stays at 512" framing this file previously used is
itself now stale. It remains logically separate from DEFAULT_AGENT_REPO_MAP_LIMIT (they just
now happen to share a value) — the raise was safe only because #478 had already threaded a
--deadline hard-bound through the caller-scan loop, closing the task #52 ~100s-hang risk
("~100s on a 1941-file repo at the old 512 cap", repo_map.py:163-164) that originally kept this
ceiling frozen below the map default (repo_map.py:1638-1642). It still backstops the flag-less
(--deadline omitted) default path and a -raised mega-repo; raising past 2000
needs fresh cost data (). If you see the bare number anywhere in this
subsystem going forward, it is describing HISTORY — check which constant before assuming either
reading is still live.
--deadline SECONDS: the wall-clock twin of --max-repo-files (registered main.py:8297-8423)
Threads a deadline_seconds float (inventory.py:187) into build_inventory() so a huge/slow
tree returns a partial, honestly-labeled manifest instead of hanging. inventory's own
--deadline predates and is unrelated to #585 (2026-07-14), which extended --deadline to
source/docs-coverage/blast-radius-plan instead (a disjoint set of commands) — inventory's
flag shipped earlier (#395, issue #53), was hardened into a true wall-clock bound by #478
(issue #52), then had a zero-count bug fixed (next bullet) by #516.
- The walk and the per-file loop split the budget, they do not share it
(
inventory.py:204-223; _WALK_PHASE_DEADLINE_FRACTION = 0.7, inventory.py:48) — the
in-source comment calls this the "#130(a) fix" (shipped as #516 per commit history). The walk
phase (_iter_repo_files) gets only the first 70% of deadline_seconds; the remainder is
reserved for the per-file stat()/binary-sniff loop. Before this split, a slow walk could
consume the entire deadline, so the per-file loop's very first deadline check fired
immediately and totals.files read 0 despite the walk having discovered real files — the
in-source comment names the repro outright (inventory.py:204-208: "the 76s dogfood gap on
tg inventory --deadline 30").
- Either phase running out of budget sets
truncation_cause = "deadline" (inventory.py:297-306)
— including when the walk was ALSO count-capped by --max-repo-files; "deadline" wins the label
because a longer --deadline would help where a higher --max-repo-files would not. An earlier
version of this same guard mislabeled a real 20s deadline hit on C:/dev/projects as a file-cap
truncation (inventory.py:303-304, dogfood 2026-07-05) — exactly the bug this two-cause split
exists to prevent from recurring.
- Known narrow gap (low-priority, not load-bearing): the walk's initial listing of the SCANNED
ROOT itself is not deadline-interruptible.
_iter_repo_files (grep -n "^def _iter_repo_files" src/tensor_grep/cli/repo_map.py -- :1183 as of 2026-08-14, was :1143), on the
max_files is not None branch inventory always takes (it always calls with
max_files=max_files + 1, inventory.py:231-236), does one eager, unconditional
entries = list(os.scandir(normalized_root)) (repo_map.py:1009-1010) before any deadline
check exists in that branch — every subsequent bucket pull IS deadline-checked
(repo_map.py:1052-1057), just not this one root-level call. On a root whose own immediate
directory listing is itself pathologically large/slow, --deadline cannot preempt it. Dogfooded
against a 300k+-file workspace-union tree: holds up fine per-project
and on most roots — this edge needs a pathologically huge flat fan-out sitting directly at the
scanned path to trigger, and is not worth a load-bearing lazy-scandir rewrite on its own; see
for the broader deadline-scale work this sits alongside.
Registration follows the standard 4-site table (KNOWN_COMMANDS in commands.py, native Rust
Commands::Inventory in rust_core/src/main.rs, PUBLIC_TOP_LEVEL_COMMANDS in
tests/e2e/test_routing_parity.py, the @app.command() in main.py) — see
tensor-grep-architecture-contract for why each site exists.
# Re-verify the two caps and why they differ
grep -n 'DEFAULT_MAX_INVENTORY_FILES\|max_repo_files' src/tensor_grep/cli/inventory.py src/tensor_grep/cli/main.py
grep -n 'DEFAULT_AGENT_REPO_MAP_LIMIT = \|CALLER_SCAN_FILE_CEILING = ' src/tensor_grep/cli/repo_map.py
# Re-verify --deadline registration + the walk/per-file budget split
grep -n '"--deadline"' src/tensor_grep/cli/main.py
grep -n 'deadline_seconds\|_WALK_PHASE_DEADLINE_FRACTION' src/tensor_grep/cli/inventory.py
# Smoke-test the command against the real binary (not CliRunner)
tg inventory . --json | python -m json.tool | head -30
Checklist: adding a flag or command
This is the single highest-value thing to get right in this repo — miss a registration site and the
new flag/command misroutes silently, passing CliRunner tests while breaking the real published
binary. See tensor-grep-architecture-contract for the full 4-site command / 2-site search-flag
registration table and the rationale for why each site exists, and tensor-grep-change-control for
the PR/merge gate around it (AGENTS.md:178-196) — this skill does not restate that table.
Worked non-registration example — tg prepare --out FILE (v1.93.0/#705, A12(d)). Not every new
flag triggers all 3 registration concerns this skill and its siblings track — a useful example of
"none of the above applied" to calibrate against: adding --out to the already-registered tg prepare
command needed (1) no new 4-site command registration (the command already existed), (2) no 2-site
search-flag registration (--out is not a search flag — it never reaches bootstrap's rg-passthrough
front door), and (3) no SearchConfig field-coverage classification (prepare doesn't build a
SearchConfig at all). It was a same-command, same-registration-footprint addition — a new typer.Option
on an existing @app.command, nothing else. Recognizing when a change genuinely needs none of the 3
checklist items (vs. assuming every new flag does) saves a wasted registration audit.
The third checklist item: native-delegation field coverage (SearchConfig)
Adding a new field to SearchConfig (src/tensor_grep/core/config.py) is a third registration
concern, separate from the 4-site command table and the 2-site search-flag table above — and it is
easy to miss because it fails silently, not loudly.
Why it exists: _can_delegate_to_native_tg_search (grep -n "^def _can_delegate_to_native_tg_search" src/tensor_grep/cli/main.py -- :4095 as of 2026-08-14, was :3709) hands an entire search
off to the native tg subprocess, which sys.exit()s before the Python-side BM25 rerank and
the in-backend file sort ever run. Any SearchConfig field that is output-affecting but neither
forwarded into the native argv (_build_native_tg_search_command -- grep -n "^def _build_native_tg_search_command" src/tensor_grep/cli/main.py, :4117 as of 2026-08-14, was :3731) nor listed in the refuse-tuple
_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS (grep -n "_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS = " src/tensor_grep/cli/main.py -- :1966 as of 2026-08-14, was :1894 onward) gets silently dropped —
the search still runs and returns a result, just the wrong one (unranked/unsorted), which is worse
than a crash because suppression reads as absence. This is the same bug class as the -u/-uu
no-op fixed in #336; the receipt this time was #342 (commit 5e6f780, v1.18.6->v1.19.0 range):
rank_bm25 and sort_files were parsed but forwarded nowhere, so tg search --rank --cpu /
--sort-files --cpu silently returned unranked/unsorted output on the delegated fast path.
The gate is now a governance ratchet, not a convention — tests/unit/test_native_delegation_field_coverage.py
AST-derives the forwarded-field set directly from _build_native_tg_search_command's source (via
ast.walk over config.<attr> reads, so it can't drift from a hand-maintained list) and asserts
test_every_field_classified: every dataclasses.fields(SearchConfig) name must be one of:
- Forwarded — read by
_build_native_tg_search_command and passed into the native argv.
- Refused — listed in
_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS; a non-default value
forces _can_delegate_to_native_tg_search to return False and fall through to the
Python/backend path. rank_bm25 and sort_files live here as of #342 — native tg has no
BM25 (it routes --rank back to the Python sidecar) and sort_files is applied in-backend
(ripgrep_backend.py/rust_backend.py), so neither is reproducible on a delegated sys.exit
path.
- Gate-handled —
files_with_matches/files_without_match; read off explicit keyword args
at the call site rather than the config object, so the gate itself covers them.
KNOWN_GAP — _NATIVE_TG_DELEGATION_KNOWN_GAP_FIELDS in the test file: pre-existing
fields (AST-mode selectors, NLP threshold, internal telemetry, ignore_* scope flags, no_*
double-negation flags) that were already dropped through delegation before #342 and are
acknowledged tech debt, not blessed as safe — a documented gap, not a silently-dropped one.
case_sensitive is NOT in this set anymore — audit #19 forwarded it into the native argv
via -s (main.py:3367), so it's now bucket 1 (Forwarded), not a gap; the test file's own
comment at the KNOWN_GAP set records this explicitly. Don't describe case_sensitive as a
native-delegation gap in new docs. A companion test (test_known_gap_has_no_stale_entries)
fails if a KNOWN_GAP entry is later forwarded/refused/removed and the entry isn't pruned, so
the gap set can't rot into a false-safe list either.
When you add a SearchConfig field, this test goes RED until you classify it — that red is the
checklist: forward it (native argv), refuse it (_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS),
confirm it's gate-handled, or add it to _NATIVE_TG_DELEGATION_KNOWN_GAP_FIELDS with a one-line
reason. Do not add to KNOWN_GAP just to make the test pass — that's exactly the silent-drop this
ratchet exists to prevent; only pre-existing, reasoned gaps belong there.
Landmine already hit once — do not re-attempt the "differs from default" runtime gate. The
2026-06-30 #1 naive-fix failure mode: query_pattern is auto-set on every search
(main.py ~line 6045), so a generic "does any field differ from SearchConfig() defaults" gate
would trip on query_pattern on literally every call and kill the fast path entirely. The fix must
be a specific field added to the tuple, not a blanket differs-from-default check.
# Run the ratchet directly (fast, no fixtures needed)
uv run pytest tests/unit/test_native_delegation_field_coverage.py -v
# Confirm the refuse-tuple + KNOWN_GAP set still exist at these names
grep -n '_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS = ' src/tensor_grep/cli/main.py
grep -n '_NATIVE_TG_DELEGATION_KNOWN_GAP_FIELDS = ' tests/unit/test_native_delegation_field_coverage.py
# See exactly which fields the ratchet currently derives as "forwarded"
uv run python -c "
from tensor_grep.cli import main as tg_main
import ast, inspect
src = inspect.getsource(tg_main._build_native_tg_search_command)
print(sorted({n.attr for n in ast.walk(ast.parse(src)) if isinstance(n, ast.Attribute) and isinstance(n.value, ast.Name) and n.value.id == 'config'}))
"
Miss either search-flag site and the flag reaches rg unrecognized for users on the published binary
— an rg: unrecognized flag crash that CliRunner-only tests cannot see, because CliRunner calls the
Typer app directly and bypasses the bootstrap front door (tensor-grep-architecture-contract covers
why). Dogfood the real binary (scripts/dogfood/) after any flag/command change — do not rely on
CliRunner alone.
As of v1.17.1 (#282) the CI registration-completeness gate is blocking: a mismatch between these
sites fails CI, not just warns (AGENTS.md:196). There is also a standalone checker,
src/tensor_grep/core/registration_check.py, driven by .tg-registration.toml and wired into
.github/workflows/ci.yml:319-324 — run it locally to catch a mismatch before pushing (see
Provenance and maintenance).
Discovering effective config: tg doctor --json
Setting an env var is not the same as confirming it took effect — a bad float value silently falls
back to the default (see the Timeouts table). tg doctor --json echoes back the currently observed
value for the routing/timeout/LSP-budget env vars it knows about, under an env key that only includes
vars that are actually set (_build_doctor_payload -- grep -n "^def _build_doctor_payload" src/tensor_grep/cli/main.py, :3452 as of 2026-08-14, was :3142-3164; the env filter (built from the env_keys list -- grep -n "env_keys = " src/tensor_grep/cli/main.py, :3463 as of 2026-08-14, was :3000)
filters to os.environ.get(key) truthy). It reports:
TG_NATIVE_TG_BINARY, TG_FORCE_CPU, TG_RESIDENT_AST, TG_RUST_FIRST_SEARCH, TG_RUST_EARLY_RG,
TG_RUST_EARLY_POSITIONAL_RG, TENSOR_GREP_LSP_REQUEST_TIMEOUT_SECONDS,
TENSOR_GREP_LSP_INITIALIZE_TIMEOUT_SECONDS, TENSOR_GREP_LSP_OPERATION_BUDGET_SECONDS, plus the
LSP-probe timeout env var. It does not currently echo every var in this skill's catalog (e.g. the
session/GPU/classify vars are not in that list) — when in doubt, check the source, not just doctor.
For launcher-route diagnostics (which tg binary actually ran) rather than config-value diagnostics,
use tensor-grep-diagnostics-and-tooling.
Provenance and maintenance
Flags, defaults, and registration sites drift every release — re-verify before trusting this file on
anything but the day it's dated.
# Confirm the current version this skill was verified against
grep -n '^version = ' pyproject.toml
# Re-pull the authoritative env-var help text from both front doors (diff them against this file)
sed -n '187,200p' src/tensor_grep/cli/main.py
grep -n 'ENVIRONMENT_OVERRIDES_HELP' -A1 rust_core/src/main.rs | head -5
# Re-list every TG_*/TENSOR_GREP_* env var referenced anywhere in source (catch new ones this file misses)
grep -rhoE '"(TG_|TENSOR_GREP_)[A-Z0-9_]+"' src rust_core/src | sort -u
# Re-check the 2 search-flag front doors
sed -n '160,272p' rust_core/src/main.rs # SEARCH_PYTHON_PASSTHROUGH_FLAGS
sed -n '23,58p' src/tensor_grep/cli/bootstrap.py # _TG_ONLY_SEARCH_FLAGS
# Re-check the 4 command registration sites
sed -n '9,54p' src/tensor_grep/cli/commands.py # KNOWN_COMMANDS
grep -n 'enum Commands' rust_core/src/main.rs
grep -n 'PUBLIC_TOP_LEVEL_COMMANDS = ' -A5 tests/e2e/test_routing_parity.py
# Run the standalone registration checker locally (same as CI)
PYTHONPATH=src python -m tensor_grep.core.registration_check .tg-registration.toml
# Confirm provider-mode validation still lists exactly these three
grep -n 'native.*lsp.*hybrid' src/tensor_grep/cli/main.py
# Confirm the GPU fail-loud contract still raises (not falls back)
grep -n '_raise_explicit_gpu_configuration_error\|class ConfigurationError' src/tensor_grep/core/pipeline.py
# Re-verify tg inventory's two caps (50_000 walk-only vs the AST map limit) still diverge deliberately
grep -n 'DEFAULT_MAX_INVENTORY_FILES\|max_repo_files' src/tensor_grep/cli/inventory.py src/tensor_grep/cli/main.py
grep -n 'DEFAULT_AGENT_REPO_MAP_LIMIT = \|CALLER_SCAN_FILE_CEILING = ' src/tensor_grep/cli/repo_map.py
# Re-verify the native-delegation field-coverage ratchet still exists and passes
grep -n '_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS = ' src/tensor_grep/cli/main.py
uv run pytest tests/unit/test_native_delegation_field_coverage.py -v
Verified against source as of 2026-07-08 (v1.49.3): tg inventory section including
the DEFAULT_AGENT_REPO_MAP_LIMIT 512->2000 raise + the CALLER_SCAN_FILE_CEILING trap; the
native-delegation field-coverage ratchet including case_sensitive's removal from KNOWN_GAP
(audit #19); the new TG_SESSION_DAEMON_RESPONSE_TIMEOUT_SECONDS env var (#390).
Re-verified as of 2026-07-16 (v1.78.1): the new TG_FIND_DENSE_WEIGHT row above, read directly
against main.py:4021-4072.
Re-verified as of 2026-07-22 (v1.93.2): the 3 native-delegation cites (_can_delegate_to_native_tg_search
→ main.py:3712, _build_native_tg_search_command → main.py:3745,
_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS → main.py:1908); the new TG_CAPSULE_INLINE_CALLERS
catalog row (agent_capsule_constants.py (find it: grep -n "_CAPSULE_INLINE_CALLER_ANNOTATION_ENV = " src/tensor_grep/cli/agent_capsule_constants.py)); the new "Internal constants" subsection
(IMPLICIT_SEARCH_WALK_FILE_CEILING = 1500, defined in io/scan_limits.py, re-exported by
io/directory_scanner.py); and the tg prepare --out
worked non-registration example. The rest of this file (env-var catalog, front-door tables, GPU/LSP/
provider sections above) was not re-walked line-by-line in this pass — treat those sections'
exact line numbers as needing a fresh check per the rule below, independent of the sections just
re-verified.
Re-verified as of 2026-07-23 (v1.95.0): the tg inventory section end-to-end. One factual fix:
the CALLER_SCAN_FILE_CEILING bullet had called it "the different constant that stays at 512" —
repo_map.py shows backlog #57 (2026-07-09) raised it to 2000 alongside
DEFAULT_AGENT_REPO_MAP_LIMIT, one day after this file's 2026-07-08 tg-inventory pass captured the
512 value accurately, and it went uncaught through the 2026-07-16 and 2026-07-22 passes because
neither re-scoped to this section. Every other citation in the section was re-walked against
current line numbers (main.py's inventory registration moved 7090→8403 as the file grew past
17k lines). Also added the previously entirely-undocumented --deadline SECONDS flag: registration,
the walk/per-file budget-split history (the 76s-dogfood-gap bug, "#130(a)"/#516), the two-cause
truncation_cause split, and a known narrow root-level-os.scandir gap that --deadline cannot
preempt (low-priority, not load-bearing). The env-var catalog, front-door tables, GPU/LSP/provider
sections, and the native-delegation checklist citations (all last verified 2026-07-22 or earlier)
were not re-walked in this pass — treat every exact line number outside the tg inventory
section as needing a fresh check, same rule as before.
Release-cadence note (added 2026-07-23): main picked up two MINOR version bumps
(v1.93→v1.94→v1.95) in roughly the one-day gap between this file's 2026-07-22 and 2026-07-23
passes, and merges land one-at-a-time rather than batched, so line-number drift here is continuous,
not occasional. Re-verify this file on a cadence of a few releases, not only when a citation is
reported broken — the CALLER_SCAN_FILE_CEILING miss above sat wrong for two whole passes because
nothing forced a re-check of a section nobody had reported as broken.
Retention pass, 2026-08-12, verified against v1.110.14 (base 568065a): recorded the v1.110.14
doctor rows — the TG_DOCTOR_OFFLINE routing/launcher row added for the doctor schema-3 freshness
fields had its citation re-verified in the same pass (grep -n "TG_DOCTOR_OFFLINE" src/tensor_grep/cli/main.py — :489-492, unchanged). Same pass: rewrote the TG_FIND_DENSE_WEIGHT
row to the shipped adaptive state (reader re-grepped: env/default consts :4271-4272 → :4593-4594,
adaptive const :4600, _find_dense_weight :4613-4684); softened the boolean-convention claim
with the two strict == "1" / != "1" exceptions (main.py:492, rust_core/src/main.rs:7581) and
re-anchored env_flag_enabled (runtime_paths.py :13-15 → :20-22); and added the
evidence-signing rows (evidence_signing.py:56-60,127-152) plus the four ledger env rows
(ledger_store.py:124-129,168-169,178-179,302). The rest of this file (GPU/LSP/provider sections,
front-door tables, tg inventory section, native-delegation checklist) was not re-walked in this
pass — treat those sections' exact line numbers as needing a fresh check, same rule as before.
If AGENTS.md's release_docs_current_tag no longer says v1.78.1, treat every default/line-number
claim in this file as needing re-verification, not just the version string.
