tensor-grep

Use when searching code, logs, or repositories with tensor-grep; validating rg or AST parity; using tg MCP tools; checking GPU/search routing; or producing agent-friendly context, source, refs, or blast-radius output.

Use when you need the domain theory behind tensor-grep's search/retrieval behavior, not just the command syntax — ripgrep exit codes (including the exit-2-but-kept partial-results contract)/PCRE2/binary-NUL-detection/-uuu/-- sentinels, ast-grep + tree-sitter routing, BM25 vs the flat no-IDF capsule scorer, PageRank vs in-degree centrality, the trigram index, PyO3 + the GIL, MCP argv surface, LSP 3.17 framing, and Model2Vec/potion-code (SHIPPED as `tg search --semantic`). Load before explaining WHY tg behaves a certain way, reasoning about the protocol/algorithm THEORY underneath a backend/router change (exit-code semantics, scoring math, wire framing), or writing docs that touch these subsystems — for the invariants a backend/router change must not break, use `tensor-grep-architecture-contract` instead (or in addition). Not a how-to-run or how-to-debug guide — see the sibling table below for those.

Use when you need the load-bearing design of tensor-grep and WHY it holds before touching cli/bootstrap.py, rust_core/src/main.rs, backends/, core/result.py, cli/main.py's native-delegation gate, routing, the agent capsule, or before reviewing/planning any change to the front door, command/flag registration, or backend contract. Explains the bootstrap intercept-before-Typer front door, native-vs-Python routing, the 4 command + 2 flag registration sites, the Backend Fail-Closed Contract, the native-delegation forward-or-refuse contract (`_can_delegate_to_native_tg_search` + its field-coverage ratchet), the partial-results `result_incomplete`/`incomplete_reason` envelope, `MatchLine`'s frozen-but-hashable dataclass contract, the ASCII-only CLI output rule, the agent-context moat, the invariants that must hold, and the known-weak points (flat no-IDF scorer, GPU not viable, rg parity gap, FFI not the dir-scan speed path). Read this to build the right mental model; use sibling skills for the how-to of changing, de

Use when asked to deep-dive, audit, fix, or drain tensor-grep backlog — OR investigate/rank next work and produce SPEC/TDD plans (docs/plans/requirements|design|tasks-*.md) without implementing. Triggers: "work the backlog", "what next", "investigate and plan", backlog-completion campaign. META-ORCHESTRATOR — 20-skill library. Semantic-search flagship: tensor-grep-semantic-search-campaign. Scale/hang campaign: tensor-grep-large-repo-scale-campaign. Load tensor-grep-change-control before edit.

Use when about to claim, review, or dispute a speedup/regression in tensor-grep (tg vs rg, hot-cache, AST, agent-workflow, or GPU changes) — which benchmark script to run, how to read check_regression.py, the noise-floor/absolute-jitter rule for sub-10ms rows, the fair-baseline rule (never compare tg against a strawman comparator), and the launcher-attribution rules (tg_launcher_mode, tg_launcher_command_kind, stale-binary refusal) that make a benchmark artifact claim-quality instead of noise.

Use when setting up the tensor-grep dev environment from a fresh clone, rebuilding the Rust/PyO3 extension or standalone `tg` binary after touching `rust_core/`, or debugging a build/toolchain problem — uv install, `maturin develop`, `cargo build`, the pinned 1.96.0 Rust toolchain, Python >=3.11 floor, a "hanging" cargo build, cargo/rustc missing from PATH, ruff CRLF false-alarms, or a dependency upper-cap silently downgrading tensor-grep on a newer Python. Gives exact copy-paste setup commands and the traps that have each cost a real cycle.

Use when about to change, review, merge, or release ANY code in tensor-grep — adding a tg command or search flag, touching a backend/router/pipeline, editing CI/release/docs contracts, merging a PR, or claiming a fix or speedup is done. Encodes the non-negotiable gates (draft-PR-only autonomy, never-trust-a-self-report, no-speed-claim-without-numbers, experimental-until-proven, TDD-first, smallest-change, benchmark-hot-paths, the 4 registration sites, one-merge-per-tick / the push-race, dogfood-the-real-binary, contract-changes-need-validator-tests) and the historical incident behind each.

Use when adding, changing, or auditing a tg environment variable, CLI flag, or provider mode (native/lsp/hybrid); when a search flag silently leaks to ripgrep or a command misroutes; when deciding whether a config axis (GPU, LSP, classify, semantic) is production or EXPERIMENTAL default-OFF; when adding a new `SearchConfig` field and needing to know whether it must be forwarded/refused/KNOWN_GAP'd for native delegation; or before registering a new `tg search --flag` or `tg COMMAND` (including `tg inventory`'s `--max-repo-files`). Catalogs the load-bearing TG_*/TENSOR_GREP_* env vars (routing, timeouts, GPU, classify, session, MCP security, LSP) with their default and guard, the 2-front-door / 4-site registration checklist, and the native-delegation field-coverage ratchet.

Use when a tensor-grep (tg) run fails, hangs, returns wrong/empty/silently-degraded results, a CI check goes red, or a release doesn't publish. Symptom-to-triage table, each row giving a discriminating experiment and a fix pointer, for CI red, release not published (push-race), search hangs/slow, silent-empty result (fail-closed contract), argv/flag injection, mock-green-but-real-dead FFI, dependency-cap silent downgrade, ranking flip, a `CliRunner`/`capfd` test that goes green-on-PR-red-on-main after a delegation/routing change, and a latency fix/regression report that needs profiling-at-scale instead of a code-reading guess. Load BEFORE theorizing from a traceback or re-running a failing gate blind.

Use when you need to MEASURE tensor-grep's health instead of eyeballing it -- interpreting a `tg doctor --json` field, reading a PASS/FAIL/SKIP from `tg dogfood` or `scripts/agent_readiness.py`, or explaining what a diagnostic field actually proves (and does not prove). Not for CLI syntax (tensor-grep-run-and-operate), fixing a found bug (tensor-grep-debugging-playbook), or deciding which `benchmarks/*.py` script to run / constructing a claim-quality benchmark artifact (tensor-grep-benchmark-and-proof-toolkit).

Use when about to "fix" or "optimize" something in tensor-grep that feels novel — before proposing PyO3/FFI for directory walking, re-enabling free-threading, adding a --json self-test, tightening a dependency upper-cap, blaming an IDF/ranking flip, trusting a green mock/FFI test, diagnosing a release that "didn't publish", chasing a reported latency "regression" without profiling at scale, shipping a doc-drift/precision heuristic off green fixtures alone, adding a "differs-from-default" native-delegation gate, reading a `capfd`-based CliRunner test result, or micro-optimizing a hot loop without checking who actually consumes the value. A chronicle of settled battles (symptom -> root cause -> evidence -> status) so no one re-fights them. Load it to check "has this already been tried and lost?" before spending effort. For a live NEW failure use tensor-grep-debugging-playbook; for the process gates to re-attempt one use tensor-grep-change-control.

Use when tg hangs, stalls, or runs for minutes on a large/unscoped repo; when `--deadline` "seems ignored" and a symbol query still overruns its budget; when working task #52 (end-to-end deadline ineffective on a ~1800-file TS repo), the #390 daemon-path deadline gap, caller-scan / build_repo_map latency, the unscoped-`tg search` hang, or the exit-2 partial-result semantics. The decision-gated campaign to finish agent-native SCALE HONESTY: tg must never hang and never silently lie (return an empty/partial result as if it were complete) on a customer-scale repo. Gives the reproduce -> phase-instrument (cProfile) -> ranked solution menu -> fail-closed build -> change-control promotion runbook with exact commands, expected numbers, and branch-on-mismatch forks. Verified against v1.49.3 on 2026-07-08 (#400 fully shipped since v1.40.4; the exit-code contract is exit-2- regardless-of-found, per #401); `tg find` addition spot-checked 2026-07-16 at v1.78.1.

Use when merging a release-bearing PR, writing a PR title for tensor-grep, diagnosing "why didn't my release publish" or "why hasn't npm/the docs site updated", running the post-publish dogfood, or making any external-facing speed/GPU/LSP/benchmark claim about tg. Covers semantic-release mechanics in ci.yml, the push-race + one-merge-per-tick discipline (worked example: the #384-#399 sequence), the npm/docs manual-dispatch publish gap, PyPI/npm/Homebrew/winget publish gates, and the not-faster-grep positioning + reproducibility standard for comparator claims. As of 2026-07-16, v1.78.1.

Use when scoping, pitching, planning, or judging a SOTA-advancing bet in tensor-grep (tg) — the OPEN research problems, not a bug fix or a shipped feature. Load when asked "where can tg beat the state of the art / what's the moat / is this worth building", or when touching the five frontier programs: the GPU PFAC many-pattern/resident wedge (Phase-0 shipped v1.75.0-v1.75.4, crossover still unproven), de-fragilizing the flat no-IDF ranking scorer (the IDF blast-radius), closing raw-grep parity via a native launcher/control-plane, the arXiv moat-deepeners (AST-node MCP read/write, graph-traversal tools, intent-aware blast-radius), or the parked `tg diff-docs` precision rebuild (naive absence-from-symbol-table doc-drift detection floods 20k+ false positives — the DocPrism trap). Everything here is candidate/experimental — to actually build or merge one, route to tensor-grep-change-control; for settled dead-ends, tensor-grep-failure-archaeology.

Use when you have a hunch, a candidate optimization, a novel technique, or an "I think X is faster/better/possible" idea for tensor-grep and must turn it into an ACCEPTED result or a DOCUMENTED RETIREMENT — the research discipline. Covers the evidence bar (one mechanism must explain every observation INCLUDING the negatives, and survive an assigned adversarial refutation whose every claim cites file:line), predicting the number + noise band BEFORE you run, the idea lifecycle (experimental default-OFF -> dogfood/benchmark -> council-verify -> conscious flag-flip to adopted OR retirement recorded in docs/PAPER.md so it is never retried), verifying an AI-drafted plan against real code, and where good ideas actually come from. The scientific-method META sibling to change-control (gates) and research-frontier (targets).

Use when running the `tg` CLI day-to-day — exact syntax for orient, search --rank, defs/refs/callers/blast-radius, agent, docs-coverage, context, session open/refresh/serve/daemon, checkpoint, scan --ruleset, run, mcp, doctor, dogfood, upgrade; the symbol-command 0/1/2 exit-code contract and what an agent should branch on; bounding a scan with `--deadline` and reading `partial`/`result_incomplete`/`deadline_limit` truncation flags; excluding vendor/skill trees with `--ignore` on orient/agent/docs-coverage; `tg context --max-tokens` budgets (default 16000, 0=opt-out); where JSON artifacts and cache state land on disk; starting the MCP server; or a whole-repo `tg search` that hangs. The OPERATOR runbook (how to invoke), not theory or audit workflow.

Use when building, extending, or reviewing tensor-grep's APPROVED local hybrid semantic search — BM25 + CPU dense embeddings fused with Reciprocal Rank Fusion (RRF), no API key, no GPU (roadmap item #1). Load before adding a dense/embedding leg, RRF fusion, a `tg index` command, or changing `tg search --rank` / `--bm25`. Covers the decision-gated build phases with exact commands + expected gate numbers, the ranked solution menu (Semble / ripvec / BM25-only) with derivation obligations, the retrieval-quality + editor-plane + token-economy promotion gates, the Backend Fail-Closed Contract for the dense leg, fenced-off wrong paths (no API-key embeddings, no GPU dependency, do not break `--format rg` / `--json` / `--ndjson` semantics), and routing promotion through change-control. STATUS as of 2026-07-08, v1.49.3: the dense leg + RRF fusion described here as the target architecture (Candidate 1 / the Semble pattern) SHIPPED as `tg search --semantic` (`retrieval_dense.py` + `retrieval_fusion.py`, default-OFF, gate

Use tensor-grep for repository code search, symbol lookup, blast-radius analysis, and edit planning when solving codebase tasks or preparing patches.

Use when deciding what counts as proof that a tensor-grep (tg) change works — before trusting a subagent's "tests pass", writing a new test, claiming a routing/docs/release fix is done, shipping a doc-drift/ranking/classification heuristic off green fixture tests, or running the pre-push gate. Covers TDD-first discipline, the CliRunner-vs-real-binary trap, the fixture-green-vs-real-corpus-dogfood trap for precision/heuristic features, the `capfd`-vs-`result.stdout` capture-surface trap on routing/delegation changes (needs `tests/integration/` run with the native `tg` binary rebuilt, not just `tests/unit/`), the certified/golden inventory (routing parity, docs governance, release-asset validation), agent-readiness/`tg dogfood`, benchmark-gated speed claims, acceptance thresholds, and which suite/marker/fixture to use for a new test, plus the `--preview` / `--no-sync` / `-x` gotchas.

Use when designing or evaluating tensor-grep as an enterprise agentic code-intelligence tool — PATH narrowing for complete graphs, EvidenceReceipts, codemap, multi-repo workspaces, accuracy gates, and world-class readiness gaps.

Use when stress-testing tensor-grep against a multi-project workspace (monorepo parent, many languages) — orientation, scoped search, symbol graphs, imports/importers, codemap, evidence receipts, sessions, and readiness gates. Not the PyPI release dogfood harness (see global dogfood-the-shipped-artifact).

Use when writing or editing any tensor-grep doc of record (AGENTS.md, CLAUDE.md, root SKILL.md, docs/SESSION_HANDOFF.md, docs/CONTRACTS.md, docs/CONTINUATION_PLAN.md, docs/PAPER.md, README.md, docs/benchmarks.md, docs/gpu_crossover.md, mkdocs.yml, or any .claude/skills/*/SKILL.md) — before adding a capability claim, syncing a release-line note, touching a version-stamped line, adding a new governed doc, or a docs-only pytest fails and you don't know which fragment broke it. Covers which doc owns which contract, the two governance layers (semantic-release version_variables + scripts/stamp_release_assets.py auto-stamping vs pytest content-pinning), the cross-doc fragment discipline, mkdocs --strict, and house style.

Use tensor-grep for repository code search, symbol lookup, blast-radius analysis, and edit planning when solving codebase tasks or preparing patches.