| name | tokkit-code-intelligence |
| description | Use for ANY codebase exploration, analysis, or understanding task on Python, JavaScript, or TypeScript repositories. Triggers: "explore the codebase", "find where X is defined", "how does Y work", "what calls this function", "show me the architecture", "trace the call chain", "what files are related", "find similar code", "what tests cover this", "find dead code", "unused code", "check dependencies", "analyze code", "code structure", "what imports this", "who uses this", "find all classes/functions", "code quality", "codebase overview". Uses tokkit knowledge graph tools instead of reading raw source files — 90%+ token savings.
|
Tokkit Code Intelligence
ALWAYS prefer tokkit MCP tools over built-in Read/Grep when the task matches. These tools process data server-side — raw content never enters your context.
Hook Model (Automatic)
A PreToolUse hook intercepts every Bash tool call and rewrites it to
tokkit compress '<command>'. Shell output is compressed before it enters
your context. This is automatic — you do not need to call compact_output
for live shell commands.
Covered automatically by the hook: git diff/status/log/show/blame/branch/stash,
kubectl, docker/docker-compose, pytest, ruff, mypy, cargo test/build/clippy,
tree/ls/find, grep/rg/ag, gh, npm ls, pip list/freeze,
and any other command via the generic fallback (ANSI strip, dedup, truncation).
The lint grouper post-processor automatically collapses rules with >3 violations
to a header + 2 examples, reducing repetitive lint output by 70-85%.
Call compact_output only when reading output already saved to a file on disk.
When to Use Tokkit (decision table)
Check this table BEFORE reaching for Read, Grep, or Glob:
| You're about to... | Use this instead | Why |
|---|
| Run a shell command (git, cargo, pytest, etc.) | Run it normally — hook compresses automatically | Hook intercepts, raw output never enters context |
Read an .html file | clean_html(path=...) | Server-side read, 60-90% fewer tokens |
Read a .json file | compact_json(path=...) | Server-side read, 30-70% fewer tokens |
Read a .md file for specific info | search_markdown(path=..., query=...) | Returns matching sections only, 70-85% fewer tokens |
| Read saved test/lint/build output from a file | compact_output(path=..., hint=...) | Strips noise, keeps failures, 70-85% fewer tokens |
| Grep for function definitions + references | find_dead_code() | One call vs O(N^2) greps |
| Grep for route decorators | find_routes() | One call, detects all frameworks |
| Recursive grep+read to trace call chains | trace_fan(function_name=...) | One call replaces 10-20 grep+read |
| Glob + Read to understand a project | get_architecture() | Complete overview in one call |
The path= parameter is critical. Content tools (clean_html, compact_json, search_markdown, compact_output) read files server-side when you pass path=. The raw content never enters your context. If you use Read first and then pass the content, you've already paid the token cost.
MCP Tools (9 available)
| Tool | Use instead of | Saves |
|---|
index_repository | (prerequisite for graph tools) | — |
get_architecture | Glob + Read README + Read source files | 80%+ |
find_dead_code | Grep all defs + grep all references | 90%+ |
find_routes | Grep for route decorators | 80%+ |
trace_fan | Recursive grep → read → grep cycles | 80%+ |
clean_html | Read .html file | 60-90% |
compact_json | Read .json file | 30-70% |
search_markdown | Read .md file | 70-85% |
compact_output | Read saved shell output from a file | 70-85% |
Note: for live shell commands, the hook compresses output automatically. compact_output is for output already written to a file on disk.
Workflow
1. Index First (graph tools only)
Before using get_architecture, find_dead_code, find_routes, or trace_fan:
index_repository(path="/absolute/path/to/repo")
Takes 1-5 seconds. Only needed once per session.
2. Graph Tools (code analysis)
get_architecture() → project overview
find_dead_code(limit=200) → unreferenced functions
find_routes(limit=200) → HTTP handlers with method + path
trace_fan(function_name="authenticate", direction="outbound", depth=3) → call fan-out
trace_fan(function_name="authenticate", direction="inbound") → who calls this
trace_fan accepts plain function names — no qualified name lookup needed.
3. Content Tools (no indexing required)
All content tools accept path= for server-side file reading. Always prefer path= over passing raw content.
clean_html(path="/path/to/page.html") → clean markdown
clean_html(path="...", mode="text") → plain text only
compact_json(path="/path/to/data.json") → CSV or YAML
search_markdown(path="/path/to/README.md", query="auth setup") → matching sections
search_markdown(path="...", query="") → header tree overview
compact_output(path="/path/to/test-results.txt", hint="pytest") → failures only
compact_output(path="...", hint="ruff") → violations summary
4. Fall Back to Read/Grep
Use built-in tools only when tokkit doesn't cover the case:
- Reading a specific function body →
Grep to find line, then Read(offset=, limit=)
- Raw text search across files →
Grep
- Non-code files (configs, lock files) →
Read
- Languages other than Python/JS/TS →
Read + Grep
compact_output: Supported Parsers
| Category | Tools | Hint values |
|---|
| Python test | pytest, unittest | pytest, unittest |
| Python lint/type | ruff, mypy, pyright | ruff, mypy, pyright |
| Python other | pip, tracebacks | pip, traceback |
| JS/TS test | jest, vitest, mocha | jest, vitest, mocha |
| JS/TS lint/type | tsc, eslint | tsc, eslint |
| JS/TS build | webpack, vite | webpack, vite |
| JS/TS package | npm | npm |
| Rust | cargo test/build/clippy | cargo-test, cargo-build, cargo-clippy |
| Container | docker build | docker |
Auto-detects when hint is omitted. Best results with explicit hint.
Known Limitations
- Route detection covers decorator-based frameworks (FastAPI, Flask, Starlette, NestJS) and call-expression frameworks (Express, Koa, Hono, Fastify), plus file-based routing (Next.js, SvelteKit). Django urlpatterns and Tornado class-based handlers are not yet detected.
- Languages supported: Python, JavaScript, TypeScript only. Other languages are skipped during indexing.
- Incremental indexing — not yet implemented. Re-indexing rebuilds the full graph.
Anti-Patterns
Do NOT Read an .html/.json/.md file when a tokkit tool can process it server-side. You pay the full token cost of the raw content for no reason.
Do NOT grep for function definitions to find dead code. find_dead_code does it in one call.
Do NOT skip indexing. Without index_repository, all graph tools return errors.
