codeprobe-document

name	codeprobe-document
description	Generate a comprehensive, cross-linked codebase documentation directory at `.codeprobe/docs/` (33 markdown files across 8 subdirectories, with Mermaid diagrams and source-level citations). Use when the user asks to 'document this codebase', 'generate docs', or 'create documentation'. DO NOT use for single-section updates — pass `--section <name>` or edit the target file directly. DO NOT use if `.codeprobe/heuristic_summary.json` is absent — run `uv run codeprobe index .` first to produce the required artifacts.
allowed-tools	["Bash","Read","Write","Glob","Grep","Agent","mcp__gitnexus__query","mcp__gitnexus__context","mcp__gitnexus__impact","mcp__gitnexus__cypher","mcp__gitnexus__tool_map","mcp__gitnexus__api_impact","mcp__gitnexus__route_map","mcp__gitnexus__shape_check","mcp__gitnexus__list_repos","mcp__plugin_personal-plugins_context7__resolve-library-id","mcp__plugin_personal-plugins_context7__query-docs","ReadMcpResourceTool","ListMcpResourcesTool","TaskCreate","TaskUpdate","TaskList","TaskGet"]
argument-hint	[output-dir] [--section <name>] [--refresh]

Codebase Documentation Generator

Generate a comprehensive, cross-linked documentation directory for any codebase. Produces 33 markdown files across 8 subdirectories with Mermaid diagrams, source-level citations, and computed "See also" cross-references.

Arguments

Parse $ARGUMENTS for:

• Output directory: first positional arg (default: .codeprobe/docs/)
• --section <name>: generate only one section (architecture, analysis, reference, behavior, specialized, technical-debt, diagrams, migration)
• --refresh: re-generate only sections whose source data changed since last run

Output Directory Placement

The default output is .codeprobe/docs/ — colocated with all other codeprobe artifacts. This keeps documentation:

• Self-contained alongside index data, SARIF results, and heuristic summaries
• Covered by existing .gitignore rules for .codeprobe/
• Easy to regenerate (delete .codeprobe/docs/ and re-run)

To place documentation in a standalone directory (e.g., for committing to version control), pass an explicit path:

/document docs/              # → ./docs/
/document documentation/     # → ./documentation/
/document .codeprobe/docs/   # → .codeprobe/docs/ (default)

Pipeline Architecture

The pipeline has 4 phases. The key insight is that Phase A and Phase B have zero data dependencies on each other — both read from .codeprobe/ artifacts and the codebase directly. This means all 6 content agents can run in parallel, cutting wall-clock time roughly 3x compared to sequential execution.

Phase 0 (pre-flight)  →  Phase AB (6 agents in parallel)  →  Phase CD (2 agents in parallel)
                          ├─ doc-architecture                  ├─ doc-diagrams
                          ├─ doc-analysis                      └─ doc-migration
                          ├─ doc-reference
                          ├─ doc-behavior
                          ├─ doc-specialized
                          └─ doc-technical-debt
                                                            →  Cross-reference assembly (inline)

Phase CD agents can also run in parallel: doc-diagrams reads Phase AB output for cross-referencing (but writes independent files), while doc-migration reads architecture + analysis output for ordering. Both produce independent file sets.

Progress Tracking

Use TaskCreate and TaskUpdate throughout the pipeline to show the user real-time progress. Create tasks at the start:

TaskCreate: "Phase 0 — Pre-flight and data assembly"
TaskCreate: "Phase AB — Content generation (6 agents in parallel)"
TaskCreate: "Phase CD — Diagrams + migration (2 agents in parallel)"
TaskCreate: "Phase E — Cross-references + README"

Mark each task in_progress when starting and completed when done. Use activeForm to show which agents are currently running.

Phase 0 — Pre-flight and Shared Data Assembly

Verify Prerequisites

Read .codeprobe/heuristic_summary.json. If missing, run the index first:

uv run codeprobe index .

Wait for completion, then read the generated heuristic_summary.json.

Gather Shared Data

Read all .codeprobe/ artifacts into working context:

Artifact	Purpose
heuristic_summary.json	Volume, languages, frameworks, health, git, complexity, gitnexus metadata
CONTEXT.signatures.md	Tree-sitter compressed code signatures (API surface)
CONTEXT.orientation.md	Token-aware directory tree with file sizes
files.all.txt	Complete file manifest
git_hotspots.json	Top 30 files by churn
git_cochanges.json	Co-change pairs for coupling analysis
complexity.json	Radon cyclomatic complexity per function
semgrep_auto.json	General SAST findings
semgrep_owasp.json	OWASP Top Ten findings
typecheck.json	Type errors (ty or pyright)
lint.json	Ruff lint violations
dead_code_py.json	Python dead code (vulture)
dead_code_ts.json	TypeScript dead code (knip)
deps.json	Dependency tree
scan.sarif	Merged SARIF (if scan was run)

For large artifacts (>256KB — typically CONTEXT.signatures.md and complexity.json), read only the first 100 lines to get the format, then let agents read specific sections via offset/limit. Do NOT try to read these in full.

Write Shared Context File

Write <output-dir>/.context.md — a single file summarizing the project that all agents can read instead of duplicating this in every agent prompt. Include:

• Project name and description (from README.md H1 + first paragraph)
• Package structure (from CONTEXT.orientation.md top-level dirs)
• Volume metrics (from heuristic_summary.json)
• Health metrics summary (findings, errors, complexity)
• Dependency chain (from deps.json or pyproject.toml)
• Key architectural decisions (from CLAUDE.md if present)

This file should be under 200 lines. Its purpose is to give agents project context without bloating prompts.

GitNexus Pre-fetch and Persist

If heuristic_summary.gitnexus.indexed is true, run three strategic queries and write results to disk so agents can read them:

1. Communities: mcp__gitnexus__cypher — MATCH (c:Community) RETURN c.label, c.symbolCount, c.cohesion ORDER BY c.symbolCount DESC LIMIT 20
2. Processes: mcp__gitnexus__cypher — MATCH (p:Process) RETURN p.heuristicLabel, p.stepCount ORDER BY p.stepCount DESC LIMIT 20
3. Overview: Read GitNexus resource gitnexus://repo/{repo_name}/context

Write results to <output-dir>/.gitnexus-prefetch.md as three markdown sections. Agents read this file rather than re-running queries.

Create Output Structure

Create the output directory and all subdirectories:

<output-dir>/
  architecture/
  behavior/
  analysis/
  diagrams/behavioral/
  diagrams/architecture/
  diagrams/structural/
  reference/
  specialized/
  technical-debt/
  migration/

Refresh Mode

If --refresh is set, read <output-dir>/.docmeta.json. Compare each section's generated_at timestamp against the mtime of its data source artifacts. Skip sections whose sources have not changed. Always re-run Phase E (cross-references may need updating).

Section Mode

If --section <name> is set, run Phase 0 fully, then dispatch only the corresponding agent, then run Phase E.

Phase AB — Content Generation (6 agents in parallel)

Dispatch all 6 agents in a single message with 6 Agent tool calls. Each agent reads .context.md and .gitnexus-prefetch.md for shared context, plus its section-specific artifacts. Each agent writes files directly to the output directory.

Tell each agent: "Read <output-dir>/.context.md for project context and <output-dir>/.gitnexus-prefetch.md for GitNexus data before starting."

1. doc-architecture agent

Produces: project-overview.md, architecture/system-overview.md, architecture/components.md, architecture/dependencies.md, architecture/patterns.md

Key inputs: CONTEXT.signatures.md (chunk-read), CONTEXT.orientation.md, deps.json, .gitnexus-prefetch.md

Scope for architecture/dependencies.md: Internal dependency graph (which packages/modules import which), external dependency inventory (versions, purposes), and dependency health assessment. Focus on the structure of dependencies — what depends on what and why.

2. doc-analysis agent

Produces: analysis/code-metrics.md, analysis/complexity-analysis.md, analysis/dependency-analysis.md, analysis/security-patterns.md

Key inputs: complexity.json (chunk-read), semgrep_auto.json, semgrep_owasp.json, typecheck.json, lint.json, git_hotspots.json, git_cochanges.json

Scope for analysis/dependency-analysis.md: Coupling metrics and change patterns — co-change frequency, coupling clusters, fan-in/fan-out numbers, bus factor risks. Focus on the dynamics of how dependencies evolve and which couplings are problematic. This complements architecture/dependencies.md (structure) with empirical data.

3. doc-reference agent

Produces: reference/program-structure.md, reference/interfaces.md, reference/data-models.md, reference/api-reference.md

Key inputs: CONTEXT.signatures.md (chunk-read), CONTEXT.orientation.md, files.all.txt

Scope for reference/api-reference.md: Compact catalog of all public API surfaces — tables of endpoints, CLI commands, MCP tools, module exports. One-line descriptions, handler references. Think "API index" — enough to find what you need, links to detailed docs.

4. doc-behavior agent

Produces: behavior/business-logic.md, behavior/workflows.md, behavior/decision-logic.md, behavior/error-handling.md

Key inputs: .gitnexus-prefetch.md (processes), CONTEXT.signatures.md (chunk-read). Agent uses mcp__gitnexus__query and Read() for deep source analysis.

5. doc-specialized agent

Produces: specialized/api-documentation.md, specialized/database-schemas.md, specialized/infrastructure.md

Key inputs: heuristic_summary.json (for stack detection). Agent uses Glob, Grep, Read to discover and analyze Dockerfiles, terraform, SQL migrations, API definitions.

Scope for specialized/api-documentation.md: Detailed endpoint/tool documentation — parameters, response bodies, error codes, examples, configuration. This is the "deep dive" companion to reference/api-reference.md (the "index"). Think OpenAPI-level detail.

6. doc-technical-debt agent

Produces: technical-debt/summary.md, technical-debt/outdated-components.md, technical-debt/maintenance-burden.md, technical-debt/remediation-plan.md

Key inputs: git_hotspots.json, git_cochanges.json, dead_code_*.json, complexity.json (chunk-read), semgrep findings, lint.json

Note: There is no separate root-level technical-debt-report.md. The technical-debt/summary.md serves as both the executive summary and the detailed inventory. It should lead with a severity distribution table and key recommendations before diving into per-category findings.

Phase CD — Diagrams + Migration (2 agents in parallel)

Wait for all Phase AB agents to complete, then dispatch both agents in a single message with 2 Agent tool calls.

7. doc-diagrams agent

Produces: diagrams/behavioral/sequence-and-activity.md, diagrams/behavioral/data-flow.md, diagrams/architecture/context-and-security.md, diagrams/architecture/integration-patterns.md, diagrams/structural/class-diagrams.md, diagrams/structural/component-and-deployment.md

Key inputs: .gitnexus-prefetch.md, deps.json, CONTEXT.signatures.md (chunk-read), plus paths to all Phase AB output for cross-referencing. For Mermaid patterns, consult references/mermaid-patterns.md.

8. doc-migration agent

Produces: migration/component-order.md, migration/test-specifications.md, migration/validation-criteria.md

Key inputs: deps.json, heuristic_summary.json, architecture output, analysis output

Phase E — Cross-Reference Assembly (inline, no agent)

After all agents complete, run this step inline (no subagent needed). The goal is to link related documents together based on shared source file citations.

Step 1: Extract metadata from all generated files

For each .md file in the output directory (recursively):

1. Extract the H1 title (first line matching ^# (.+)$)
2. Extract all source file references — backtick-wrapped tokens containing a dot-extension (e.g., `dispatch.py`, `work_dao.py`). Strip trailing parentheticals like (298 LOC).

Step 2: Build co-occurrence index

Invert to source_file → [documents_referencing_it]. Two documents that share 2+ source file references are candidates for cross-linking.

Step 3: Compute and append "See also" footers

For each document (except files under diagrams/ and README.md):

1. Rank candidates by shared source file count (descending)
2. Select top 3-5 candidates (minimum 3 if available, maximum 5)
3. Append footer:

   ---
   *See also: [Title](relative-path) | [Title](relative-path)*
   

   Use relative paths from the current file to the target. Use the target's H1 title as link text.

If the number of documents is small enough that computing this manually is impractical, use a script. Write a Python script to /tmp/xref.py that:

• Walks the output directory for .md files
• Extracts titles and source file refs via regex
• Computes co-occurrence matrix
• Appends footers
• Run it with uv run /tmp/xref.py <output-dir>

For cross-referencing rules and README format, consult references/cross-reference-spec.md.

Step 4: Generate README.md

Write the navigation hub with:

• Section tables linking to all files with one-line descriptions
• Cross-reference index table (topic clusters from co-occurrence data)
• Generation metadata (date, file count, line count)

Step 5: Write .docmeta.json

Generation timestamps per section plus data source file paths for refresh mode. Include the list of files produced per section.

Document Format Rules

All generated documents follow these conventions:

• H1 title: Single # heading, one per file — this is the document identifier
• No YAML frontmatter: H1 serves as the title
• Source citations: Reference specific files with LOC — e.g., `dispatch.py` (298 LOC)
• Tables: Use markdown tables for structured data (dependencies, metrics, configs)
• Mermaid: All diagrams as Mermaid code blocks in markdown (no image files)
• Cross-references: "See also" footer linking 3-5 related documents with relative paths

For structural templates of each output file, consult references/document-templates.md.

Scope Differentiation Guide

Three document pairs cover overlapping topics. Here's how they differ:

Topic	Document A (structure/catalog)	Document B (metrics/detail)
Dependencies	architecture/dependencies.md — What depends on what. Internal graph, external inventory, health.	analysis/dependency-analysis.md — How dependencies behave. Coupling metrics, co-change frequency, fan-in/fan-out.
API surface	reference/api-reference.md — Compact index. One table per API surface with handler refs.	specialized/api-documentation.md — Full docs. Parameters, responses, errors, examples.
Tech debt	technical-debt/summary.md — Executive severity table + per-category findings inventory.	technical-debt/remediation-plan.md — Prioritized fix plan with effort estimates and blast radius.

Agents producing these documents should respect these boundaries. If content could go in either document, use this rule: catalog/structure → the A document; metrics/actionable detail → the B document.

Tool Selection Guide

Agents have access to codeprobe artifacts, GitNexus MCP tools, context7, and Claude Code native tools. Use the right tool for each job:

Need	Tool	Why
Code structure overview	CONTEXT.signatures.md	Pre-built, covers entire codebase
Specific symbol's callers/callees	mcp__gitnexus__context	Graph-powered 360° view
Execution flows for a concept	mcp__gitnexus__query	Ranked process results
Community/cluster structure	mcp__gitnexus__cypher	Custom graph queries
Blast radius of a component	mcp__gitnexus__impact	Upstream/downstream analysis
3rd-party library patterns	context7 resolve-library-id → query-docs	Current docs for detected frameworks
Deep reading of business logic	Read()	Source-level detail for behavior docs
Finding interface definitions	Grep	Pattern search across codebase
Discovering config/schema files	Glob	File pattern matching
Git history/blame	Bash (git commands)	Authorship, commit history
On-demand code bundle	Bash (repomix)	Targeted file bundles

For the complete data-source-to-document routing table, consult references/data-source-map.md.

Reference Files

• references/data-source-map.md — Which tool/artifact feeds which output document
• references/document-templates.md — Structural templates for all output files
• references/cross-reference-spec.md — Cross-linking rules and "See also" generation
• references/mermaid-patterns.md — Mermaid diagram templates per diagram type

Acceptance

Success = 33 markdown files exist under the output directory (default .codeprobe/docs/), every file is greater than 500 bytes, README.md cross-links resolve to real files, every non-diagram document ends with a See also footer linking 3-5 related docs, and .docmeta.json records generation timestamps and data sources for each section.