repo-xray

name	repo-xray
description	AST-based Python codebase analysis. Use for exploring architecture, extracting interfaces, mapping dependencies, or generating onboarding documentation.

repo-xray

X-Ray scanner for Python codebases. Extracts 37+ signals across structure, behavior, and history to solve the cold start problem for AI coding assistants.

The Problem

Codebase:        2,000,000 tokens
Context Window:    200,000 tokens
Gap:                    10x

AI cannot read the whole codebase. It needs an intelligent map.

The Solution

X-Ray produces two outputs:

Output	Size	Purpose
Markdown	~8-15K	Curated summary — read this first
JSON	~30-50K	Complete reference — query as needed

Together, they compress a multi-million token codebase into actionable intelligence.

---

Quick Start

# Full analysis with both outputs
python xray.py /path/to/project --output both

# This creates:
#   output/<repo-name>/xray.md        — Curated summary for orientation
#   output/<repo-name>/data/xray.json — Complete data for reference

X-Ray Presets

python xray.py . --preset minimal   # ~2K tokens — quick survey
python xray.py . --preset standard  # ~8K tokens — balanced
python xray.py . --preset full      # ~15K tokens — comprehensive (default)

Key Options

# Output format
--output markdown     # Markdown only
--output json         # JSON only
--output both         # Both formats (recommended for agent)

# Output location
--out ./analysis      # Write to analysis.md and/or analysis.json (overrides default output/<repo>/ layout)

# Disable sections
--no-logic-maps       # Skip complex function analysis
--no-hazards          # Skip large file warnings
--no-git              # Skip git history analysis

# Other
--verbose            # Show progress
--help               # All options

---

What X-Ray Extracts

Structure (understand what exists)

• File tree with token estimates
• Class and function skeletons
• Type annotation coverage
• Decorator inventory

Architecture (understand relationships)

• Import graph with layers (orchestration → core → foundation)
• Dependency distance (BFS shortest paths)
• Circular dependencies
• Hub modules (most connected)
• Orphan files (dead code candidates)

History (understand evolution)

• Risk scores (churn × hotfixes × author count)
• Co-modification coupling (files that change together)
• Freshness categories (active/aging/stale/dormant)
• Git expertise per file

Complexity (understand difficulty)

• Cyclomatic complexity per function
• Complexity hotspots (ranked)
• Async patterns

Behavior (understand what code does)

• Side effects by type (DB, API, file, subprocess)
• Cross-module call graph
• State mutations
• Logic maps for complex functions

Context (understand how to use)

• Entry points (CLI, main, API routes)
• CLI arguments (argparse, Click, Typer)
• Environment variables with defaults
• Pydantic validators and constraints
• Linter rules from config files
• Test patterns and example

Safety (understand what to avoid)

• Hazard files (large/generated — don't read)
• Glob patterns for exclusion

---

Agent Workflow (Four Phases)

The repo_xray agent uses this skill in four phases:

Phase 1: ORIENT

python xray.py . --output both

Agent reads markdown summary for quick orientation. Builds mental model of codebase shape.

Phase 2: INVESTIGATE (Two Passes)

Pass 1: Signal Verification

• Agent uses Read/Grep/Glob to verify X-Ray signals
• Confirms or downgrades architectural pillars
• Assesses complexity: essential vs accidental
• Queries JSON for specific lookups when needed

Pass 2: Gap Discovery

• Search for patterns X-Ray didn't surface
• Look for error handling, retry logic, implicit dependencies
• Document counterintuitive behaviors (Gotchas)

Phase 3: SYNTHESIZE

Agent produces curated onboarding document using template. Not a dump — intelligent analysis with judgment.

Required sections:

• TL;DR (5 bullets)
• Quick Orientation
• Architecture diagram
• Critical Components (with [VERIFIED] tags)
• Data Flow
• Gotchas (counterintuitive behaviors)
• What X-Ray Missed

Phase 4: VALIDATE

Agent tests its own output before delivering.

• Can common questions be answered from the doc?
• Are quality metrics met?

---

Investigation Depth Presets

Preset	Pillars	Hotspots	Side Effects	Target Output
quick	Top 3 (skeleton only)	None	None	~5K tokens
standard	Top 5 (100 lines each)	Top 3	Critical only	~15K tokens
thorough	Top 10 (full read small, 200 lines large)	Top 5	All	~25K tokens

Usage:

@repo_xray analyze              # standard depth (default)
@repo_xray analyze --depth thorough   # deep investigation
@repo_xray survey               # quick depth

---

Confidence Levels

All agent insights must be marked with confidence:

Level	Meaning	Example
[VERIFIED]	Read actual code and confirmed	"[VERIFIED] Retry uses exponential backoff (2s, 4s, 8s)"
[INFERRED]	Logical deduction from related code	"[INFERRED] Cache invalidates on config reload"
[X-RAY SIGNAL]	Directly from X-Ray, not independently verified	"[X-RAY SIGNAL] CC=67 for main()"

---

Quality Metrics

Agent must report these metrics with every onboarding document:

Metric	Target
Pillars investigated	≥50% of top 10
Hotspots with verdicts	≥3
[VERIFIED] insights	≥10
Gotchas documented	≥3
Error paths documented	≥2
Compression ratio	≥50:1

---

Domain Detection

Agent auto-detects domain and adjusts investigation:

Domain	Indicators	Extra Investigation
Web API	FastAPI, Flask, Django, routes/	Auth, rate limiting, validation
ML/AI	torch, tensorflow, models/, training	Training loop, inference pipeline
Scientific	hypothesis, experiment, research	Validation logic, rigor scoring
CLI Tool	argparse, click, typer, commands/	Command structure, config loading
Data Pipeline	airflow, dagster, etl, pipeline	DAG structure, idempotency

---

Output Structure

Markdown Summary

# Codebase Analysis: project-name

## Summary
[File counts, lines, tokens, type coverage]

## Architecture
[Mermaid diagram showing layers and connections]

## Architectural Pillars
[Top 10 most important files, ranked]

## Complexity Hotspots
[Functions with highest cyclomatic complexity]

## Critical Classes
[Key class skeletons with signatures]

## Logic Maps
[Control flow analysis for complex functions]

## Side Effects
[I/O operations by category]

## Hazards
[Large files to avoid reading]

## Entry Points
[CLI commands, main functions]

## Environment Variables
[Required and optional env vars]

[...additional sections based on preset...]

JSON Reference

{
  "metadata": {
    "generated_at": "...",
    "preset": "standard",
    "file_count": 247,
    "total_tokens": 890000
  },
  "summary": { ... },
  "structure": {
    "files": { "path": { "lines": N, "tokens": N, "classes": [], "functions": [] } }
  },
  "imports": {
    "graph": { ... },
    "layers": { "orchestration": [], "core": [], "foundation": [] },
    "circular": [],
    "distance": { ... }
  },
  "complexity": {
    "hotspots": [ { "function": "", "file": "", "cc": N } ]
  },
  "git": {
    "risk": [],
    "coupling": [],
    "freshness": { "active": [], "aging": [], "stale": [], "dormant": [] }
  },
  "side_effects": {
    "by_type": { "db": [], "api": [], "file": [], "subprocess": [] }
  },
  "hazards": [],
  "entry_points": [],
  "environment_variables": [],
  ...
}

---

Token Budgets

X-Ray Output Sizes

Preset	Markdown	JSON	Total
minimal	~2K	~10K	~12K
standard	~8K	~30K	~38K
full	~15K	~50K	~65K

Agent Workflow Budget

Phase	Tokens	Purpose
ORIENT	~15K	Read X-Ray markdown
INVESTIGATE	~15-25K	Selective file reads, JSON queries
SYNTHESIZE	~15K	Output document
VALIDATE	~1K	Self-test
Total	~45-55K	Full analysis workflow

Note: Agent reads markdown (~8-15K), queries JSON selectively (~5K). Total agent consumption is typically 15-25K, not the full JSON size.

---

Large Codebase Strategy

For codebases >500 files:

Strategy: Divide and Conquer

# 1. Quick survey first
python xray.py . --preset minimal

# 2. Focus on specific areas
python xray.py ./src/core --preset full
python xray.py ./src/api --preset full

# 3. Or use full scan with selective --no-X flags
python xray.py . --no-logic-maps --no-test-example

Strategy: Prioritize Ruthlessly

1. Full X-Ray scan (get all signals)
2. Investigate ONLY:
   - Top 10 pillars (not all)
   - Hotspots with CC > 20 (not CC > 10)
   - Critical path side effects only
3. Document gaps explicitly

Size-Based Strategy

Codebase Size	Files	Strategy	Investigation Depth
Small	<100	Full	Everything
Medium	100-500	Full	Top signals
Large	500-2000	Prioritized	Top 10 each category
Very Large	>2000	Divide & Conquer	Subsystem focus

---

Configuration

Project Config

Place .xray.json in project root for automatic detection:

{
  "sections": {
    "logic_maps": { "enabled": true, "count": 5 },
    "critical_classes": { "enabled": true, "count": 10 },
    "hazards": true,
    "git": true
  }
}

Generate Template

python xray.py --init-config > .xray.json

---

Files

repo-xray/
├── xray.py                 # Main entry point
├── lib/
│   ├── ast_analysis.py     # Skeleton, complexity, types
│   ├── import_analysis.py  # Dependencies, layers, distance
│   ├── call_analysis.py    # Cross-module calls
│   ├── git_analysis.py     # Risk, coupling, freshness
│   ├── gap_features.py     # Logic maps, hazards, models
│   └── ...
├── formatters/
│   ├── markdown_formatter.py
│   └── json_formatter.py
├── configs/
│   └── presets.json
└── .claude/
    ├── agents/
    │   └── repo_xray.md    # Agent definition
    └── skills/
        └── repo-xray/
            ├── SKILL.md    # This file
            ├── COMMANDS.md # Quick reference
            └── templates/
                └── ONBOARD.md.template

---

Requirements

• Python 3.8+
• No external dependencies (stdlib only)
• Git (for history analysis)