graphify

# /graphify Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md. ## Usage ``` /graphify # full pipeline on current directory → Obsidian vault /graphify <path> # full pipeline on specific path /graphify https://github.com/<owner>/<repo> # clone repo then run full pipeline on it /graphify https://github.com/<owner>/<repo> --branch <branch> # clone a specific branch /graphify <url1> <url2> ... # clone multiple repos, build each, merge into one cross-repo graph /graphify <path> --mode deep # thorough extraction, richer INFERRED edges /graphify <path> --update # incremental - re-extract only new/changed files /graphify <path> --directed # build directed graph (preserves edge direction: source→target) /graphify <path> --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify <path> --cluster-only # rerun clustering on existing graph /graphify <path> --no-viz # skip visualization, just report + JSON /graphify <path> --html # (HTML is generated by default - this flag is a no-op) /graphify <path> --svg # also export graph.svg (embeds in Notion, GitHub) /graphify <path> --graphml # export graph.graphml (Gephi, yEd) /graphify <path> --neo4j # generate graphify-out/cypher.txt for Neo4j /graphify <path> --neo4j-push bolt://localhost:7687 # push directly to Neo4j /graphify <path> --mcp # start MCP stdio server for agent access /graphify <path> --watch # watch folder, auto-rebuild on code changes (no LLM needed) /graphify <path> --wiki # build agent-crawlable wiki (index.md + one article per community) /graphify <path> --obsidian --obsidian-dir ~/vaults/my-project # write vault to custom path (e.g. existing vault) /graphify add <url> # fetch URL, save to ./raw, update graph /graphify add <url> --author "Name" # tag who wrote it /graphify add <url> --contributor "Name" # tag who added it to the corpus /graphify query "<question>" # BFS traversal - broad context /graphify query "<question>" --dfs # DFS - trace a specific path /graphify query "<question>" --budget 1500 # cap answer at N tokens /graphify path "AuthModule" "Database" # shortest path between two concepts /graphify explain "SwinTransformer" # plain-language explanation of a node ``` ## What graphify is for Drop any folder of code, docs, papers, images, or video into graphify and get a queryable knowledge graph. Persistent across sessions, honest audit trail (EXTRACTED/INFERRED/AMBIGUOUS), community detection surfaces cross-document connections you wouldn't think to ask about. ## What You Must Do When Invoked If the user invoked `/graphify --help` or `/graphify -h` (with no other arguments), print the contents of the `## Usage` section above verbatim and stop. Do not run any commands, do not detect files, do not default the path to `.`. Just print the Usage block and return. If no path was given, use `.` (current directory). Do not ask the user for a path. If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. ### Step 0 - Clone GitHub repo(s) (only if a GitHub URL was given) **Single repo:** ```bash LOCAL_PATH=$(graphify clone <github-url> [--branch <branch>]) # Use LOCAL_PATH as the target for all subsequent steps ``` **Multiple repos (cross-repo graph):** ```bash # Clone each repo, run the full pipeline on each, then merge graphify clone <url1> # → ~/.graphify/repos/<owner1>/<repo1> graphify clone <url2> # → ~/.graphify/repos/<owner2>/<repo2> # Run /graphify on each local path to produce their graph.json files # Then merge: graphify merge-graphs \ ~/.graphify/repos/<owner1>/<repo1>/graphify-out/graph.json \ ~/.graphify/repos/<owner2>/<repo2>/graphify-out/graph.json \ --out graphify-out/cross-repo-graph.json ``` Graphify clones into `~/.graphify/repos/<owner>/<repo>` and reuses existing clones on repeat runs. Each node in the merged graph carries a `repo` attribute so you can filter by origin. ### Step 1 - Ensure graphify is installed ```bash # Detect the correct Python interpreter (handles uv tool, pipx, venv, system installs) PYTHON="" GRAPHIFY_BIN=$(which graphify 2>/dev/null) # 1. uv tool installs — most reliable on modern Mac/Linux if [ -z "$PYTHON" ] && command -v uv >/dev/null 2>&1; then _UV_PY=$(uv tool run graphifyy python -c "import sys; print(sys.executable)" 2>/dev/null) if [ -n "$_UV_PY" ]; then PYTHON="$_UV_PY"; fi fi # 2. Read shebang from graphify binary (pipx and direct pip installs) if [ -z "$PYTHON" ] && [ -n "$GRAPHIFY_BIN" ]; then _SHEBANG=$(head -1 "$GRAPHIFY_BIN" | tr -d '#!') case "$_SHEBANG" in *[!a-zA-Z0-9/_.-]*) ;; *) "$_SHEBANG" -c "import graphify" 2>/dev/null && PYTHON="$_SHEBANG" ;; esac fi # 3. Fall back to python3 if [ -z "$PYTHON" ]; then PYTHON="python3"; fi if ! "$PYTHON" -c "import graphify" 2>/dev/null; then if command -v uv >/dev/null 2>&1; then uv tool install --upgrade graphifyy -q 2>&1 | tail -3 _UV_PY=$(uv tool run graphifyy python -c "import sys; print(sys.executable)" 2>/dev/null) if [ -n "$_UV_PY" ]; then PYTHON="$_UV_PY"; fi else "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out "$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w', encoding='utf-8').write(sys.executable)" # Save scan root so `graphify update` (no args) knows where to look next time echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` If the import succeeds, print nothing and move straight to Step 2. **In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** ### Step 2 - Detect files ```bash $(cat graphify-out/.graphify_python) -c " import json from graphify.detect import detect from pathlib import Path result = detect(Path('INPUT_PATH')) print(json.dumps(result, ensure_ascii=False)) " > graphify-out/.graphify_detect.json ``` Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead: ``` Corpus: X files · ~Y words code: N files (.py .ts .go ...) docs: N files (.md .txt ...) papers: N files (.pdf ...) images: N files video: N files (.mp4 .mp3 ...) ``` Omit any category with 0 files from the summary. Then act on it: - If `total_files` is 0: stop with "No supported files found in [path]." - If `skipped_sensitive` is non-empty: mention file count skipped, not the file names. - If `total_words` > 2,000,000 OR `total_files` > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding. - Otherwise: proceed directly to Step 2.5 if video files were detected, or Step 3 if not. ### Step 2.5 - Transcribe video / audio files (only if video files detected) Skip this step entirely if `detect` returned zero `video` files. Video and audio files cannot be read directly. Transcribe them to text first, then treat the transcripts as doc files in Step 3. **Strategy:** Read the god nodes from `graphify-out/.graphify_detect.json` (or the analysis file if it exists from a previous run). You are already a language model — write a one-sentence domain hint yourself from those labels. Then pass it to Whisper as the initial prompt. No separate API call needed. **However**, if the corpus has *only* video files and no other docs/code, use the generic fallback prompt: `"Use proper punctuation and paragraph breaks."` **Step 1 - Write the Whisper prompt yourself.** Read the top god node labels from detect output or analysis, then compose a short domain hint sentence, for example: - Labels: `transformer, attention, encoder, decoder` → `"Machine learning research on transformer architectures and attention mechanisms. Use proper punctuation and paragraph breaks."` - Labels: `kubernetes, deployment, pod, helm` → `"DevOps discussion about Kubernetes deployments and Helm charts. Use proper punctuation and paragraph breaks."` Set it as `WHISPER_PROMPT` to use in the next command. **Step 2 - Transcribe:** ```bash GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed $(cat graphify-out/.graphify_python) -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) video_files = detect.get('files', {}).get('video', []) prompt = os.environ.get('GRAPHIFY_WHISPER_PROMPT', 'Use proper punctuation and paragraph breaks.') transcript_paths = transcribe_all(video_files, initial_prompt=prompt) print(json.dumps(transcript_paths, ensure_ascii=False)) " > graphify-out/.graphify_transcripts.json ``` After transcription: - Read the transcript paths from `graphify-out/.graphify_transcripts.json` - Add them to the docs list before dispatching semantic subagents in Step 3B - Print how many transcripts were created: `Transcribed N video file(s) -> treating as docs` - If transcription fails for a file, print a warning and continue with the rest **Whisper model:** Default is `base`. If the user passed `--whisper-model <name>`, set `GRAPHIFY_WHISPER_MODEL=<name>` in the environment before running the command above. ### Step 3 - Extract entities and relationships **Before starting:** note whether `--mode deep` was given. You must pass `DEEP_MODE=true` to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it. This step has two parts: **structural extraction** (deterministic, free) and **semantic extraction** (LLM, costs tokens). **Before dispatching subagents:** check whether `GEMINI_API_KEY` or `GOOGLE_API_KEY` is set. If neither is set, print this one-liner to the user: > Tip: set `GEMINI_API_KEY` or `GOOGLE_API_KEY` to use Gemini for semantic extraction (`pip install 'graphifyy[gemini]'`). Print it once, then continue. If `GEMINI_API_KEY` or `GOOGLE_API_KEY` IS set, use `graphify.llm.extract_corpus_parallel(files, backend="gemini")` for semantic extraction instead of dispatching Claude subagents. The default Gemini model is `gemini-3-flash-preview`; set `GRAPHIFY_GEMINI_MODEL` or pass `--model` in headless CLI flows to override it. > **No other API keys are read.** If `GEMINI_API_KEY`/`GOOGLE_API_KEY` are unset, fall straight through to Claude Code subagent dispatch (Part B below) — the host session itself is the LLM. graphify does **not** read `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or any other provider key from the environment. If a host agent prompts the user for `ANTHROPIC_API_KEY` to run extraction, that prompt is a misread of this skill — ignore it and dispatch subagents as written. **Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.** Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers. #### Part A - Structural extraction for code files For any code files detected, run AST extraction in parallel with Part B subagents: ```bash $(cat graphify-out/.graphify_python) -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path import json code_files = [] detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) for f in detect.get('files', {}).get('code', []): code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)]) if code_files: result = extract(code_files, cache_root=Path('.')) Path('graphify-out/.graphify_ast.json').write_text(json.dumps(result, indent=2, ensure_ascii=False), encoding=\"utf-8\") print(f'AST: {len(result[\"nodes\"])} nodes, {len(result[\"edges\"])} edges') else: Path('graphify-out/.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0}, ensure_ascii=False), encoding=\"utf-8\") print('No code files - skipping AST extraction') " ``` #### Part B - Semantic extraction (parallel subagents) **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.** Before dispatching subagents, print a timing estimate: - Load `total_words` and file counts from `graphify-out/.graphify_detect.json` - Estimate agents needed: `ceil(uncached_non_code_files / 22)` (chunk size is 20-25) - Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit)) - Print: "Semantic extraction: ~N files → X agents, estimated ~Ys" **Step B0 - Check extraction cache first** Before dispatching any subagents, check which files already have cached extraction results: ```bash $(cat graphify-out/.graphify_python) -c " import json from graphify.cache import check_semantic_cache from pathlib import Path detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) all_files = [f for files in detect['files'].values() for f in files] cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files) if cached_nodes or cached_edges or cached_hyperedges: