| name | graphify |
| description | any input (code, docs, papers, images) -> knowledge graph -> clustered communities -> static Ontology Studio + JSON + audit report. Use when user asks any question about a codebase, project content, architecture, or file relationships, especially if .graphify/ exists. Provides persistent graph with god nodes, community detection, and BFS/DFS query tools. |
| trigger | /graphify |
/graphify
Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: a static Ontology Studio, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md.
Usage
/graphify # full pipeline on current directory → Obsidian vault
/graphify <path> # full pipeline on specific path
/graphify https://github.com/<owner>/<repo> # clone repo locally, then run the full pipeline
/graphify https://github.com/<owner>/<repo> --branch <branch> # clone a specific branch before graphing
/graphify <path> --scope auto # safe default for code/review repos
/graphify <path> --scope tracked # include newly staged files too
/graphify <path> --all # full recursive folder walk for knowledge bases
/graphify <path> --directed # build directed graph (preserves source→target)
/graphify <path> --mode deep # thorough extraction, richer INFERRED edges
/graphify <path> --pdf-ocr auto # preflight PDFs; OCR scanned/low-text PDFs with mistral-ocr when needed
/graphify <path> --update # incremental - re-extract only new/changed files
/graphify <path> --cluster-only # rerun clustering on existing graph
graphify studio export .graphify/studio # build the self-contained static Ontology Studio (open by serving it with any static file server)
/graphify <path> --svg # also export graph.svg (embeds in Notion, GitHub)
/graphify <path> --graphml # export graph.graphml (Gephi, yEd)
/graphify <path> --neo4j # generate .graphify/cypher.txt for Neo4j
/graphify <path> --neo4j-push bolt://localhost:7687 # push directly to Neo4j
/graphify <path> --mcp # start MCP stdio server for agent access
/graphify <path> --watch # watch folder, auto-rebuild on code changes (no LLM needed)
/graphify <path> --wiki # build agent-crawlable wiki (index.md + one article per community)
graphify wiki describe --graph .graphify/graph.json --mode assistant --targets all # opt-in description sidecars
graphify export wiki --graph .graphify/graph.json --descriptions .graphify/wiki/descriptions.json
graphify export obsidian --graph .graphify/graph.json --descriptions .graphify/wiki/descriptions.json
/graphify <path> --obsidian --obsidian-dir ~/vaults/my-project # write vault to custom path (e.g. existing vault)
/graphify add <url> # fetch URL, save to ./raw, update graph
/graphify add <url> --author "Name" # tag who wrote it
/graphify add <url> --contributor "Name" # tag who added it to the corpus
/graphify migrate-state --dry-run # plan graphify-out -> .graphify migration
/graphify query "<question>" # BFS traversal - broad context
/graphify query "<question>" --dfs # DFS - trace a specific path
/graphify query "<question>" --budget 1500 # cap answer at N tokens
/graphify summary --graph .graphify/graph.json # compact first-hop orientation before deep traversal
/graphify minimal-context --task "review PR" --graph .graphify/graph.json # first review call
/graphify review-delta --files src/auth.ts --graph .graphify/graph.json # review impact for changed files
/graphify review-analysis --files src/auth.ts --graph .graphify/graph.json # blast radius + review views
/graphify recommend-commits --files src/auth.ts,src/session.ts --graph .graphify/graph.json # advisory commit grouping
/graphify scope inspect <path> --scope auto # inspect the resolved file inventory first
/graphify path "AuthModule" "Database" # shortest path between two concepts
/graphify explain "SwinTransformer" # plain-language explanation of a node
Input scope policy
- Default to
--scope auto for codebase and review work. In Git repos this means committed files plus .graphify/memory/*.
- Use
--scope tracked when newly staged files must influence the graph before commit.
- Use
--all only when the user clearly wants a knowledge-base style crawl of docs, notes, papers, screenshots, audio, or video.
- If the repo is dirty or the right scope is unclear, run
graphify scope inspect <path> --scope auto first and summarize what will be included or excluded.
What graphify is for
graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected.
Three things it does that Claude alone cannot:
- Persistent graph - relationships are stored in
.graphify/graph.json and survive across sessions. Ask questions weeks later without re-reading everything.
- Honest audit trail - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented.
- Cross-document surprise - community detection finds connections between concepts in different files that you would never think to ask about directly.
Use it for:
- A codebase you're new to (understand architecture before touching anything)
- A reading list (papers + tweets + notes → one navigable graph)
- A research corpus (citation graph + concept graph in one)
- Your personal /raw folder (drop everything in, let it grow, query it)
What You Must Do When Invoked
If no path was given, use . (current directory). Do not ask the user for a path.
If the path argument starts with https://github.com/ or http://github.com/, treat it as a GitHub URL and run Step 0 before anything else. Replace the target path for all later steps with the resolved local clone path.
Follow these steps in order. Do not skip steps.
Step 0 - Clone GitHub repos when the input is a GitHub URL
GRAPHIFY_BRANCH_FLAG=""
if the original invocation included --branch <name>, set GRAPHIFY_BRANCH_FLAG="--branch <name>"
LOCAL_PATH=$(graphify clone "INPUT_GITHUB_URL" $GRAPHIFY_BRANCH_FLAG)
Use LOCAL_PATH as the input path for all subsequent commands.
Step 1 - Ensure graphify is installed
command -v graphify >/dev/null 2>&1 || npm install -g @sentropic/graphify 2>&1 | tail -3
mkdir -p .graphify
If the install succeeds, print nothing and move straight to Step 2.
Step 2 - Detect files
graphify detect "INPUT_PATH" --scope auto --out .graphify/.graphify_detect.json
Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead:
Corpus: X files · ~Y words
code: N files (.py .ts .go ...)
docs: N files (.md .txt ...)
papers: N files (.pdf ...)
images: N files
video: N files (.mp4 .mp3 ...)
Then act on it:
- If
total_files is 0: stop with "No supported files found in [path]."
- If
skipped_sensitive is non-empty: mention file count skipped, not the file names.
- If the user asked for a knowledge-base style crawl, rerun detection with
--all.
- If newly staged files should count, rerun with
--scope tracked.
- If
total_words > 2,000,000 OR total_files > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding.
- Otherwise: proceed to Step 2.5. It is a safe no-op if no video or PDF files need preprocessing.
Step 2.5 - Prepare semantic detection, including audio/video transcripts and PDF preflight/OCR when needed
Always run this step. It transcribes audio/video when present, runs local PDF preflight for papers, and converts text-layer PDFs locally. For scanned/low-text PDFs, keep two OCR paths explicit: the assistant vision model may interpret the original PDF or extracted image chunks during semantic extraction, and mistral-ocr may be used when OCR is explicitly requested or when a configured MISTRAL_API_KEY is available and preflight detects a scanned/low-text PDF. Do not call Mistral blindly. In auto mode, if no key/provider is available, leave the source PDF in semantic inputs so the assistant path can still handle it. Generated transcripts, PDF Markdown sidecars, and relevant PDF-extracted images are treated as semantic inputs in Step 3.
node -e "
(async () => {
const fs = require('fs');
const { prepareSemanticDetection } = require('@sentropic/graphify');
const detect = JSON.parse(fs.readFileSync('.graphify/.graphify_detect.json', 'utf-8'));
const analysis = fs.existsSync('.graphify/.graphify_analysis.json')
? JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'))
: null;
const { detection: semanticDetect, transcriptPaths, pdfArtifacts } = await prepareSemanticDetection(detect, {
transcriptOutputDir: '.graphify/transcripts',
pdfOutputDir: '.graphify/converted/pdf',
godNodes: (analysis && analysis.gods) || [],
});
fs.writeFileSync('.graphify/.graphify_detect_semantic.json', JSON.stringify(semanticDetect, null, 2));
fs.writeFileSync('.graphify/.graphify_transcripts.json', JSON.stringify(transcriptPaths, null, 2));
fs.writeFileSync('.graphify/.graphify_pdf_ocr.json', JSON.stringify(pdfArtifacts, null, 2));
console.log('Prepared semantic inputs: ' + transcriptPaths.length + ' transcript(s), ' + pdfArtifacts.filter((item) => item.markdownPath).length + ' PDF sidecar(s)');
)().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
"
After semantic preparation:
- use
.graphify/.graphify_detect_semantic.json for semantic cache and semantic extraction
- keep
.graphify/.graphify_detect.json for corpus summary, manifest, and final reporting
Step 3 - Extract entities and relationships
Before starting: note whether --mode deep and --directed were given. You must pass DEEP_MODE=true to every subagent in Step B2 if it was. Track both flags from the original invocation - do not lose them.
This step has two parts: structural extraction (deterministic, free) and semantic extraction (Claude, costs tokens).
Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.
Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers.
Part A - Structural extraction for code files
For any code files detected, run AST extraction in parallel with Part B subagents:
node -e "(async () => {
const fs = require('fs');
const { collectFiles, extract } = require('@sentropic/graphify');
const detect = JSON.parse(fs.readFileSync('.graphify/.graphify_detect.json', 'utf-8'));
let codeFiles = [];
for (const f of (detect.files || {}).code || []) {
codeFiles = codeFiles.concat(collectFiles(f));
}
if (codeFiles.length > 0) {
const result = await extract(codeFiles);
fs.writeFileSync('.graphify/.graphify_ast.json', JSON.stringify(result, null, 2));
console.log(\`AST: \${result.nodes.length} nodes, \${result.edges.length} edges\`);
} else {
fs.writeFileSync('.graphify/.graphify_ast.json', JSON.stringify({nodes:[],edges:[],input_tokens:0,output_tokens:0}));
console.log('No code files - skipping AST extraction');
}
})()"
Part B - Semantic extraction (parallel subagents)
Fast path: If semantic detection found zero docs, papers, images, and transcripts (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do.
MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.
Before dispatching subagents, print a timing estimate:
- Load
total_words and file counts from .graphify/.graphify_detect.json
- Estimate agents needed:
ceil(uncached_non_code_files / 22) (chunk size is 20-25)
- Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit))
- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys"
Step B0 - Check extraction cache first
Before dispatching any subagents, check which files already have cached extraction results:
node -e "
const fs = require('fs');
const { checkSemanticCache } = require('@sentropic/graphify');
const detect = JSON.parse(fs.readFileSync('.graphify/.graphify_detect_semantic.json', 'utf-8'));
const allFiles = [
...((detect.files || {}).document || []),
...((detect.files || {}).paper || []),
...((detect.files || {}).image || []),
];
const [cachedNodes, cachedEdges, cachedHyperedges, uncached] = checkSemanticCache(allFiles);
if (cachedNodes.length || cachedEdges.length || cachedHyperedges.length) {
fs.writeFileSync('.graphify/.graphify_cached.json', JSON.stringify({nodes: cachedNodes, edges: cachedEdges, hyperedges: cachedHyperedges}));
}
fs.writeFileSync('.graphify/.graphify_uncached.txt', uncached.join('\n'));
console.log(\`Cache: \${allFiles.length - uncached.length} files hit, \${uncached.length} files need extraction\`);
"
Only dispatch subagents for files listed in .graphify/.graphify_uncached.txt. If all files are cached, skip to Part C directly.
Step B1 - Split into chunks
Load files from .graphify/.graphify_uncached.txt. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). PDF sidecar Markdown can reference extracted image artifacts under .graphify/converted/pdf/*_images/; when those images contain diagrams, tables, captions, or embedded text that carry meaning, include them as image chunks or describe the delegated OCR/vision output with provenance back to the source PDF. When splitting, group files from the same directory together so related artifacts land in the same chunk and cross-file relationships are more likely to be extracted.
Step B2 - Dispatch ALL subagents in a single message
Call the Agent tool multiple times IN THE SAME RESPONSE - one call per chunk. This is the only way they run in parallel. If you make one Agent call, wait, then make another, you are doing it sequentially and defeating the purpose.
Concrete example for 3 chunks:
[Agent tool call 1: files 1-15]
[Agent tool call 2: files 16-30]
[Agent tool call 3: files 31-45]
All three in one message. Not three separate messages.
Each subagent receives this exact prompt (substitute FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, and DEEP_MODE):
You are a graphify extraction subagent. Read the files listed and extract a knowledge graph fragment.
Output ONLY valid JSON matching the schema below - no explanation, no markdown fences, no preamble.
Files (chunk CHUNK_NUM of TOTAL_CHUNKS):
FILE_LIST
Rules:
- EXTRACTED: relationship explicit in source (import, call, citation, "see §3.2")
- INFERRED: reasonable inference (shared data structure, implied dependency)
- AMBIGUOUS: uncertain - flag for review, do not omit
- Node IDs must stay stable across chunks and reruns. Base them on the entity label or file-relative identity only. Never append chunk counters like `_c1`, `_c2`, `_chunk3`, or similar suffixes.
Code files: focus on semantic edges AST cannot find (call relationships, shared data, arch patterns).
Do not re-extract imports - AST already has those.
Doc/paper files: extract named concepts, entities, citations. For rationale (WHY decisions were made, trade-offs, design intent): store it as a `rationale` attribute on the relevant concept node. Do not create a separate rationale node or fragment node unless it is itself a named concept.
Code files: when adding `calls` edges, source MUST be the caller and target MUST be the callee. Never reverse the direction.
Image files: use vision to understand what the image IS - do not just OCR. For images extracted from PDFs, decode figures, tables, diagrams, captions, and embedded text when they carry meaning; use the assistant vision model by default, or a delegated OCR/vision model when configured, and keep provenance to the source PDF and sidecar.
UI screenshot: layout patterns, design decisions, key elements, purpose.
Chart: metric, trend/insight, data source.
Tweet/post: claim as node, author, concepts mentioned.
Diagram: components and connections.
Research figure: what it demonstrates, method, result.
Handwritten/whiteboard: ideas and arrows, mark uncertain readings AMBIGUOUS.
DEEP_MODE (if --mode deep was given): be aggressive with INFERRED edges - indirect deps,
shared assumptions, latent couplings. Mark uncertain ones AMBIGUOUS instead of omitting.
Semantic similarity: if two concepts in this chunk solve the same problem or represent the same idea without any structural link (no import, no call, no citation), add a `semantically_similar_to` edge marked INFERRED with a confidence_score reflecting how similar they are (0.6-0.95). Examples:
- Two functions that both validate user input but never call each other
- A class in code and a concept in a paper that describe the same algorithm
- Two error types that handle the same failure mode differently
Only add these when the similarity is genuinely non-obvious and cross-cutting. Do not add them for trivially similar things.
Hyperedges: if 3 or more nodes clearly participate together in a shared concept, flow, or pattern that is not captured by pairwise edges alone, add a hyperedge to a top-level `hyperedges` array. Examples:
- All classes that implement a common protocol or interface
- All functions in an authentication flow (even if they don't all call each other)
- All concepts from a paper section that form one coherent idea
Use sparingly — only when the group relationship adds information beyond the pairwise edges. Maximum 3 hyperedges per chunk.
If a file has YAML frontmatter (--- ... ---), copy source_url, captured_at, author,
contributor onto every node from that file.
confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a default:
- EXTRACTED edges: confidence_score = 1.0 always
- INFERRED edges: reason about each edge individually.
Direct structural evidence (shared data structure, clear dependency): 0.8-0.9.
Reasonable inference with some uncertainty: 0.6-0.7.
Weak or speculative: 0.4-0.5. Most edges should be 0.6-0.9, not 0.5.
- AMBIGUOUS edges: 0.1-0.3
Output exactly this JSON (no other text):
{"nodes":[{"id":"filestem_entityname","label":"Human Readable Name","file_type":"code|document|paper|image|concept|rationale","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null,"rationale":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
Step B3 - Collect, cache, and merge
Wait for all subagents. For each result:
- If a subagent returned valid JSON with
nodes and edges, include it and save each file's nodes/edges to the cache
- If a subagent failed or returned invalid JSON, print a warning and skip that chunk - do not abort
If more than half the chunks failed, stop and tell the user.
Save new results to cache. The saveSemanticCache call is wrapped in validateSemanticFragment + sanitizeSemanticFragment so a malformed agent response cannot poison the per-file semantic cache:
node -e "
const fs = require('fs');
const { saveSemanticCache, validateSemanticFragment, sanitizeSemanticFragment } = require('@sentropic/graphify');
const raw = fs.existsSync('.graphify/.graphify_semantic_new.json') ? JSON.parse(fs.readFileSync('.graphify/.graphify_semantic_new.json', 'utf-8')) : {nodes:[],edges:[],hyperedges:[]};
const errors = validateSemanticFragment(raw);
if (errors.length > 0) {
console.error('Refusing to cache invalid semantic fragment: ' + errors.slice(0, 3).join('; '));
process.exit(1);
}
const clean = sanitizeSemanticFragment(raw);
const saved = saveSemanticCache(clean.nodes || [], clean.edges || [], clean.hyperedges || []);
console.log(\`Cached \${saved} files\`);
"
Merge cached + new results into .graphify/.graphify_semantic.json. Each chunk is independently validated; invalid chunks are skipped with a warning rather than crashing the merge (mirrors upstream PR #825):
node -e "
const fs = require('fs');
const { validateSemanticFragment, sanitizeSemanticFragment } = require('@sentropic/graphify');
function loadAndClean(path, label) {
if (!fs.existsSync(path)) return {nodes:[],edges:[],hyperedges:[]};
let raw;
try { raw = JSON.parse(fs.readFileSync(path, 'utf-8')); }
catch (e) { console.warn(\`Skipping invalid \${label} fragment \${path}: \${e.message}\`); return {nodes:[],edges:[],hyperedges:[]}; }
const errs = validateSemanticFragment(raw);
if (errs.length > 0) { console.warn(\`Skipping invalid \${label} fragment \${path}: \` + errs.slice(0, 3).join('; ')); return {nodes:[],edges:[],hyperedges:[]}; }
return sanitizeSemanticFragment(raw);
}
const cached = loadAndClean('.graphify/.graphify_cached.json', 'cached');
const raw = loadAndClean('.graphify/.graphify_semantic_new.json', 'new');
const allNodes = (cached.nodes || []).concat(raw.nodes || []);
const allEdges = (cached.edges || []).concat(raw.edges || []);
const allHyperedges = (cached.hyperedges || []).concat(raw.hyperedges || []);
const seen = new Set();
const deduped = [];
for (const n of allNodes) { if (!seen.has(n.id)) { seen.add(n.id); deduped.push(n); } }
const merged = {
nodes: deduped,
edges: allEdges,
hyperedges: allHyperedges,
input_tokens: raw.input_tokens || 0,
output_tokens: raw.output_tokens || 0,
};
fs.writeFileSync('.graphify/.graphify_semantic.json', JSON.stringify(merged, null, 2));
console.log(\`Extraction complete - \${deduped.length} nodes, \${allEdges.length} edges (\${(cached.nodes||[]).length} from cache, \${(raw.nodes||[]).length} new)\`);
"
Clean up temp files: rm -f .graphify/.graphify_cached.json .graphify/.graphify_uncached.txt .graphify/.graphify_semantic_new.json .graphify/.graphify_detect_semantic.json .graphify/.graphify_transcripts.json .graphify/.graphify_pdf_ocr.json
Part C - Merge AST + semantic into final extraction
The AST side comes from Tree-sitter and is trusted as-is. The semantic side is LLM-generated and must be sanitized one more time before the final merge:
node -e "
const fs = require('fs');
const { sanitizeSemanticFragment } = require('@sentropic/graphify');
const ast = JSON.parse(fs.readFileSync('.graphify/.graphify_ast.json', 'utf-8'));
const sem = sanitizeSemanticFragment(JSON.parse(fs.readFileSync('.graphify/.graphify_semantic.json', 'utf-8')));
const seen = new Set(ast.nodes.map(n => n.id));
const mergedNodes = [...ast.nodes];
for (const n of sem.nodes) { if (!seen.has(n.id)) { mergedNodes.push(n); seen.add(n.id); } }
const mergedEdges = ast.edges.concat(sem.edges);
const mergedHyperedges = sem.hyperedges || [];
const merged = {
nodes: mergedNodes,
edges: mergedEdges,
hyperedges: mergedHyperedges,
input_tokens: sem.input_tokens || 0,
output_tokens: sem.output_tokens || 0,
};
fs.writeFileSync('.graphify/.graphify_extract.json', JSON.stringify(merged, null, 2));
console.log(\`Merged: \${mergedNodes.length} nodes, \${mergedEdges.length} edges (\${ast.nodes.length} AST + \${sem.nodes.length} semantic)\`);
"
Step 4 - Build graph, cluster, analyze, generate outputs
Before starting: if --directed was given, replace each buildFromJson(...) call in the build/export steps below with buildFromJson(..., { directed: true }), and when manually reconstructing graph.json, use new Graph({type: 'directed'}) instead of undirected.
mkdir -p .graphify
node -e "
const fs = require('fs');
const { buildFromJson } = require('@sentropic/graphify');
const { cluster, scoreAll } = require('@sentropic/graphify');
const { godNodes, surprisingConnections, suggestQuestions } = require('@sentropic/graphify');
const { generateReport } = require('@sentropic/graphify');
const { toJson } = require('@sentropic/graphify');
const extraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const detection = JSON.parse(fs.readFileSync('.graphify/.graphify_detect.json', 'utf-8'));
const G = buildFromJson(extraction);
const communities = cluster(G);
const cohesion = scoreAll(G, communities);
const tokens = {input: extraction.input_tokens || 0, output: extraction.output_tokens || 0};
const gods = godNodes(G);
const surprises = surprisingConnections(G, communities);
const labels = new Map(Array.from(communities.keys(), cid => [cid, 'Community ' + cid]));
const questions = suggestQuestions(G, communities, labels);
const report = generateReport(G, communities, cohesion, labels, gods, surprises, detection, tokens, '.', {suggestedQuestions: questions});
fs.writeFileSync('.graphify/GRAPH_REPORT.md', report);
toJson(G, communities, '.graphify/graph.json');
const analysis = {
communities: Object.fromEntries(Array.from(communities.entries(), ([k, v]) => [String(k), v])),
cohesion: Object.fromEntries(Array.from(cohesion.entries(), ([k, v]) => [String(k), v])),
gods,
surprises,
questions,
};
fs.writeFileSync('.graphify/.graphify_analysis.json', JSON.stringify(analysis, null, 2));
if (G.order === 0) {
console.log('ERROR: Graph is empty - extraction produced no nodes.');
console.log('Possible causes: all files were skipped, binary-only corpus, or extraction failed.');
process.exit(1);
}
console.log(\`Graph: \${G.order} nodes, \${G.size} edges, \${communities.size} communities\`);
"
If this step prints ERROR: Graph is empty, stop and tell the user what happened - do not proceed to labeling or visualization.
Replace INPUT_PATH with the actual path.
Step 5 - Label communities and node descriptions (no API key needed)
graphify update handles descriptions + labels automatically. By default (no API key), it runs in assistant/skill mode:
- It emits instruction files to
.graphify/description-instructions/ (one per node batch) and .graphify/label-instructions/communities.md for the host assistant to fill.
- You (the host assistant) read those files and write JSON answers (see below).
- A second
graphify update ingests the completed answers and stamps descriptions + labels onto graph.json.
For community labels, read .graphify/label-instructions/communities.md which lists every community with representative node names. Write 2-5 word names to .graphify/label-instructions/communities.json:
{"0": "Attention Mechanism", "1": "Training Pipeline", "2": "Data Loading"}
For node descriptions, each .graphify/description-instructions/batch-NNN.md file lists nodes with their context. Write a one-sentence description per node to the corresponding batch-NNN.json:
{
"some_node_id": "Resolves the configured ontology profile from graphify.yaml.",
"another_id": "Colonel James Barclay, an antagonist introduced in The Crooked Man."
}
Then re-run graphify update to ingest:
graphify update .
With an API key (headless/automation), descriptions and labels are generated directly in the same graphify update call — no two-step needed. To force assistant mode even with a key: --description-mode assistant / --label-mode assistant. To force direct mode: --description-mode direct / --label-mode direct.
Opt out of either step: --no-description / --no-label.
REQUIRED before the studio/wiki export — describe is NOT optional for a populated studio. There is an asymmetry to know about: the default graphify update flow wires the community-labels pass automatically, but the node-descriptions pass only fills graph.json when you actually complete the assistant two-step (or run with a key). If you skip it, every node ships with description: null and the static studio / wiki export shows empty entity panels silently (field report ia-aero: 620/620 null descriptions). The export now warns on ~0% description coverage and provisionally backfills entity panels from each node's extractor rationale (clearly marked "provisional" in the studio) — but that is a stop-gap, not a real description.
So, for non-null descriptions, the explicit order is:
graphify update . (emits .graphify/description-instructions/ + .graphify/label-instructions/).- Fill both the
batch-NNN.json description answers and communities.json labels.
graphify update . again to ingest — OR, on an already-built/curated graph, graphify describe . (descriptions) + graphify label . (labels) + graphify cite . (grounded node.citations[]) non-destructively.- Then generate the static Ontology Studio (Step 6). Confirm descriptions landed: a 0%-description studio means Step 5 was skipped — go back and run
describe (and cite for non-null citations).
Language attention (per source). Descriptions and community names are written in the language of each node's SOURCE document, detected per node from its label + citations (and the community's sampled node names). A 100%-French corpus yields French descriptions and French community names; a mixed corpus gets each node's description in its own source language — never a random FR/EN mix left to the backend's whim. The directive is injected into every prompt: the assistant batch-NNN.md files (look for the Write every description in <Language> line, or per-node lang= markers on a mixed batch) AND the direct API path. When you (the host assistant) fill an instruction file, honor that language directive — answer in the language the instruction states, per node.
- Default is
--description-lang auto / --label-lang auto (detect per source).
- Force one language for the whole run with a code:
--description-lang fr, --label-lang fr (also en, de, es, it, pt, nl). A forced code overrides detection for every node.
- The ontology profile
default_language (graphify.yaml) is the fallback for nodes with no detectable signal (e.g. proper-noun-only nodes).
graphify describe . --description-lang fr
graphify label . --label-lang fr
graphify update . --description-lang fr --label-lang fr
graphify describe .
To describe an existing curated graph without re-extraction (e.g. after manual curation or corpus build), use graphify describe [path] — the non-destructive counterpart to graphify label [path]:
graphify describe .
graphify describe . --fill-missing
graphify describe . --description-mode direct --description-backend anthropic
graphify describe loads the existing graph.json, runs the same description pipeline as graphify update --description, and writes back — without touching node IDs, edges, communities, or any other node attributes. Only description is added/updated. Use it when graphify update would re-extract and destroy a curated graph.
Ground citations into the graph — graphify cite (alias ground-citations). Symmetric to describe/label, and like them opt-in (NOT auto-run in the default pipeline): it loads the existing graph.json, scans the corpus, and grounds per-entity node.citations[] entries {quote, source_file, source_location} — populating new verbatim citations, not just projecting the ones already present. Heuristic mode is the no-key DEFAULT; --mode heuristic|assistant|api (assistant/api are opt-in recall boosters gated by the SAME verbatim check). It is anti-hallucination by construction: every emitted quote is a verified verbatim substring of the source text — a quote that cannot be relocated in the source is dropped, never invented. It UNIONS with existing citations (never clobbers) and re-runs the shipped aggregation (count + inline top-K + .graphify/ontology/citations.json). Run it BEFORE the studio/wiki export so entity panels ship non-null citations.
graphify cite .
graphify cite . --only-missing
graphify cite . --types person,concept,reference --top-k 6
graphify cite . --dry-run
graphify cite . --mode assistant
Citation density per corpus type (no key needed). describe/update/extract/watch auto-tune two citation knobs from the corpus type they read out of .graphify/.graphify_detect.json (no extra step): the describe-prompt cap (citation snippets grounding each description) and the inline top-K (citations kept per node in graph.json; the full per-entity tail always lands in .graphify/ontology/citations.json). Resolution precedence is CLI flag > corpus-type default > global default (cap 10, K 8). Corpus-type defaults:
| Corpus type | describe cap | inline K |
|---|
| code (code dominates, no docs/papers) | 3 | 3 |
| mixed (the middle default) | 10 | 8 |
| long-document (docs/papers, large word count) | all | 8 |
| entity-corpus (ontology/profile mode) | all | 8 |
You normally rely on the auto-resolved policy. Override it explicitly per detected corpus type when needed — e.g. a long-form prose corpus should ground descriptions on many distinct sources, while a code project stays terse:
graphify describe . --citation-cap all
graphify describe . --citation-cap 3
graphify extract . --citations-top-k 8
graphify update . --citation-cap all --citations-top-k 8
--citation-cap <n|all> is on describe/label/update; --citations-top-k <n> is on extract/update/watch; --description-lang <auto|code> is on describe/update and --label-lang <auto|code> is on label/update. All are flag-only overrides (no key, no config file required) — absent, the corpus-type default (citations) or per-source detection (language) applies. To project a pre-feature graph into the new schema without re-extracting (lower-bound counts; re-extract for true exhaustive counts), run graphify backfill-citations ..
Legacy manual label approach (still works; skip if using the CLI two-step above):
Read .graphify/.graphify_analysis.json. For each community key, look at its node labels and write a 2-5 word name. Then regenerate the report:
node -e "
const fs = require('fs');
const { buildFromJson } = require('@sentropic/graphify');
const { scoreAll } = require('@sentropic/graphify');
const { godNodes, surprisingConnections, suggestQuestions } = require('@sentropic/graphify');
const { generateReport } = require('@sentropic/graphify');
const extraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const detection = JSON.parse(fs.readFileSync('.graphify/.graphify_detect.json', 'utf-8'));
const analysis = JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'));
const G = buildFromJson(extraction);
const communities = Object.fromEntries(Object.entries(analysis.communities).map(([k, v]) => [Number(k), v]));
const cohesion = Object.fromEntries(Object.entries(analysis.cohesion).map(([k, v]) => [Number(k), v]));
const tokens = {input: extraction.input_tokens || 0, output: extraction.output_tokens || 0};
// LABELS - replace these with the names you chose above
const labels = LABELS_DICT;
const questions = suggestQuestions(G, communities, labels);
const report = generateReport(G, communities, cohesion, labels, analysis.gods, analysis.surprises, detection, tokens, '.', {suggestedQuestions: questions});
fs.writeFileSync('.graphify/GRAPH_REPORT.md', report);
fs.writeFileSync('.graphify/.graphify_labels.json', JSON.stringify(Object.fromEntries(Object.entries(labels).map(([k, v]) => [String(k), v]))));
console.log('Report updated with community labels');
"
Replace LABELS_DICT with the actual dict you constructed (e.g. {0: "Attention Mechanism", 1: "Training Pipeline"}).
Replace INPUT_PATH with the actual path.
Step 6 - Generate Obsidian vault (opt-in) + static Ontology Studio
Generate the static Ontology Studio always. Obsidian vault only if --obsidian was explicitly given — skip it otherwise, it generates one file per node.
Node descriptions from graph.json render automatically — graphify update fills node.description on each node (via the description pipeline), and export wiki, export obsidian, and the static studio export now render those descriptions by default with no extra step. You do NOT need wiki describe just to get descriptions in the output.
graphify wiki describe is for RICHER multi-sentence, evidence-grounded descriptions generated by an LLM. It is optional and additive: the sidecar fills only the gaps where no node.description exists. When both exist, the graph.json node.description wins (canonical source of truth).
graphify wiki describe --graph .graphify/graph.json --mode assistant --targets all
graphify wiki describe --graph .graphify/graph.json --mode assistant --targets all --max-nodes 0 --max-communities 0
graphify wiki describe --graph .graphify/graph.json --mode direct --backend openai --targets all
graphify export wiki --graph .graphify/graph.json
graphify export wiki --graph .graphify/graph.json --descriptions .graphify/wiki/descriptions.json
graphify export obsidian --graph .graphify/graph.json --descriptions .graphify/wiki/descriptions.json
Sidecars live under .graphify/wiki/descriptions/ with an index at .graphify/wiki/descriptions.json. They record per-node content hashes (unrelated graph.json changes no longer stale the sidecar), prompt/generator provenance, evidence refs, and cache keys; existing generated sidecars may be reused when a fresh generation does not complete. insufficient_evidence sidecars render no Description section. This never mutates .graphify/graph.json.
If --obsidian was given:
- If
--obsidian-dir <path> was also given, use that path as the vault directory. Otherwise default to .graphify/obsidian.
node -e "
const fs = require('fs');
const { buildFromJson } = require('@sentropic/graphify');
const { toWiki, toCanvas } = require('@sentropic/graphify');
const extraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const analysis = JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'));
const labelsRaw = fs.existsSync('.graphify/.graphify_labels.json') ? JSON.parse(fs.readFileSync('.graphify/.graphify_labels.json', 'utf-8')) : {};
const G = buildFromJson(extraction);
const communities = Object.fromEntries(Object.entries(analysis.communities).map(([k, v]) => [Number(k), v]));
const cohesion = Object.fromEntries(Object.entries(analysis.cohesion).map(([k, v]) => [Number(k), v]));
const labels = Object.fromEntries(Object.entries(labelsRaw).map(([k, v]) => [Number(k), v]));
const obsidianDir = 'OBSIDIAN_DIR'; // replace with --obsidian-dir value, or '.graphify/obsidian' if not given
const n = toWiki(G, communities, obsidianDir, {communityLabels: Object.keys(labels).length ? labels : undefined, cohesion});
console.log(\`Obsidian vault: \${n} notes in \${obsidianDir}/\`);
toCanvas(G, communities, \`\${obsidianDir}/graph.canvas\`, {communityLabels: Object.keys(labels).length ? labels : undefined});
console.log(\`Canvas: \${obsidianDir}/graph.canvas - open in Obsidian for structured community layout\`);
console.log();
console.log(\`Open \${obsidianDir}/ as a vault in Obsidian.\`);
console.log(' Graph view - nodes colored by community (set automatically)');
console.log(' graph.canvas - structured layout with communities as groups');
console.log(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries');
"
Generate the static Ontology Studio (always). It bundles the prebuilt studio SPA (index.html + assets/) with the data artifacts the SPA reads next to it — graph.json, scene.json (with pre-computed force-layout positions), entities.json, reconciliation-candidates.json, and class-hierarchies.json (the latter only when a profile carries a class_hierarchies block):
graphify studio export .graphify/studio
The studio scales to large graphs (WebGL render + pre-computed positions), so there is no node-count cap — even oversized graphs export to the studio. To open it, serve the export dir (.graphify/studio) with any static file server and load index.html in a browser.
Step 7 - Neo4j export (only if --neo4j or --neo4j-push flag)
If --neo4j - generate a Cypher file for manual import:
node -e "
const fs = require('fs');
const { buildFromJson, toCypher } = require('@sentropic/graphify');
const G = buildFromJson(JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8')));
toCypher(G, '.graphify/cypher.txt');
console.log('cypher.txt written - import with: cypher-shell < .graphify/cypher.txt');
"
If --neo4j-push <uri> - push directly to a running Neo4j instance. Ask the user for credentials if not provided:
node -e "
(async () => {
const fs = require('fs');
const { buildFromJson, pushToNeo4j } = require('@sentropic/graphify');
const extraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const analysis = JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'));
const G = buildFromJson(extraction);
const communities = Object.fromEntries(Object.entries(analysis.communities).map(([k, v]) => [Number(k), v]));
const result = await pushToNeo4j(G, {uri: 'NEO4J_URI', user: 'NEO4J_USER', password: 'NEO4J_PASSWORD', communities});
console.log(\`Pushed to Neo4j: \${result.nodes} nodes, \${result.edges} edges\`);
})().catch((e) => {
console.error(\`error: \${e.message}\`);
process.exit(1);
});
"
Replace NEO4J_URI, NEO4J_USER, NEO4J_PASSWORD with actual values. Default URI is bolt://localhost:7687, default user is neo4j. Uses MERGE - safe to re-run without creating duplicates.
Step 7b - SVG export (only if --svg flag)
node -e "
const fs = require('fs');
const { buildFromJson, toSvg } = require('@sentropic/graphify');
const extraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const analysis = JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'));
const labelsRaw = fs.existsSync('.graphify/.graphify_labels.json') ? JSON.parse(fs.readFileSync('.graphify/.graphify_labels.json', 'utf-8')) : {};
const G = buildFromJson(extraction);
const communities = Object.fromEntries(Object.entries(analysis.communities).map(([k, v]) => [Number(k), v]));
const labels = Object.fromEntries(Object.entries(labelsRaw).map(([k, v]) => [Number(k), v]));
toSvg(G, communities, '.graphify/graph.svg', {communityLabels: Object.keys(labels).length ? labels : undefined});
console.log('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs');
"
Step 7c - GraphML export (only if --graphml flag)
node -e "
const fs = require('fs');
const { buildFromJson, toGraphml } = require('@sentropic/graphify');
const extraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const analysis = JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'));
const G = buildFromJson(extraction);
const communities = Object.fromEntries(Object.entries(analysis.communities).map(([k, v]) => [Number(k), v]));
toGraphml(G, communities, '.graphify/graph.graphml');
console.log('graph.graphml written - open in Gephi, yEd, or any GraphML tool');
"
Step 7d - MCP server (only if --mcp flag)
npx graphify serve .graphify/graph.json
This starts a stdio MCP server that exposes tools: query_graph, get_node, get_neighbors, get_community, god_nodes, graph_stats, shortest_path. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live.
To configure in Claude Desktop, add to claude_desktop_config.json:
{
"mcpServers": {
"graphify": {
"command": "npx",
"args": ["graphify", "serve", "/absolute/path/to/.graphify/graph.json"]
}
}
}
Step 8 - Token reduction benchmark (only if total_words > 5000)
If total_words from .graphify/.graphify_detect.json is greater than 5,000, run:
node -e "
const fs = require('fs');
const { runBenchmark, printBenchmark } = require('@sentropic/graphify');
const detection = JSON.parse(fs.readFileSync('.graphify/.graphify_detect.json', 'utf-8'));
const result = runBenchmark('.graphify/graph.json', {corpusWords: detection.total_words});
printBenchmark(result);
"
Print the output directly in chat. If total_words <= 5000, skip silently - the graph value is structural clarity, not token compression, for small corpora.
Step 9 - Save manifest, update cost tracker, clean up, and report
node -e "
const fs = require('fs');
const { saveManifest } = require('@sentropic/graphify');
const detect = JSON.parse(fs.readFileSync('.graphify/.graphify_detect.json', 'utf-8'));
saveManifest(detect.files);
const extract = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const inputTok = extract.input_tokens || 0;
const outputTok = extract.output_tokens || 0;
const costPath = '.graphify/cost.json';
const cost = fs.existsSync(costPath) ? JSON.parse(fs.readFileSync(costPath, 'utf-8')) : {runs: [], total_input_tokens: 0, total_output_tokens: 0};
cost.runs.push({
date: new Date().toISOString(),
input_tokens: inputTok,
output_tokens: outputTok,
files: detect.total_files || 0,
});
cost.total_input_tokens += inputTok;
cost.total_output_tokens += outputTok;
fs.writeFileSync(costPath, JSON.stringify(cost, null, 2));
console.log(\`This run: \${inputTok.toLocaleString()} input tokens, \${outputTok.toLocaleString()} output tokens\`);
console.log(\`All time: \${cost.total_input_tokens.toLocaleString()} input, \${cost.total_output_tokens.toLocaleString()} output (\${cost.runs.length} runs)\`);
"
rm -f .graphify/.graphify_detect.json .graphify/.graphify_extract.json .graphify/.graphify_ast.json .graphify/.graphify_semantic.json .graphify/.graphify_analysis.json
rm -f .graphify/needs_update 2>/dev/null || true
Tell the user (omit the obsidian line unless --obsidian was given):
Graph complete. Outputs in PATH_TO_DIR/.graphify/
studio/ - static Ontology Studio, serve with any static file server and open index.html
GRAPH_REPORT.md - audit report
graph.json - raw graph data
obsidian/ - Obsidian vault (only if --obsidian was given)
Replace PATH_TO_DIR with the actual absolute path of the directory that was processed.
Then paste these sections from GRAPH_REPORT.md directly into the chat:
- God Nodes
- Surprising Connections
- Suggested Questions
Do NOT paste the full report - just those three sections. Keep it concise.
Then immediately offer to explore. Pick the single most interesting suggested question from the report - the one that crosses the most community boundaries or has the most surprising bridge node - and ask:
"The most interesting question this graph can answer: [question]. Want me to trace it?"
If the user says yes, run /graphify query "[question]" on the graph and walk them through the answer using the graph structure - which nodes connect, which community boundaries get crossed, what the path reveals. Keep going as long as they want to explore. Each answer should end with a natural follow-up ("this connects to X - want to go deeper?") so the session feels like navigation, not a one-shot report.
The graph is the map. Your job after the pipeline is to be the guide.
For --update (incremental re-extraction)
Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time.
node -e "
const fs = require('fs');
const { detectIncremental } = require('@sentropic/graphify');
const result = detectIncremental('INPUT_PATH');
const newTotal = result.new_total || 0;
console.log(JSON.stringify(result, null, 2));
fs.writeFileSync('.graphify/.graphify_incremental.json', JSON.stringify(result));
if (newTotal === 0) {
console.log('No files changed since last run. Nothing to update.');
process.exit(0);
}
console.log(\`\${newTotal} new/changed file(s) to re-extract.\`);
"
If new files exist, first check whether all changed files are code files:
node -e "
const fs = require('fs');
const path = require('path');
const result = fs.existsSync('.graphify/.graphify_incremental.json') ? JSON.parse(fs.readFileSync('.graphify/.graphify_incremental.json', 'utf-8')) : {};
const codeExts = new Set(['.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts','.lua','.toc']);
const newFiles = result.new_files || {};
const allChanged = Object.values(newFiles).flat();
const codeOnly = allChanged.every(f => codeExts.has(path.extname(f).toLowerCase()));
console.log('code_only:', codeOnly);
"
If code_only is True: print [graphify update] Code-only changes detected - skipping semantic extraction (no LLM needed), run only Step 3A (AST) on the changed files, skip Step 3B entirely (no subagents), then go straight to merge and Steps 4–8.
If code_only is False (any changed file is a doc/paper/image/video): first prepare transcripts and PDF sidecars if needed, then run the full Steps 3A–3C pipeline as normal.
When code_only is False, run this before Step 3B:
node -e "
(async () => {
const fs = require('fs');
const { prepareSemanticDetection } = require('@sentropic/graphify');
const detect = JSON.parse(fs.readFileSync('.graphify/.graphify_incremental.json', 'utf-8'));
const analysis = fs.existsSync('.graphify/.graphify_analysis.json')
? JSON.parse(fs.readFileSync('.graphify/.graphify_analysis.json', 'utf-8'))
: null;
const { detection: semanticDetect, transcriptPaths, pdfArtifacts } = await prepareSemanticDetection(detect, {
transcriptOutputDir: '.graphify/transcripts',
pdfOutputDir: '.graphify/converted/pdf',
godNodes: (analysis && analysis.gods) || [],
incremental: true,
});
fs.writeFileSync('.graphify/.graphify_incremental_semantic.json', JSON.stringify(semanticDetect, null, 2));
fs.writeFileSync('.graphify/.graphify_transcripts.json', JSON.stringify(transcriptPaths, null, 2));
fs.writeFileSync('.graphify/.graphify_pdf_ocr.json', JSON.stringify(pdfArtifacts, null, 2));
console.log('Prepared semantic inputs: ' + transcriptPaths.length + ' transcript(s), ' + pdfArtifacts.filter((item) => item.markdownPath).length + ' PDF sidecar(s)');
)().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
"
When re-running Step 3B in update mode, use .graphify/.graphify_incremental_semantic.json instead of .graphify/.graphify_detect_semantic.json.
Then:
node -e "
const fs = require('fs');
const Graph = require('graphology');
const { buildFromJson } = require('@sentropic/graphify');
// Load existing graph
const existingData = JSON.parse(fs.readFileSync('.graphify/graph.json', 'utf-8'));
const GExisting = new Graph({type: 'undirected'});
for (const n of existingData.nodes) { const {id, ...a} = n; GExisting.mergeNode(id, a); }
for (const l of existingData.links) { const {source, target, ...a} = l; if (GExisting.hasNode(source) && GExisting.hasNode(target)) try { GExisting.mergeEdge(source, target, a); } catch {} }
// Load new extraction
const newExtraction = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const GNew = buildFromJson(newExtraction);
// Prune nodes from deleted files
const incremental = JSON.parse(fs.readFileSync('.graphify/.graphify_incremental.json', 'utf-8'));
const deleted = new Set(incremental.deleted_files || []);
if (deleted.size > 0) {
const toRemove = GExisting.filterNodes((n, a) => deleted.has(a.source_file));
toRemove.forEach(n => GExisting.dropNode(n));
console.log(\`Pruned \${toRemove.length} ghost nodes from \${deleted.size} deleted file(s)\`);
}
// Merge: new nodes/edges into existing graph
GNew.forEachNode((n, a) => GExisting.mergeNode(n, a));
GNew.forEachEdge((e, a, s, t) => { try { GExisting.mergeEdge(s, t, a); } catch {} });
console.log(\`Merged: \${GExisting.order} nodes, \${GExisting.size} edges\`);
"
Then run Steps 4–8 on the merged graph as normal.
After Step 4, show the graph diff:
node -e "
const fs = require('fs');
const Graph = require('graphology');
const { graphDiff, buildFromJson } = require('@sentropic/graphify');
const oldData = fs.existsSync('.graphify/.graphify_old.json') ? JSON.parse(fs.readFileSync('.graphify/.graphify_old.json', 'utf-8')) : null;
const newExtract = JSON.parse(fs.readFileSync('.graphify/.graphify_extract.json', 'utf-8'));
const GNew = buildFromJson(newExtract);
if (oldData) {
const GOld = new Graph({type: 'undirected'});
for (const n of oldData.nodes) { const {id, ...a} = n; GOld.mergeNode(id, a); }
for (const l of oldData.links) { const {source, target, ...a} = l; if (GOld.hasNode(source) && GOld.hasNode(target)) try { GOld.mergeEdge(source, target, a); } catch {} }
const diff = graphDiff(GOld, GNew);
console.log(diff.summary);
if (diff.new_nodes && diff.new_nodes.length) {
console.log('New nodes:', diff.new_nodes.slice(0, 5).map(n => n.label).join(', '));
}
if (diff.new_edges && diff.new_edges.length) {
console.log('New edges:', diff.new_edges.length);
}
}
"
Before the merge step, save the old graph: cp .graphify/graph.json .graphify/.graphify_old.json
Clean up after: rm -f .graphify/.graphify_old.json .graphify/.graphify_incremental_semantic.json .graphify/.graphify_transcripts.json .graphify/.graphify_pdf_ocr.json
For --cluster-only
Skip Steps 1–3. Load the existing graph from .graphify/graph.json and re-run clustering:
node -e "
const fs = require('fs');
const Graph = require('graphology');
const { cluster, scoreAll } = require('@sentropic/graphify');
const { godNodes, surprisingConnections } = require('@sentropic/graphify');
const { generateReport } = require('@sentropic/graphify');
const { toJson } = require('@sentropic/graphify');
const data = JSON.parse(fs.readFileSync('.graphify/graph.json', 'utf-8'));
const G = new Graph({type: 'undirected'});
for (const n of data.nodes) { const {id, ...a} = n; G.mergeNode(id, a); }
for (const l of data.links) { const {source, target, ...a} = l; if (G.hasNode(source) && G.hasNode(target)) try { G.mergeEdge(source, target, a); } catch {} }
const detection = {total_files: 0, total_words: 99999, needs_graph: true, warning: null,
files: {code: [], document: [], paper: []}};
const tokens = {input: 0, output: 0};
const communities = cluster(G);
const cohesion = scoreAll(G, communities);
const gods = godNodes(G);
const surprises = surprisingConnections(G, communities);
const labels = new Map(Array.from(communities.keys(), cid => [cid, 'Community ' + cid]));
const report = generateReport(G, communities, cohesion, labels, gods, surprises, detection, tokens, '.');
fs.writeFileSync('.graphify/GRAPH_REPORT.md', report);
toJson(G, communities, '.graphify/graph.json');
const analysis = {
communities: Object.fromEntries(Array.from(communities.entries(), ([k, v]) => [String(k), v])),
cohesion: Object.fromEntries(Array.from(cohesion.entries(), ([k, v]) => [String(k), v])),
gods,
surprises,
};
fs.writeFileSync('.graphify/.graphify_analysis.json', JSON.stringify(analysis, null, 2));
console.log(\`Re-clustered: \${communities.size} communities\`);
"
Then run Steps 5–9 as normal (label communities, generate viz, benchmark, clean up, report).
For /graphify query
Two traversal modes - choose based on the question:
| Mode | Flag | Best for |
|---|
| BFS (default) | (none) | "What is X connected to?" - broad context, nearest neighbors first |
| DFS | --dfs | "How does X reach Y?" - trace a specific chain or dependency path |
First check the graph exists:
node -e "
const fs = require('fs');
if (!fs.existsSync('.graphify/graph.json')) {
console.log('ERROR: No graph found. Run /graphify <path> first to build the graph.');
process.exit(1);
}
"
If it fails, stop and tell the user to run /graphify <path> first.
Step 0 - Constrained query expansion (before traversal)
graphify query matches nodes by case-folded substring + IDF - no stemming, no synonyms, no cross-language match. When the question uses different vocabulary than the graph labels (user says "обработчик" / graph says "handler"; "authentication" / "Guardian"), the literal matcher returns 0 hits. Expand the query against the actual graph vocabulary first - never invent tokens:
node -e "
const fs = require('fs');
const g = JSON.parse(fs.readFileSync('.graphify/graph.json','utf-8'));
const vocab = new Set();
for (const n of (g.nodes || [])) {
for (const c of String(n.label || '').match(/[^\W\d_]+/gu) || []) {
for (const p of (c.match(/[A-Z]+(?=[A-Z][a-z])|[A-Z]?[a-z]+|[A-Z]+/g) || [c])) {
if (p.length > 1) vocab.add(p.toLowerCase());
}
}
}
fs.writeFileSync('.graphify/.vocab.txt', [...vocab].sort().join('\n'));
console.log('vocab: ' + vocab.size + ' tokens');
"
Read .graphify/.vocab.txt, then pick up to 12 tokens from that exact list that match the query intent. Hard constraints:
- Use only tokens present in
.vocab.txt - do not invent tokens.
- A concept with no plausible vocab token: skip it, no near-synonym from memory.
- No vocab token matches at all: output an empty list and tell the user the corpus has no relevant vocabulary; do not fabricate a search.
- Cross-language: e.g. Russian "аутентификация" - look for
auth, credential, token, security iff present in the vocab.
Print the selection before querying (Query expanded to (from graph vocab, N tokens): [...]), then run graphify query with the joined expanded tokens (keep the original question only for save-result).
Before deep traversal, run the compact first-hop summary and use it to choose the right graph action:
graphify summary --graph .graphify/graph.json
graphify recommend-commits --files src/auth.ts,src/session.ts --graph .graphify/graph.json
graphify review-analysis --files src/auth.ts --graph .graphify/graph.json
graphify review-eval --cases .graphify/review-cases.json --graph .graphify/graph.json
Load .graphify/graph.json, then:
- Find the 1-3 nodes whose label best matches key terms in the question.
- Run the appropriate traversal from each starting node.
- Read the subgraph - node labels, edge relations, confidence tags, source locations.
- Answer using only what the graph contains. Quote
source_location when citing a specific fact.
- If the graph lacks enough information, say so - do not hallucinate edges.
node -e "
const fs = require('fs');
const Graph = require('graphology');
const data = JSON.parse(fs.readFileSync('.graphify/graph.json', 'utf-8'));
const G = new Graph({type: 'undirected'});
for (const n of data.nodes) { const {id, ...a} = n; G.mergeNode(id, a); }
for (const l of data.links) { const {source, target, ...a} = l; if (G.hasNode(source) && G.hasNode(target)) try { G.mergeEdge(source, target, a); } catch {} }
const question = 'QUESTION';
const mode = 'MODE'; // 'bfs' or 'dfs'
const terms = question.split(/\s+/).filter(t => t.length > 3).map(t => t.toLowerCase());
// Find best-matching start nodes
const scored = [];
G.forEachNode((nid, ndata) => {
const label = (ndata.label || '').toLowerCase();
const score = terms.filter(t => label.includes(t)).length;
if (score > 0) scored.push([score, nid]);
});
scored.sort((a, b) => b[0] - a[0]);
const startNodes = scored.slice(0, 3).map(s => s[1]);
if (!startNodes.length) {
console.log('No matching nodes found for query terms:', terms);
process.exit(0);
}
const subgraphNodes = new Set();
const subgraphEdges = [];
if (mode === 'dfs') {
const visited = new Set();
const stack = [...startNodes].reverse().map(n => [n, 0]);
while (stack.length) {
const [node, depth] = stack.pop();
if (visited.has(node) || depth > 6) continue;
visited.add(node);
subgraphNodes.add(node);
G.forEachNeighbor(node, neighbor => {
if (!visited.has(neighbor)) {
stack.push([neighbor, depth + 1]);
subgraphEdges.push([node, neighbor]);
}
});
}
} else {
let frontier = new Set(startNodes);
startNodes.forEach(n => subgraphNodes.add(n));
for (let i = 0; i < 3; i++) {
const nextFrontier = new Set();
for (const n of frontier) {
G.forEachNeighbor(n, neighbor => {
if (!subgraphNodes.has(neighbor)) {
nextFrontier.add(neighbor);
subgraphEdges.push([n, neighbor]);
}
});
}
nextFrontier.forEach(n => subgraphNodes.add(n));
frontier = nextFrontier;
}
}
const tokenBudget = BUDGET; // default 2000
const charBudget = tokenBudget * 4;
const relevance = nid => {
const label = (G.getNodeAttributes(nid).label || '').toLowerCase();
return terms.filter(t => label.includes(t)).length;
};
const rankedNodes = [...subgraphNodes].sort((a, b) => relevance(b) - relevance(a));
const lines = [\`Traversal: \${mode.toUpperCase()} | Start: \${JSON.stringify(startNodes.map(n => G.getNodeAttribute(n, 'label') || n))} | \${subgraphNodes.size} nodes\`];
for (const nid of rankedNodes) {
const d = G.getNodeAttributes(nid);
lines.push(\` NODE \${d.label || nid} [src=\${d.source_file || ''} loc=\${d.source_location || ''}]\`);
}
for (const [u, v] of subgraphEdges) {
if (subgraphNodes.has(u) && subgraphNodes.has(v)) {
const edge = G.hasEdge(u, v) ? G.getEdgeAttributes(G.edge(u, v)) : {};
lines.push(\` EDGE \${G.getNodeAttribute(u, 'label') || u} --\${edge.relation || ''} [\${edge.confidence || ''}]--> \${G.getNodeAttribute(v, 'label') || v}\`);
}
}
let output = lines.join('\n');
if (output.length > charBudget) {
output = output.slice(0, charBudget) + \`\n... (truncated at ~\${tokenBudget} token budget - use --budget N for more)\`;
}
console.log(output);
"
Replace QUESTION with the user's actual question, MODE with bfs or dfs, and BUDGET with the token budget (default 2000, or whatever --budget N specifies). Then answer based on the subgraph output above.
After writing the answer, save it back into the graph so it improves future queries:
node -e "
const { saveQueryResult } = require('@sentropic/graphify');
saveQueryResult({
question: 'QUESTION',
answer: 'ANSWER',
memoryDir: '.graphify/memory',
queryType: 'query',
sourceNodes: SOURCE_NODES, // list of node labels cited, or []
});
console.log('Query result saved to .graphify/memory/');
"
Replace QUESTION with the question, ANSWER with your full answer text, SOURCE_NODES with the list of node labels you cited. This closes the feedback loop: the next --update will extract this Q&A as a node in the graph.
For /graphify path
Find the shortest path between two named concepts in the graph.
First check the graph exists:
node -e "
const fs = require('fs');
if (!fs.existsSync('.graphify/graph.json')) {
console.log('ERROR: No graph found. Run /graphify <path> first to build the graph.');
process.exit(1);
}
"
If it fails, stop and tell the user to run /graphify <path> first.
node -e "
const fs = require('fs');
const Graph = require('graphology');
const { bidirectional } = require('graphology-shortest-path/unweighted');
const data = JSON.parse(fs.readFileSync('.graphify/graph.json', 'utf-8'));
const G = new Graph({type: 'undirected'});
for (const n of data.nodes) { const {id, ...a} = n; G.mergeNode(id, a); }
for (const l of data.links) { const {source, target, ...a} = l; if (G.hasNode(source) && G.hasNode(target)) try { G.mergeEdge(source, target, a); } catch {} }
const aTerm = 'NODE_A';
const bTerm = 'NODE_B';
function findNode(term) {
term = term.toLowerCase();
const scored = [];
G.forEachNode((n, a) => {
const label = (a.label || '').toLowerCase();
const score = term.split(/\s+/).filter(w => label.includes(w)).length;
if (score > 0) scored.push([score, n]);
});
scored.sort((a, b) => b[0] - a[0]);
return scored.length && scored[0][0] > 0 ? scored[0][1] : null;
}
const src = findNode(aTerm);
const tgt = findNode(bTerm);
if (!src || !tgt) {
console.log(\`Could not find nodes matching: '\${aTerm}' or '\${bTerm}'\`);
process.exit(0);
}
const path = bidirectional(G, src, tgt);
if (!path) {
console.log(\`No path found between '\${aTerm}' and '\${bTerm}'\`);
} else {
console.log(\`Shortest path (\${path.length - 1} hops):\`);
for (let i = 0; i < path.length; i++) {
const nid = path[i];
const label = G.getNodeAttribute(nid, 'label') || nid;
if (i < path.length - 1) {
const edgeKey = G.edge(nid, path[i + 1]);
const edge = edgeKey ? G.getEdgeAttributes(edgeKey) : {};
console.log(\` \${label} --\${edge.relation || ''}--> [\${edge.confidence || ''}]\`);
} else {
console.log(\` \${label}\`);
}
}
}
"
Replace NODE_A and NODE_B with the actual concept names from the user. Then explain the path in plain language - what each hop means, why it's significant.
After writing the explanation, save it back:
node -e "
const { saveQueryResult } = require('@sentropic/graphify');
saveQueryResult({
question: 'Path from NODE_A to NODE_B',
answer: 'ANSWER',
memoryDir: '.graphify/memory',
queryType: 'path_query',
sourceNodes: PATH_NODES, // list of node labels on the path
});
console.log('Path result saved to .graphify/memory/');
"
For /graphify explain
Give a plain-language explanation of a single node - everything connected to it.
First check the graph exists:
node -e "
const fs = require('fs');
if (!fs.existsSync('.graphify/graph.json')) {
console.log('ERROR: No graph found. Run /graphify <path> first to build the graph.');
process.exit(1);
}
"
If it fails, stop and tell the user to run /graphify <path> first.
node -e "
const fs = require('fs');
const Graph = require('graphology');
const data = JSON.parse(fs.readFileSync('.graphify/graph.json', 'utf-8'));
const G = new Graph({type: 'undirected'});
for (const n of data.nodes) { const {id, ...a} = n; G.mergeNode(id, a); }
for (const l of data.links) { const {source, target, ...a} = l; if (G.hasNode(source) && G.hasNode(target)) try { G.mergeEdge(source, target, a); } catch {} }
const term = 'NODE_NAME';
const termLower = term.toLowerCase();
const scored = [];
G.forEachNode((n, a) => {
const label = (a.label || '').toLowerCase();
const score = termLower.split(/\s+/).filter(w => label.includes(w)).length;
if (score > 0) scored.push([score, n]);
});
scored.sort((a, b) => b[0] - a[0]);
if (!scored.length || scored[0][0] === 0) {
console.log(\`No node matching '\${term}'\`);
process.exit(0);
}
const nid = scored[0][1];
const dataN = G.getNodeAttributes(nid);
console.log(\`NODE: \${dataN.label || nid}\`);
console.log(\` source: \${dataN.source_file || 'unknown'}\`);
console.log(\` type: \${dataN.file_type || 'unknown'}\`);
console.log(\` degree: \${G.degree(nid)}\`);
console.log();
console.log('CONNECTIONS:');
G.forEachNeighbor(nid, (neighbor) => {
const edgeKey = G.edge(nid, neighbor);
const edge = edgeKey ? G.getEdgeAttributes(edgeKey) : {};
const nlabel = G.getNodeAttribute(neighbor, 'label') || neighbor;
console.log(\` --\${edge.relation || ''}--> \${nlabel} [\${edge.confidence || ''}] (\${G.getNodeAttribute(neighbor, 'source_file') || ''})\`);
});
"
Replace NODE_NAME with the concept the user asked about. Then write a 3-5 sentence explanation of what this node is, what it connects to, and why those connections are significant. Use the source locations as citations.
After writing the explanation, save it back:
node -e "
const { saveQueryResult } = require('@sentropic/graphify');
saveQueryResult({
question: 'Explain NODE_NAME',
answer: 'ANSWER',
memoryDir: '.graphify/memory',
queryType: 'explain',
sourceNodes: ['NODE_NAME'],
});
console.log('Explanation saved to .graphify/memory/');
"
For /graphify add
Fetch a URL and add it to the corpus, then update the graph.
node -e "
(async () => {
const { ingest } = require('@sentropic/graphify');
const out = await ingest('URL', './raw', {author: 'AUTHOR', contributor: 'CONTRIBUTOR'});
console.log(\`Saved to \${out}\`);
})().catch((e) => {
console.error(\`error: \${e.message}\`);
process.exit(1);
});
"
Replace URL with the actual URL, AUTHOR with the user's name if provided, CONTRIBUTOR likewise. If the command exits with an error, tell the user what went wrong - do not silently continue. After a successful save, automatically run the --update pipeline on ./raw to merge the new file into the existing graph.
Supported URL types (auto-detected):
- Twitter/X → fetched via oEmbed, saved as
.md with tweet text and author
- arXiv → abstract + metadata saved as
.md
- YouTube / video URLs → audio downloaded locally via
yt-dlp; transcript generated on the next build/update (requires local yt-dlp, ffmpeg, and faster-whisper-ts)
- PDF → downloaded as
.pdf
- Images (.png/.jpg/.webp) → downloaded, Claude vision extracts on next run
- Any webpage → converted to markdown via html2text
For --watch
Start a background watcher that monitors a folder and auto-updates the graph when files change.
npx graphify watch INPUT_PATH --debounce 3
Replace INPUT_PATH with the folder to watch. Behavior depends on what changed:
- Code files only (.py, .ts, .go, etc.): re-runs AST extraction + rebuild + cluster immediately, no LLM needed.
graph.json and GRAPH_REPORT.md are updated automatically.
- Docs, papers, or images: writes a
.graphify/needs_update flag and prints a notification to run /graphify --update (LLM semantic re-extraction required).
Debounce (default 3s): waits until file activity stops before triggering, so a wave of parallel agent writes doesn't trigger a rebuild per file.
Press Ctrl+C to stop.
For agentic workflows: run --watch in a background terminal. Code changes from agent waves are picked up automatically between waves. If agents are also writing docs or notes, you'll need a manual /graphify --update after those waves.
For git commit hook
Install a post-commit hook that auto-rebuilds the graph after every commit. No background process needed - triggers once per commit, works with any editor.
graphify hook install
graphify hook uninstall
graphify hook status
After every git commit, the hook detects which code files changed (via git diff HEAD~1), re-runs AST extraction on those files, and rebuilds graph.json and GRAPH_REPORT.md. Doc/image changes are ignored by the hook - run /graphify --update manually for those.
If a post-commit hook already exists, graphify appends to it rather than replacing it.
For native CLAUDE.md integration
Run once per project to make graphify always-on in Claude Code sessions:
graphify claude install
This writes a ## graphify section to the local CLAUDE.md that instructs Claude to check the graph before answering codebase questions and rebuild it after code changes. No manual /graphify needed in future sessions.
graphify claude uninstall
For /graphify agent-stats
Attribute branches, commits, and work-packages to the agentic-CLI session that produced them, by indexing conversation transcripts already on disk (Claude ~/.claude/projects/, Codex ~/.codex/sessions/, agy/Antigravity ~/.gemini/). Use it when git authorship is uniform/uninformative (e.g. every commit is the same human author with no agent trailer) and you need to know which agent did what. Attribution is ranked evidence (commit-sha printed in tool output > Codex thread-ids > h2a registry > worktree×branch×time-window > PR-merge), never the git author. Citation excerpts are anonymized before they leave the store.
graphify agent-stats
graphify agent-stats report [--agent <id>]
graphify agent-stats sync [--full]
graphify agent-stats sessions [--agent <id>] [--branch <b>] [--since <date>] [--json]
graphify agent-stats wp <trackItemId> [--no-pr] [--json]
graphify agent-stats project-graph [--config <id.json>] [--out <graph.json>] [--studio]
The store (.graphify/agents/facts.jsonl + byte-offset cursors) is fully re-derivable; sync is incremental. The reports emit stable graphify.agent-stats/v1 (summary/report) and graphify.agent-stats.sessions/v1 (sessions) schemas. wp optionally uses gh to add PR-merge attribution (--no-pr to skip).
project-graph turns those session facts into a graphify graph.json the studio can render (nodes: project / repo / agent / session / branch / commit; edges: belongs-to, rename-lineage, worked-in, conducted-by, touched-branch, produced, derived-from). Its point is rename reconciliation: agent-stats keys repo identity off the cwd path, so a renamed/moved project fragments into several path identities; a ProjectIdentity (canonical id + ordered path/remote aliases) collapses them into ONE project node and chains the incarnations with rename-lineage edges. --config supplies the identity (defaults to the sentropic→graphify lineage); --studio also exports a renderable static studio. Stable schema graphify.agent-stats.project-graph/v1.
Honesty Rules
- Never invent an edge. If unsure, use AMBIGUOUS.
- Never skip the corpus check warning.
- Always show token cost in the report.
- Never hide cohesion scores behind symbols - show the raw number.
- The static Ontology Studio scales to large graphs (WebGL + pre-computed positions) - export it for graphs of any size; no node-count cap.
Configured Project Profiles
The profile activation rule is explicit: use this branch only when graphify.yaml, graphify.yml, .graphify/config.yaml, or .graphify/config.yml exists, or the invocation includes --config or --profile. If none is active, fallback to the existing non-profile workflow.
Configured profile workflow:
- Keep the TypeScript runtime proof in
.graphify/.graphify_runtime.json; it must contain "runtime": "typescript".
- Run
project-config to normalize config/profile artifacts.
- Run the
configured-dataprep runtime command to produce .graphify/profile/profile-state.json, semantic detection, and registry extraction.
- Run the
profile-prompt runtime command and use that prompt for assistant semantic extraction.
- Run base extraction validation, then the
profile-validate-extraction runtime command.
- Merge
.graphify/profile/registry-extraction.json with AST and semantic extraction, then finalize through the existing build/report/export runtime commands.
- Run the
profile-report runtime command to write .graphify/profile/profile-report.md.
- If ontology discovery is requested, run
profile-discovery-sample, use its prompt to produce .graphify/ontology/discovery/proposals.json, then run profile-discovery-diff; present the diff/report to the user and wait for approval before any apply step.
- If
dataprep.image_analysis.enabled is true, use image-calibration-samples and image-calibration-replay for calibration. The assistant may propose labels or rule changes, but TypeScript replay owns acceptance.
- For batch image analysis, use
image-batch-export and image-batch-import. A deep-pass export is allowed only when project-owned routing rules declare decision: accept_matrix; do not make production route decisions in the assistant.
- If the profile declares
outputs.ontology.enabled: true, run ontology-output to compile .graphify/ontology/ after validated extraction exists.
Ontology Lifecycle Patches
Use ontology lifecycle commands only when profile artifacts and .graphify/ontology/ outputs already exist. Review decisions are patches against project-owned sources, not direct graph mutations. Assistants may propose patches, but must validate before dry-run and dry-run before write.
- Validate with
ontology-patch-validate --profile-state .graphify/profile/profile-state.json --patch patch.json.
- Preview with
ontology-patch-apply --profile-state .graphify/profile/profile-state.json --patch patch.json --dry-run.
- Write with
ontology-patch-apply --profile-state .graphify/profile/profile-state.json --patch patch.json --write only after explicit user approval.
- Always warn if the Git worktree is dirty before proposing a write apply.
- Agents must not edit
.graphify/graph.json or derived .graphify/ontology/*.json directly.
- The default MCP server stays read-only; mutation tools require explicit
graphify ontology serve --config graphify.yaml --write.
- Use the Public Domain Mystery Sagas repo as an external UAT and UI-mock corpus only; do not add its real corpus as Graphify package fixtures.
