wiki-graph-export

name

wiki-graph-export

description

Export the vault as NODES + EDGES JSON for the MemoryLane frontend graph canvas. Walks moments/, sessions/, entities/{people,places,objects}/, and synthesis/, reads frontmatter, parses [[wikilinks]] for edges, computes a force-directed layout in a 1600×1100 viewport, and writes the result to $OBSIDIAN_VAULT_PATH/_meta/graph.json. Use this skill when the user says "export the graph", "rebuild the graph", "refresh frontend data", "regenerate graph.json", or after a batch of moments has been ingested. Distinct from `wiki-export` (which targets Gephi/Neo4j/GraphML for external graph tools) — this skill produces the in-app constellation viewer's exact data shape.

Wiki Graph Export — Frontend Constellation Data

You are producing the JSON file the MemoryLane frontend (memorylane/knowledge-frontend/data.jsx) reads to render the constellation graph. The output schema is fixed by the frontend — do not invent fields.

Before You Start

1. Resolve config — follow the Config Resolution Protocol in llm-wiki/SKILL.md (.env walk-up → ~/.obsidian-wiki/config → prompt setup). Sets OBSIDIAN_VAULT_PATH. For MemoryLane this resolves to memorylane/knowledge_base/.
2. Read $OBSIDIAN_VAULT_PATH/AGENTS.md — confirm the vault's category-to-frontend-type mapping (the table titled "Frontend type mapping"). If that table is missing, abort and tell the user the vault is not configured for graph export.
3. Pin Python to the project venv. Per tasks/todo.md, MemoryLane uses a single uv-managed venv at memorylane/.venv (Python 3.13). All Python invocations in this skill MUST go through <repo_root>/.venv/bin/python — bare python resolves to whatever is first on $PATH and likely lacks networkx. Resolve <repo_root> once: REPO_ROOT=$(cd "$OBSIDIAN_VAULT_PATH/.." && pwd). Verify the venv has networkx ("$REPO_ROOT/.venv/bin/python" -c "import networkx"). If missing: uv pip install --python "$REPO_ROOT/.venv/bin/python" networkx.

Output Schema (fixed)

Write to $OBSIDIAN_VAULT_PATH/_meta/graph.json:

{
  "nodes": [
    { "id": "<file_basename>", "type": "memory|person|place|object|session|synthesis",
      "label": "<title from frontmatter>", "x": 540, "y": 520 }
  ],
  "edges": [
    ["<src_id>", "<dst_id>"]
  ],
  "generated_at": "<ISO 8601>"
}

x, y are integers in a 1600×1100 logical canvas (matches the frontend's VB_W/VB_H in graph.jsx). Edges are undirected — emit each pair once, lexicographically smaller id first.

CATEGORY_COLORS is NOT exported. It lives in the frontend as a styling constant.

Steps

Step 1: Walk the Vault and Collect Nodes

Use Glob to enumerate, in this order:

Glob pattern	Frontend type
moments/*.md	memory
sessions/*.md	session
synthesis/*.md	synthesis
entities/people/*.md	person
entities/places/*.md	place
entities/objects/*.md	object

For each file:

1. Read the frontmatter (everything between the first --- and the second --- at the top of the file).
2. Filter: skip the file if ingest_status is present and equals pending or failed. Files without ingest_status (entities, synthesis) are always included.
3. Skip any file under _raw/ regardless of glob match — those are debugging artifacts.
4. Extract:
   • id = file basename without .md (e.g. abc123_000142, maya, outdoor-calm-pattern)
   • label = title field from frontmatter; fall back to id if missing
   • type = from the table above

Record everything into a list nodes.

Step 2: Parse Edges from Wikilinks

Wikilinks in the vault reference pages by title (e.g. [[Maya]], [[Riverside Park]]), not by file slug. The contract examples in docs/ingest_contract.md and data.jsx confirm this. Resolving by slug alone drops most edges as false orphans.

2a. Build a title→id resolution map first. While collecting nodes in Step 1, also record title_to_id[title.lower()] = id for every node. Where titles collide (rare — same name across categories), prefer this priority order: entity > moment > synthesis > session. Also record slug_to_id[id] = id as a fallback.

2b. Scan bodies with the alias-aware regex. Use Grep with the pattern \[\[([^\]|]+)(?:\|[^\]]+)?\]\] — the first capture group is the link target; any |display alias is discarded. For each match in file <src_id>'s body:

1. target_raw = capture group 1, trimmed.
2. Strip any leading path segments: target_raw = target_raw.split('/')[-1].
3. Strip a trailing .md if present.
4. Resolve to a node id:
   • First try title_to_id[target_raw.lower()]
   • Then try slug_to_id[target_raw]
   • If neither matches → drop (orphan; counted but not emitted; wiki-lint reports separately).
5. Edge = (src_id, resolved_target_id).

2c. Deduplicate and order. Store edges as a set of (min(a,b), max(a,b)) tuples (undirected). Drop self-loops (a == b). Emit in alphabetical order for diff stability across runs.

Step 3: Compute Layout

Use the project's Python venv (resolved as $REPO_ROOT/.venv/bin/python per "Before You Start"). The layout must be deterministic and stable across runs — fixed seed=42 plus prior-position seeding from any existing _meta/graph.json.

Do NOT pass JSON inline to python -c. JSON containing ", ', or newlines breaks bash quoting in nasty, silent ways. Always go through a temp file.

3a. Write inputs to a temp file. Compose { "nodes": [...], "edges": [...] } (the in-memory data from Steps 1 + 2) and write it to /tmp/wgex-input-$$.json. Also write the prior _meta/graph.json to /tmp/wgex-prior-$$.json if it exists, else write {}.

3b. Run layout via the venv python:

"$REPO_ROOT/.venv/bin/python" - <<'PY' > /tmp/wgex-output-$$.json
import json, sys
import networkx as nx

with open('/tmp/wgex-input-$$.json') as f:
    data = json.load(f)
try:
    with open('/tmp/wgex-prior-$$.json') as f:
        prior = json.load(f)
except Exception:
    prior = {}

current_ids = {n['id'] for n in data['nodes']}

# Seed from prior positions, but ONLY for ids that still exist (drop stale).
# Convert from 1600x1100 viewport coords back to spring_layout's [-1, 1] space.
initial = {}
for n in prior.get('nodes', []):
    if n['id'] in current_ids and 'x' in n and 'y' in n:
        initial[n['id']] = ((n['x'] - 800) / 800.0, (n['y'] - 550) / 550.0)
initial = initial or None  # networkx wants None, not empty dict, when no seed

G = nx.Graph()
for n in data['nodes']:
    G.add_node(n['id'])
for a, b in data['edges']:
    G.add_edge(a, b)

n_nodes = G.number_of_nodes()
if n_nodes == 0:
    print('{}'); sys.exit(0)
if n_nodes == 1:
    only = next(iter(G.nodes()))
    print(json.dumps({only: {'x': 800, 'y': 550}})); sys.exit(0)

pos = nx.spring_layout(G, pos=initial, seed=42, k=0.6, iterations=200)

xs = [p[0] for p in pos.values()]
ys = [p[1] for p in pos.values()]
xmin, xmax = min(xs), max(xs)
ymin, ymax = min(ys), max(ys)
# Guard against degenerate collapse (all points at same x or y)
xspan = (xmax - xmin) or 1.0
yspan = (ymax - ymin) or 1.0

def sx(x): return int(80 + (x - xmin) / xspan * (1600 - 160))
def sy(y): return int(80 + (y - ymin) / yspan * (1100 - 160))

out = {nid: {'x': sx(pos[nid][0]), 'y': sy(pos[nid][1])} for nid in pos}
print(json.dumps(out))
PY

Read /tmp/wgex-output-$$.json (a { id: {x, y} } map) and merge x, y into the nodes list as integers.

3c. Cleanup. rm -f /tmp/wgex-input-$$.json /tmp/wgex-prior-$$.json /tmp/wgex-output-$$.json. Use $$ (PID) so concurrent runs don't collide.

Step 4: Write Output

Compose the final JSON:

{
  "nodes": [...],
  "edges": [...],
  "generated_at": "<ISO 8601 now>"
}

Write to $OBSIDIAN_VAULT_PATH/_meta/graph.json. Create _meta/ if missing.

Step 5: Log

Append a line to $OBSIDIAN_VAULT_PATH/log.md:

- <ISO 8601> | wiki-graph-export | nodes=<N>, edges=<E>, dropped_orphan_edges=<K>

Report to the user: node count by type, edge count, output path. If anything was dropped, say what.

Things NOT to do

• Do not write per-page positions into individual frontmatter. Layout is a vault-level concern; the answer is the _meta/graph.json file, not edits to every moment.
• Do not include pages with ingest_status: pending or failed. The frontend would render half-baked nodes.
• Do not export CATEGORY_COLORS or any other styling. That stays in the frontend.
• Do not reshuffle positions on every run — always seed from the prior _meta/graph.json for stability. Users will notice if the graph swims when they refresh.
• Do not run wiki-graph-export from inside _raw/ or any directory that isn't the vault root.

When to invoke

This skill is a build step, not an interactive flow. It runs:

• After any batch of moments arrives from the §7.5 pipeline (manual trigger after a session ingests).
• As an extra step in daily-update (append to that skill's flow once stable).
• When the user explicitly asks to refresh the graph data ("rebuild the graph", "refresh frontend").

It is NOT invoked per-query. Queries use wiki-ask (which reads the existing _meta/graph.json for activatedNodeIds resolution).