amg-retrieve

name

amg-retrieve

description

Assemble a focused context pack from the AMG graph BEFORE doing a coding or docs task, so the model sees the strategic surround (purpose, related code, prior decisions) plus the operational detail — without loading the whole project. USE THIS whenever AMG is active and the user gives a task scoped to part of the project: "work on / fix / extend / refactor X", "how does X work", "what touches X", "continue on the Y feature". Run it first, then work from the pack. It is read-only and safe. Also exposes an eval harness to measure retrieval recall and tune it. Triggers: any task naming a function/module/subsystem/feature; "pull context for", "what's relevant to", "retrieve before we start". Also renders a read-only 3D graph viewer (export_graph.py) on request: "open / show / visualize the memory graph".

AMG Retrieve

Turn a task into a small, high-signal context pack by spreading activation over the graph from the task as a query. Retrieval is query-biased Personalized PageRank: a BM25 lexical pass builds the teleport vector (seeds + relevance), structural edges carry the spread (so multi-hop neighbors that share no words with the query are still reached), and the result is assembled greedily under per-tier token budgets. The math and rationale are in scripts/retrieve.py.

Retrieval is read-only with respect to the graph: it never edits nodes or edges. Its only side effects are writing the pack to .claude/amg/cache/pack.md and appending a co-activation signal for the consolidation pass to fold in later.

When to run

When AMG is active (.claude/amg/config.yml → active: true) and the user gives a task scoped to part of the project. Do this before touching code or docs. Do not dump the codebase into context; let activation pull the relevant subgraph.

Workflow

The default is a direct call: one cheap command whose printed pack becomes your working context — no subagent round-trip. The isolated retriever subagent is the deliberate variant for when the pack itself should NOT enter your window.

1. Frame the query. Use the user's task plus any concrete identifiers it names (function/module/feature). A query like "extend charge retries in the billing card-charge flow" seeds well. If embedding seeding is enabled (see below), close paraphrases also work; without it, seeds are purely lexical, so include the words that actually appear in the code/docs.

2. Run the retriever directly (the default):

   python .claude/skills/amg-retrieve/scripts/retrieve.py "<query>" --store .claude/amg
   

   It prints the tiered pack (your context, in one turn) plus the ranked nodes, and writes the same pack to .claude/amg/cache/pack.md. Recognize the query's intent yourself and pass it as a flag: a history/audit query ("what was X before", "why was it changed") → --intent history; a contradictions query ("show the conflicts") → --intent conflict — either surfaces retired/contradicted nodes that are otherwise pushed down (you classify the intent from meaning, in any language; the script has no keyword list).

   Pick the profile by the query's nature — you are the only one who knows it. A targeted lookup — "where is X", "which file holds Y", a pointer question whose answer is a few locations — add --compact: modest built-in budgets and pointer lines (path:line — name — summary) instead of unfolded bodies, at a fraction of the full pack's size (decision/adr keep their rationale). Entering an unfamiliar topic or subsystem → the full profile (no flag): deep, multi-hop context is exactly what you came for, and compact's smaller periphery can lose it.

   Spawn amg-retriever instead only when the isolation is the point: the user asks the memory a question whose answer is a short distillate (the full pack in your window would be waste), or your context is already crowded. The subagent runs the same command in its own context and returns the pack path plus a 3–5 line summary — at the price of a subagent's fixed per-step overhead, so it is the exception, not the default.

   Spawn amg-retriever-fork (Claude Code only) for a context-informed consult: it is a FORK — it inherits your whole conversation, retrieves in its own window, and returns a distillate judged against everything the session already knows (what the memory adds, confirms, or CONTRADICTS — usually 5–15 lines, but a real contradiction gets the space it needs), while the pack never enters your window. Pass the ASK in the spawn prompt — the question and the judgment form you want (a briefing / a delta / a contradiction check / a revision); seed hints are optional, the fork frames the query itself from the session it inherited. It earns its price at every stage, and the price is honest (the fork's turns re-send the inherited context — mostly prompt-cache reads — traded for a lean window):

Lazy derivation: first touch is synchronous (only if derivation: lazy)

When config.yml → derivation: lazy, the graph may hold nodes that are not yet summarized (a structural skeleton awaiting first use). retrieve.py reports these as stale_in_pack (printed under --- stale in pack ---): the nodes a query just activated that are still stale. Before working from the pack, derive them, so the activated node answers with a real summary instead of an empty one — the lazy mechanism's first-touch guarantee:

1. take the stale_in_pack ids from the retriever's output;
2. spawn an amg-builder on just those units (their work/queue.json / queue-deferred.json entries) → a derived-*.json, and apply it (reconcile.py apply ...) — the same steps 3–4 as bootstrap (see the amg-bootstrap skill);
3. re-read the pack (or re-run retrieve) and proceed.

Under the default eager, stale_in_pack is normally empty, so this is a no-op. This is the read-side half of lazy derivation; the build-side (priority map + background fill) is in the amg-bootstrap skill.

Verify a code claim before you answer (cheap, mandatory)

The pack is memory, not ground truth: a summary can lag the source it points to (refactors happen between consolidations). Confidently-wrong memory is worse than no memory — the model answers convincingly and incorrectly. So before you state anything about code on the strength of the pack, confirm it against the live source.

The pack already flags a node whose trust is in doubt with a ⟨…⟩ suffix: stale (summary may lag), unverified (a code claim not yet checked against source), contradicted (a check failed), low confidence. Treat any flag as a prompt to verify before relying on that node.

verify_claims.py runs the check for you — read-only: the file exists, the symbol is still there, and the content hash still matches what was summarized:

python .claude/skills/amg-retrieve/scripts/verify_claims.py \
    <node-id> [<node-id> ...] --store .claude/amg

It prints verified / stale / contradicted per node. On any conflict the source wins over the summary (current code > a stale summary). A maintenance or CI sweep can stamp the verdict back into the graph with --write; the default run touches nothing.

Verify the specific claims you are about to make — only the claims you actually use, not the whole pack. This is the behavioral half of the trust layer; the schema fields (provenance, confidence, verification) and the marking above carry the rest.

Measuring and tuning recall

Retrieval quality is not a matter of taste — measure it. The eval harness compares the AMG pack against a lexical (RAG-like) top-k baseline and reports recall, precision, and hop-recall (recall on gold nodes that don't lexically match the query — what spreading activation uniquely adds).

# Reproducible demo on a built-in labeled graph:
python .claude/skills/amg-retrieve/scripts/eval_retrieval.py --make-demo /tmp/amg-demo

# On your real graph, with your own labeled cases:
python .claude/skills/amg-retrieve/scripts/eval_retrieval.py \
    --store .claude/amg --cases .claude/amg/evals/cases.json --out results.json

cases.json is a list of {"id", "query", "gold_ids": [...]}. Keep it under the store (.claude/amg/evals/ — the eval_gate.cases default), NOT inside the skills tree: an engine reinstall replaces the skills wholesale and would wipe your labels, while the store is never touched. Label a handful of real tasks with the node ids that should surface, then tune these knobs in config.yml → retrieval against the numbers (never by feel): damping (reach), activation_threshold (pack tightness), token_budget per tier, and relation_priors (how strongly each edge type conducts).

Visualize the graph (3D viewer)

To see the memory's structure — clusters, hubs, conflicts, what links to what — render it as a 3D viewer. It is read-only and offline: ONE self-contained HTML with the graph data and the library inlined, so it opens by double-click with no server and nothing fetched (the graph can hold sensitive project knowledge).

python .claude/skills/amg-retrieve/scripts/export_graph.py --store .claude/amg --open

--open writes .claude/amg/cache/graph.html and opens it; omit it to just write the file. --json [path] instead writes the raw {nodes, links, meta} for external graph tooling (it is not needed by the viewer). Click a node for its full frontmatter and edges; color is by bucket (the arbitration verdicts disputed/rejected highlighted, superseded/stale dimmed), size by degree (hubs read large), with filters (type/status/bucket), search (id/summary), a light/dark toggle, and — on a large graph — a hubs-first mode that expands on click. Tunables (quality, large_graph_mode, large_graph_nodes, raw 3d-force-graph options) live in config.yml → viewer. Read-only w.r.t. the graph (writes only cache/graph.html). Triggers: "open / show / visualize the memory graph" (in any language — match the meaning, not the words, but only when the request explicitly refers to this memory / AMG).

Reference

• scripts/retrieve.py — the retriever (importable retrieve(...) + CLI).
• scripts/export_graph.py — read-only export to a self-contained 3D HTML viewer (or --json).
• scripts/embed.py — OPTIONAL semantic seed enrichment (see below).
• scripts/eval_retrieval.py — recall/precision/hop-recall vs lexical baseline.
• scripts/verify_claims.py — verify a code claim against live source (file/symbol/hash); read-only by default, --write stamps the verification block.
• evals/cases.json — your labeled eval set (template provided).
• Subagent: ../../agents/amg-retriever.md.

Optional: embedding seed enrichment

By default seeding is lexical (BM25). If an embedding backend is installed, retrieval also seeds by meaning, so paraphrased queries still light up the right nodes. It blends into the teleport vector only — the PPR spread and the pack are unchanged — so it is safe and its effect is measurable (run the eval with embeddings.enabled off vs on). Enable in config.yml under retrieval.embeddings; install a backend with pip install model2vec (light) or pip install sentence-transformers. With no backend, retrieval silently stays pure-BM25. Node vectors are cached and recomputed only when a node changes. Self-test: scripts/selftest_embed.py.