agent-wiki

name	agent-wiki
description	Incremental LLM-friendly wiki generator for Obsidian note vaults. Use when: (1) Building wiki from notes, (2) Ingesting notes to wiki, (3) Obsidian LLM wiki, (4) Incremental knowledge base management. Triggers: 'build wiki from notes', 'ingest notes to wiki', 'Obsidian LLM wiki', 'incremental knowledge base'.

agent-wiki

增量式 Obsidian 笔记仓库 Wiki 生成器，为 LLM 优化的知识库管理工具。

Prerequisites

pip install PyYAML

References (load on demand)

Detailed specs live under references/ in this skill directory — read them only when the task needs them:

File	Read when
references/topic-authoring.md	Authoring/enriching topic pages (type taxonomy, per-type section templates, conflict convention, quality metric detail)
references/homepage.md	Working on wiki/index.md (layout templates, managed cards, REST write-through, optional CSS)
references/site-export.md	Running/debugging gen-site (design system, page anatomy, determinism guarantees)
references/index-schema.md	Consuming/producing index or frontmatter fields (full .wiki-index.json schema, Bases views, capture contract)

Execution

The skill provides a Python CLI with the following subcommands:

# Initialize wiki structure
python scripts/agent_wiki_cli.py init --vault /path/to/vault

# Scan for changed sources
python scripts/agent_wiki_cli.py scan --vault /path/to/vault

# Plan a batched ingest: split pending sources into rounds (default 20/round)
python scripts/agent_wiki_cli.py plan --batch-size 20 --vault /path/to/vault

# Mark a round complete (verifies every doc in the batch was cache-put)
python scripts/agent_wiki_cli.py batch-done --batch 1 --vault /path/to/vault

# Get cache entry for a source
python scripts/agent_wiki_cli.py cache-get <relative-path> --vault /path/to/vault

# Record ingest result
python scripts/agent_wiki_cli.py cache-put <relative-path> --topics topic1.md,topic2.md --vault /path/to/vault

# Clean up deleted sources
python scripts/agent_wiki_cli.py cleanup --vault /path/to/vault

# Get wiki health status
python scripts/agent_wiki_cli.py status --vault /path/to/vault

# Rebuild the retrieval index (wiki/.wiki-index.json) without writing .base files
python scripts/agent_wiki_cli.py index --vault /path/to/vault

# Backfill source_type frontmatter to match each topic's sources[] file formats
python scripts/agent_wiki_cli.py normalize-source-type --vault /path/to/vault

# Generate Obsidian Bases (.base) views: wiki/index.base + <name>.base master table
python scripts/agent_wiki_cli.py gen-base --name sources --vault /path/to/vault

# Register an Agent-authored research report (wiki/queries/<name>.md) and tag kind: query
python scripts/agent_wiki_cli.py save-report <name> --vault /path/to/vault

# Generate per-topic JSON Canvas knowledge graphs under wiki/graphs/ (one topic or all)
python scripts/agent_wiki_cli.py gen-canvas --topic <name> --vault /path/to/vault
python scripts/agent_wiki_cli.py gen-canvas --all --vault /path/to/vault

# Build/refresh the wiki/index.md skeleton + its managed "工作区" card block
python scripts/agent_wiki_cli.py gen-home --vault /path/to/vault

# Extract raw 作者 rows from each topic's source notes (read-only)
python scripts/agent_wiki_cli.py extract-authors --vault /path/to/vault

# Deduplicated first-author list per topic, for frontmatter backfill (read-only)
python scripts/agent_wiki_cli.py aggregate-authors --vault /path/to/vault

# Compute quality tier distribution and per-topic metrics (read-only)
python scripts/agent_wiki_cli.py quality --vault /path/to/vault

# Identify covered sources vs gaps (read-only)
python scripts/agent_wiki_cli.py coverage --vault /path/to/vault

# Get maintenance worklists: wanted (broken links) and stale topics (read-only)
python scripts/agent_wiki_cli.py worklist --vault /path/to/vault

# Generate static HTML site (optional, requires markdown package)
python scripts/agent_wiki_cli.py gen-site --vault /path/to/vault

Vault Path Resolution: Use --vault PATH or set environment variable AGENT_WIKI_VAULT.

CLI Command Matrix

Command	Purpose	Input	Output (JSON)
init	Create wiki structure	vault path	{"status": "ok"|"already_initialized", "created": [...]}
scan	Classify sources as new/modified/deleted	vault path	{"version": 1, "vault": "...", "stats": {...}, "new": [...], "modified": [...], "deleted": [...]}
plan	Split pending sources (new+modified) into batches; write task report to wiki/_archived/ingest-tasks.md	vault path, --batch-size (default 20)	{"ok": true, "total": N, "batch_size": N, "report": "...", "batches": [{"id": 1, "status": "pending", "count": N, "items": [...]}]}
batch-done	Mark a round complete after verifying every doc in it was cache-put	vault path, --batch	{"ok": true, "batch": N, "remaining": [...], "complete": bool} or {"error": "batch_incomplete", "missing": [...]}
cache-get	Query cache entry	source relative path	{"path": "...", "sha256": "...", ...} or {"path": "...", "status": "absent"}
cache-put	Record ingest completion	source path, topic list	{"ok": true, "path": "...", "sha256": "..."}; topic paths outside wiki/topics/ → {"error": "invalid_topic_path"}
cleanup	Remove deleted sources from topics	vault path	{"removed": N, "archived": M, "details": [...], "errors": [...]}
status	Wiki health metrics (read-only)	vault path	{"vault": "...", "sources_tracked": N, "topics_total": N, "index_exists": bool, "index_topics": N, "index_stale": bool, "index_errors": [...], "batch": {...}|null, "quality_distribution": {...}, "featured_count": N, "aliases_count": N, "backlinks_max": N, "gaps_count": N, "wanted_count": N, "stale_count": N, "site_exists": bool, "site_stale": bool, ...}
index	Rebuild wiki/.wiki-index.json from topic frontmatter (no .base written)	vault path	{"ok": true, "topics": N, "errors": [...]}
normalize-source-type	Rewrite each topic's source_type frontmatter to its sources[] file format (in place; no-source topics skipped)	vault path	{"ok": true, "changed": [{"path": "...", "source_type": "..."}], "skipped": N, "errors": [...]}
gen-base	Rebuild the index, then write Obsidian Bases views (index + master table)	vault path, --name	{"ok": true, "prefix": "...", "written": [...]}
save-report	Register an Agent-authored research report under wiki/queries/, ensure kind: query, log it	name, vault path	{"ok": true, "path": "queries/<name>.md", "kind": "query"}
gen-canvas	Generate per-topic JSON Canvas 1.0 graph(s) under wiki/graphs/ from the index (topic center + sources[] ring + 1-hop neighbor topics)	vault path, --topic <name> or --all	{"ok": true, "path": "wiki/graphs/<name>.canvas", "nodes": N, "edges": M} or {"ok": true, "written": [...], "count": K}
gen-home	Build/refresh the wiki/index.md skeleton + one managed "工作区" block (Dataview card grid when detected, else static list); refreshes only the managed block on re-run (agent prose preserved); never touches index.base	vault path, --cards auto|on|off (default auto), --no-rest	{"ok": true, "path": "wiki/index.md", "cards": bool, "write_via": "rest|atomic"}
extract-authors	Raw 作者 row per topic source note (read-only)	vault path	{"ok": true, "topics": {"<topic>.md": [{"src": "...", "file": "...", "authors": "..."}]}}
aggregate-authors	Deduplicated first author per topic for frontmatter backfill (read-only)	vault path	{"ok": true, "authors": {"<topic>.md": ["作者1", ...]}}
quality	Compute quality tier distribution and metrics per topic (read-only)	vault path	{"ok": true, "tiers": {"<topic>.md": {"tier": "...", "metrics": {...}}}, "distribution": {"stub": N, ...}, "errors": [...]}
coverage	Identify covered sources vs gaps (read-only)	vault path	{"ok": true, "covered": N, "gaps": [{"path": "..."}], "coverage_ratio": 0.0-1.0}
worklist	Get maintenance worklists: wanted (broken link targets ranked by demand) and stale (low-quality or index-stale topics) for bounded enrichment (read-only)	vault path	{"ok": true, "wanted": [{"target": "...", "inbound": N, "linked_from": [...]}], "stale": [{"path": "...", "tier": "...", "reason": "low_tier"|"index_stale"}]}
gen-site	Generate self-contained static HTML site under wiki/site/ (optional; requires markdown package; degrades gracefully to escaped plaintext if absent)	vault path	{"ok": true, "pages": N, "out": "wiki/site", "degraded": bool, "errors": [...]}

Agent Workflow

Intent Routing

Before any action, classify the user request into one of two modes. Default to Answer mode.

Trigger Signal	Mode	Action	Output
User is asking / seeking explanation / requesting lookup on a topic ("what is…", "compare…", "help me find…") — and NOT requesting wiki building	Answer (default)	Follow Hybrid Retrieval Protocol to answer → optionally save-report as a report	wiki/queries/<name>.md (kind: query), does NOT create/modify topics
User explicitly requests "build / import / ingest / update / maintain wiki", or "create topics from these notes", or points to a vault/directory to be ingested	Ingest/Maintain	Follow Standard / Batched Ingest or Bounded Enrichment	wiki/topics/<name>.md (kind: topic)

Rules:

• Reports are the default output. A regular question never triggers topic generation — unless the user explicitly requests wiki building/maintenance, or explicitly says "make it a topic page".
• Topics are only produced in Ingest/Maintain mode: when ingesting source notes, batch ingesting, or maintaining/enriching existing topics.
• When uncertain which mode applies, treat as Answer and produce a report directly; confirm if wiki building is actually needed.
• Answer mode can read topics/index for retrieval (read-only), but does NOT write topics.

Standard Ingest Loop

1. Scan: Run scan to get new/modified/deleted sources
2. Process each source:
   • For new/modified: Read source → generate/update enriched topic pages → cache-put
   • For deleted: Run cleanup (handles topic frontmatter update and archival)
3. Refresh retrieval index: Run index to rebuild wiki/.wiki-index.json from topic frontmatter
4. Refresh views: Run gen-base to (re)write the Bases views (this also rebuilds the index), then update wiki/index.md with topic summaries and embed ![[index.base#主题总览]]
5. Log: Append to wiki/log.md

Batched Ingest (large vaults)

To avoid loading the whole vault at once, process sources in bounded rounds:

1. Plan: Run plan --batch-size 20 once. It scans, splits the pending sources into rounds of at most N, and writes a checklist report to wiki/_archived/ingest-tasks.md.
2. Process one round: Read only the docs in the current batch, author/update their topic pages, and cache-put each one. Do not read ahead into later batches.
3. Confirm the round: Run batch-done --batch <id>. It refuses (batch_incomplete, listing missing docs) until every doc in the batch is cached, then marks the batch [x] and returns remaining batch ids.
4. Repeat for each remaining batch until complete is true.
5. Finish: Run cleanup (if any deletions), then gen-base, and log as usual.

status reports batch progress under batch. Re-running plan re-derives batches from the current scan — already-ingested docs drop out automatically.

Bounded Enrichment Loop

After initial ingest, maintain topics incrementally without scanning the entire vault:

1. Check worklist: Run worklist for two bounded queues: wanted (broken wikilink targets ranked by inbound demand) and stale (low-quality stub/basic or index-stale topics)
2. Pick one page: Select a single target from wanted (create new topic) or stale (enrich existing)
3. Enrich the page: Read relevant sources, author/update the topic body and frontmatter
4. Re-index: Run index (recomputes quality tiers, backlinks, alias resolution)
5. Repeat: Run worklist again for the updated queue

One page per iteration keeps context bounded; as topics improve, they drop out of stale automatically. status reports wanted_count and stale_count for progress tracking.

Quality Tiering

Topics get a five-tier rating (stub / basic / standard / rich / premium) from structural metrics of the markdown body (sections, evidence lines, script-fair prose weight, images, lead sentence — see references/topic-authoring.md for metric definitions).

Effective prose with source grounding: effective_prose = prose_weight + 500 × unique_source_count — each deduplicated source reference adds a grounding bonus.

Tier gates (top-down first-match):

• premium: sections ≥ 6 AND effective_prose ≥ 3000 AND evidence_lines ≥ 3
• rich: sections ≥ 4 AND effective_prose ≥ 1500 AND (evidence_lines ≥ 1 OR has_image)
• standard: sections ≥ 2 AND effective_prose ≥ 600
• basic: (effective_prose ≥ 200 AND prose_weight > 0) OR sections ≥ 1
• stub: otherwise

Usage: quality reports per-topic metrics and tier distribution; worklist flags stub/basic topics as stale; index recomputes tiers on every rebuild. The formula is deterministic and monotonic — quality and index apply the same source-grounding bonus.

Authors Backfill

When source notes carry a 作者: metadata row: aggregate-authors resolves each topic's sources to root notes and returns the deduplicated first author per topic (read-only). Write the returned lists into each topic's authors frontmatter, then rebuild via index/gen-base. Use extract-authors to inspect raw rows when a result looks off.

Report Capture (research reports)

Persist valuable Agent research reports as first-class, cross-linkable wiki nodes. Capture is passive: the Agent authors the page, then registers it — the CLI writes no prose. This is the default landing spot for Answer mode output.

1. Author the page directly under wiki/queries/<name>.md, with topic-compatible frontmatter (title, sources [may be empty], last_updated, optional summary/keywords). Preserve any [[wikilinks]]/![[embeds]] verbatim.
2. Register it: run save-report <name>. The CLI ensures the kind: query discriminator (directory-derived), appends a log entry, and emits the page path. <name> is sanitized to its final path component with .md ensured.
3. Re-ingest / cross-link: run index (or gen-base) to pick the page up into the retrieval index under queries. To relate a report to a topic, add a [[wikilink]] in either page body — relations are surfaced by gen-canvas.

The CLI touches only wiki/queries/ and wiki/log.md; an uninitialized wiki → wiki_not_initialized, a missing page → capture_not_found, and unparseable frontmatter fails with no write and no log entry.

Web Augmentation & Citations (Supplement when information is insufficient)

When the vault's sources are insufficient to answer fully, supplement with web search — and always cite:

1. Exhaust the vault first: route via the Hybrid Retrieval Protocol and ground in sources[]. Go to the web only for gaps the vault cannot fill.
2. Search the web: use the websearch if available. For pages, prefer defuddle parse <url> --md. Do NOT fetch PDF links — record the URL and link text only.
3. Cite every external claim: inline citation per statement, plus a closing ## 参考来源 section listing each source as - [标题](URL) in citation order. Never present web-derived facts without an attributable URL.
4. Mark provenance: keep vault-grounded and web-supplemented content distinguishable (e.g. > 来源：网络检索). Never fabricate — if neither vault nor web yields an answer, say so explicitly.

Optional Static HTML Export

gen-site generates a self-contained static site under wiki/site/ for local offline browsing — export is opt-in; Obsidian remains the primary interface. Optional markdown package; degrades gracefully to escaped plaintext when absent. Skipped topics are reported in errors. Design system, themes, page anatomy, and determinism guarantees: see references/site-export.md.

Workflow:

1. Run gen-site to generate/refresh the site
2. Check status for site_exists and site_stale (true if any topic is newer than the site)
3. Open wiki/site/index.html directly (fully offline)

Knowledge Graph (Canvas)

gen-canvas renders a deterministic JSON Canvas 1.0 subgraph per topic under wiki/graphs/<topic>.canvas, consumed purely from the retrieval index:

• Scope: the topic at visual center, one node per sources[] entry on an inner ring, and one node per 1-hop neighbor topic on an outer ring.
• Neighbor rule: topics sharing ≥1 sources[] entry ∪ topics the target's body [[wikilinks]] resolve to ∪ topics whose [[wikilinks]] resolve back (by topic-stem), excluding the target.
• Layout: closed-form radial — no randomness; ring radii scale with member count. A vault-file source becomes a clickable file node; an http(s):// source becomes a link node.
• The canvas is a derived, hand-editable artifact, never written back into frontmatter; status.graphs_stale flags topics newer than (or missing) their canvas. Rebuild the index first so neighbors are current.

Homepage (gen-home)

gen-home builds the wiki/index.md skeleton plus one managed "workspace" block (delimited by <!-- agent-wiki:auto start … --> / <!-- … end --> markers): the script owns the skeleton and managed block; the agent writes the semantic prose (regroup topics, fill range, author the relationship narrative). Cards render as a Dataview grid when detected (--cards auto|on|off), else a static list.

Re-run semantics (never clobber): markers present → only the managed block is refreshed (agent prose preserved byte-for-byte); content without markers → block appended; empty/placeholder → full skeleton. index.base is never touched.

Three layout templates (academic / dashboard / magazine) are bundled under templates/home/ in the skill directory — copy one into {vault}/wiki/index.md and fill the _待补充_ placeholders, keeping the auto markers intact.

Details (cards detection, Obsidian Local REST API write-through for open-editor safety, optional CSS): see references/homepage.md. REST env vars: AGENT_WIKI_OBSIDIAN_API_KEY (+ optional AGENT_WIKI_OBSIDIAN_API_URL, default https://127.0.0.1:27124; TLS verification skipped only for loopback hosts) — see .env.example.

Hybrid Retrieval Protocol

Answer questions in two passes — route cheaply, then ground precisely:

1. Route (fast): Read wiki/.wiki-index.json and use indexed fields to identify likely-relevant topics:

   • Alias resolution: Check alias_index first (maps alternative names → canonical topic keys)
   • Primary fields: title, keywords, summary, source_type, sources paths
   • Ranking signals: quality_tier (premium/rich/standard prioritized), backlinks (popularity/centrality), featured flag
   • Do not read every topic file during routing

2. Ground (deep): For detailed evidence, methods, paper data, or comparisons:

   • Follow each topic's sources entries to read the original notes
   • Check topics with high backlinks counts for cross-references
   • Use coverage to verify completeness (identify gaps in source coverage)

3. Conflict rule: If an indexed summary conflicts with source content, the source note is authoritative; correct the topic and rebuild the index on the next ingest pass.

4. Disambiguation: When alias_index lookup fails or returns conflicts, consult .wiki-aliases.json for manual disambiguation mappings. Conflicts are reported but never auto-resolved.

The index is a derived cache: topic frontmatter is the single source of truth. index/gen-base regenerate it from wiki/topics/*.md; status reports index_stale read-only and never rebuilds.

Enriched Topic Authoring

For paper-like sources, populate the common frontmatter fields and write concise body sections for key paper data, experimental methods, technical routes, research trends, and source-grounded evidence when the source supports them. If a source lacks a dimension, omit the field or mark the section unavailable — never fabricate. Preserve existing wikilinks/embeds verbatim; never modify source notes or attachments.

Every topic body MUST open with a single positioning sentence (定位句) before the first ## heading — plain paragraph, no heading/list/quote.

The optional frontmatter type field (concept/method/paper/person/event/place/overview) selects a recommended section structure — taxonomy, per-type section templates, and the conflict-recording convention: see references/topic-authoring.md.

URL Fetching Rules

• Use grok-search or exa skills if available
• PDF links: Do NOT fetch (.pdf extension or Content-Type: application/pdf) — record URL and link text only

Obsidian Wikilink Preservation

• Preserve [[note]] wikilinks and ![[image.png]] embeds verbatim in topic bodies
• In frontmatter sources: [], use relative paths (no [[...]] wrap)

Integration with Obsidian Skills

• Source reading: prefer obsidian read file="..." (captures unsaved editor buffers); fall back to direct file read
• URL fetching: defuddle parse <url> --md (replaces WebFetch for token efficiency)
• Frontmatter updates: prefer obsidian property:set name="..." value="..." file="..."; fall back to direct YAML rewrite
• Dynamic index (Bases): run gen-base to write the two .base views deterministically; embed via ![[index.base#主题总览]]. View columns, faceting, and fallback: see references/index-schema.md

Wiki Structure

{vault}/
├── <name>.base             # Source master table (Bases, at vault root)
└── wiki/
    ├── index.md             # Homepage skeleton (gen-home); agent fills prose, cards auto-render
    ├── index.base           # Topic overview view (Bases)
    ├── log.md               # Append-only log
    ├── topics/              # Topic pages (LLM-written)
    │   └── 量子叠加原理.md
    ├── queries/             # Captured research reports (kind: query)
    ├── graphs/              # Generated JSON Canvas graphs (<topic>.canvas)
    ├── site/                # Optional static HTML export (gen-site)
    ├── _archived/{date}/    # Orphaned topics
    ├── .wiki-cache.json     # Incremental cache
    ├── .wiki-index.json     # Derived retrieval index (normalized metadata)
    └── .wiki-url-cache/     # External URL snapshots (optional)

Topic Page Frontmatter Contract

title, sources, and last_updated are required/compatible; the remaining fields are optional, Agent-authored, and normalized into wiki/.wiki-index.json. source_type is auto-derived from sources[] file formats (never hand-edit; run normalize-source-type):

---
title: 量子叠加原理
type: concept                 # optional page kind
aliases: ["叠加原理"]          # optional alternative names
featured: true                # optional emphasis flag (strict boolean)
sources:
  - "物理/量子力学/态叠加.md"
last_updated: 2026-06-04T15:30:00
summary: 一句话主题摘要，用于索引快速路由。
keywords: ["叠加态", "波函数"]
---

Full field list (year_start/year_end, authors, institutions, methods, technical_routes, research_trends), the derived source_type category table, the complete .wiki-index.json schema, and the capture-page contract: see references/index-schema.md.

Scope Boundaries

This skill includes research-report capture (save-report) and Canvas knowledge-graph generation (gen-canvas). Two boundaries hold: the CLI makes no embedded LLM API calls (all page prose is Agent-authored; the CLI only places, registers, indexes, or renders derived artifacts), and classification/visualization never physically reorganizes topic/query files into per-category folders — they stay flat under wiki/topics/ and wiki/queries/.

Notes

• All paths in cache and frontmatter use NFC-normalized POSIX separators
• Derived topic paths are constrained to wiki/topics/ — cache-put rejects and cleanup reports out-of-bounds entries (invalid_topic_path)
• Concurrent safety: single-process assumption; cache writes are atomic
• Topic pages: Agent should merge with existing content, not overwrite
• No LLM API calls embedded in CLI; all content generation by main Agent