compile-vault

name	compile-vault
description	Process unprocessed raw sources in the wiki vault into wiki articles with frontmatter, wikilinks, and index updates. Use when new sources appear in raw/ or user says "compile", "process sources", or "update wiki".

Scan the vault's `raw/` directory for unprocessed sources, compile each into wiki articles with typed frontmatter and dense wikilinks, update all domain indexes, and mark sources as processed. This is the core compilation loop of the Karpathy-style LLM knowledge base.

<quick_start>

1. Read CLAUDE.md at vault root for conventions pointer
2. Scan raw/ recursively for files where processed: is false or missing
3. For each unprocessed source, compile wiki articles
4. Update all indexes
5. Report results

# Find unprocessed sources
grep -rl "processed: false" raw/ 2>/dev/null
# Also find files with no processed field

</quick_start>

The vault root is determined by the current working directory. Key paths:

• raw/ — Source archive (articles/, papers/, repos/, datasets/)
• wiki/ — Flat LLM-owned knowledge base
• wiki/_index.md — Master index (alphabetical, every article)
• Domain indexes — files starting with _ in wiki/ (e.g. _agents.md, _models.md, etc.)
• docs/conventions.md — Full schema and rules reference

Read docs/conventions.md for the complete frontmatter schema and compilation rules.

**Step 1: Detect unprocessed sources**

Scan all files in raw/ for frontmatter where processed is false or absent. Report count to user.

If zero unprocessed sources found, report "Wiki is up to date" and stop.

Step 2: Compile each source

For each unprocessed source:

1. Read the source content
2. Determine what wiki articles to create or update:
   • Key concepts, techniques, frameworks → type: concept | technique | framework
   • Papers, reports → type: paper
   • Tools, platforms → type: tool
   • People mentioned → type: person
   • Companies mentioned → type: company
3. For each article, apply the frontmatter schema:

---
title: "Article Title"
type: concept
domains:
  - agents
status: draft
created: YYYY-MM-DD
updated: YYYY-MM-DD
sources:
  - "[[raw/articles/source.md]]"
related:
  - "[[Related Article]]"
---

4. Write articles with dense [[wikilinks]] — minimum 3 per article
5. Check if any existing wiki articles should link back to the new ones (bidirectional linking)
6. Mark the raw source processed: true

Use parallel agents when processing 2+ sources — dispatch one agent per source for speed. Each agent should NOT update indexes (Step 3 handles that centrally).

Step 3: Update all indexes

After all sources are compiled:

1. Read current state of all index files
2. Add new article entries to _index.md (alphabetical, one-liner per article)
3. Add entries to relevant domain indexes based on each article's domains field
4. Update domain index overview prose if it adds useful narrative context

Step 4: Append log entry

Append to wiki/log.md:

## [YYYY-MM-DD] ingest | Source Title

- Compiled X new articles: Article 1, Article 2, ...
- Updated X existing articles
- X new redlinks identified

Step 5: Report

Report to user:

• Sources processed (count + names)
• New wiki articles created
• Existing articles updated
• New redlinks (wikilinked articles that don't exist yet — these are future article candidates)
• Current wiki stats: total articles, stubs, drafts, evergreen

<anti_patterns>

• Don't create shallow stubs for everything. If a source only briefly mentions a person or company, a wikilink redlink is better than an empty stub.
• Don't duplicate content across articles. Each article should have its own angle. If two concepts overlap, link them and explain the relationship.
• Don't update indexes piecemeal. Read all indexes, make all updates, write them back. One pass, no conflicts.
• Don't skip bidirectional linking. When Article A links to Article B, check if Article B should link back. </anti_patterns>

<success_criteria> Compilation is complete when:

• All sources in raw/ have processed: true
• Every new concept, person, tool, company has a wiki article or intentional redlink
• All new articles have valid frontmatter with type, domains, status, sources, related
• All new articles have at least 3 wikilinks
• _index.md lists every wiki article alphabetically
• Relevant domain indexes reference new articles
• User receives a clear report of what changed </success_criteria>