// Karpathy's LLM Wiki — persistent, interlinked markdown knowledge base with auto-ingest (raw/ watcher), auto-dreaming (re-promote frozen pages whose relevance returns), three-tier memory aging, custom frontmatter dimensions, external plugin fallback, and 12 skills from init to watch.
[HINT] Download the complete skill directory including SKILL.md and all related files
| name | ak-wiki |
| description | Karpathy's LLM Wiki — persistent, interlinked markdown knowledge base with auto-ingest (raw/ watcher), auto-dreaming (re-promote frozen pages whose relevance returns), three-tier memory aging, custom frontmatter dimensions, external plugin fallback, multi-machine git sync (custom log.md merge driver), and 13 skills from init to sync. |
| version | 1.8.0 |
| author | litianyi |
| license | MIT |
Build and maintain a persistent, compounding knowledge base as interlinked markdown files. Based on Andrej Karpathy's LLM Wiki pattern.
Unlike traditional RAG (which rediscovers knowledge from scratch per query), the wiki is compiled once and kept current. Cross-references are already there. Contradictions have already been flagged. Synthesis reflects everything ingested.
Obsidian is the IDE. The LLM is the programmer. The wiki is the codebase.
Division of labor: The human curates sources, drops them in raw/, asks good
questions, and edits SCHEMA.md when conventions need to evolve. The human
never (or rarely) writes wiki pages themselves. The LLM does everything else:
reading, summarizing, cross-referencing, filing, maintaining consistency. This is
the whole point — humans abandon wikis because the maintenance burden grows faster
than the value, but LLMs don't get bored.
Every convention — page types, frontmatter fields, tag taxonomy, page creation
policy, cross-reference policy, page size limits, log rotation, custom
frontmatter dimensions, and memory-tier thresholds — lives in SCHEMA.md.
That file is user-editable and co-evolves with the wiki. wiki-init writes
a starter based on the user's domain; wiki-lint proposes updates over time.
This skill set reads and enforces SCHEMA.md rather than hardcoding opinions.
This matters because Karpathy's original is intentionally abstract: "Everything mentioned above is optional and modular — pick what's useful, ignore what isn't." A reading-a-book wiki and a research wiki need different page types and different policies. SCHEMA.md is where that per-domain customization lives.
Beyond standard fields (title, created, updated, type, tags, sources), SCHEMA.md
can declare custom dimensions — extra frontmatter fields specific to the domain.
A software project wiki might declare version: (string, required, prompted on
every ingest). A research wiki might declare venue: or citation_count:.
Each dimension has: name, type (string/date/number/enum/list), description
(prompt text), required, default, refresh_on ([ingest, import, digest, manual]), and optional applies_to (page types). wiki-init sets them up
interactively; wiki-ingest and wiki-import prompt per refresh_on;
wiki-digest surfaces stale values; wiki-lint validates completeness.
SCHEMA.md memory_tiers: controls three-tier aging:
--tier=archived|allTiers are computed on-the-fly from a driving date field (published_at with
ingested_at fallback) — never stored as frontmatter, so threshold adjustments
are instant. A wiki page's tier = most-recent-tier of its cited sources. Manual
tier_override: frontmatter pins are respected. Use wiki-tier to view/adjust.
{wiki_path}/.wiki-plugins.yaml registers CLI tools that wiki-query calls when
local search returns insufficient results. Each plugin declares an argv: list
(per-token, no shell — see PLUGINS.md for the v1.4 spec), a trigger condition
(on_empty|on_low_confidence|on_request), and safety settings.
Flow: wiki-query local miss → plugin command → stdout → raw/external/ →
wiki-ingest → wiki pages → future queries hit local first. The external output
becomes a raw source and enters the full ingest pipeline.
Default is auto_run: false — the agent shows the command and waits for user
confirmation. See PLUGINS.md (or the wiki-query skill) for the full spec.
wiki-init — it asks about your domain and proposes categories that fitgit init in the wiki path (suggested automatically at the end of init)raw/articles/ and run wiki-ingestwiki-digest to see your compiled knowledge growwiki-query; answers file back and compound| Skill | Argument hint | Description |
|---|---|---|
wiki-config | [--show] [--get <path>] [--set <path> <value>] [--explain <path>] [--validate] | Unified read/write for SCHEMA.md config — show all settings in one place, get/set scalar values by… |
wiki-digest | [--since=7d] [--focus=<topic>] [--format=brief|full] [--tier=active|all|archived|frozen] | Generate a digest of the wiki: recent activity summary, key themes by cluster, tier distribution (a… |
wiki-dream | [--since YYYY-MM-DD] [--strategy co-occurrence] [--apply --pages 1,2,3] [--explain <page>] [--out <file>] | Auto-dreaming: re-evaluate frozen and archived pages against recent activity, surface candidates wh… |
wiki-graph | [--query=<expr>] [--neighbors=<page> --depth=N] [--shortest-path=<a>,<b>] [--hubs] [--orphans] [--cluster=<tag>] [--limit=20] [--format=text|json|mermaid] [--tier=active|all|archived|frozen] | Query the wiki as a graph without maintaining a graph DB. Structured frontmatter queries, neighbor… |
wiki-import | <source-path> [--format=folder|obsidian|notion|confluence|markdown] [--map=<mapping-file>] [--dry-run] [--resume] [--priority=recency|links|manual] [--set=key=value,...] [--per-file-prompt] | Import an existing document system (folder tree, Obsidian vault, Notion/Confluence export, etc.) in… |
wiki-ingest | <url|file|text> [--batch] [--no-discuss] [--no-images] [--set=key=value,...] | Ingest a source into the wiki: save raw content and referenced images, prompt the user for any cust… |
wiki-init | [--path=~/wiki] [--domain='AI research'] [--categories='a,b,c'] [--non-interactive] [--set-tags='a,b,c'] [--set-active-days=N] [--set-archived-days=N] [--set-driving-field=published_at|ingested_at] [--set-dimension='name:type:required:refresh_on'] [--enable-dreaming] [--enable-sync] [--refresh-id [--force]] | Interactive bootstrap for a new LLM wiki: ask about domain, propose categories that fit, write a cu… |
wiki-lint | [--fix] [--report-only] [--check=orphans|links|frontmatter|stale|index|tags|size|gaps|schema|tiers|dimensions] | Health-check the wiki: structural checks (orphans, broken links, frontmatter, stale content, tier c… |
wiki-query | <question> [--file] [--format=markdown|table|slides|chart|canvas] [--tier=active|all|archived|frozen] [--external] [--no-external] [--auto-external] | Answer a question using the wiki's compiled knowledge. Searches relevant pages, synthesizes with ci… |
wiki-search | <query> [--tag=<tag>] [--type=entity|concept|comparison|query] [--limit=10] [--tier=active|all|archived|frozen] | Search the wiki by keyword, tag, topic, or type. Returns ranked results with page summaries and mat… |
wiki-sync | [--auto] [--dry-run] | Multi-machine git sync for the wiki: pull, merge with custom drivers (log.md union+sort), push. Loc… |
wiki-tier | [--show] [--set-active=Nd] [--set-archived=Nd] [--set-field=published_at|ingested_at] [--preview] [--pin=<page>:<tier>] [--unpin=<page>] [--list=<tier>] [--disable] [--enable] | Inspect and manage the memory-tier system: view the active/archived/frozen distribution, preview a… |
wiki-watch | [--start [--poll N --debounce N]] [--stop] [--status] [--drain [--pages 1,2,3]] [--remove <id>] | Watch raw/{articles,papers,transcripts,external}/ for new files, queue them, and let the user drain… |
Trigger: "create a wiki", "start my knowledge base", "initialize a wiki for {domain}"
Do not hardcode categories. Ask the user about their domain, propose a category set that fits, let them edit.
WIKI_PATH env → current wiki root →
LLM_WIKI_PROJECT / .llm-wiki.yaml / registry / git root name as
~/.llm-wiki/{repo} →
legacy ~/.ak-wiki/config.yaml → ~/.llm-wiki/common)entities/, concepts/, comparisons/, queries/characters/, places/, events/, themes/, timeline/, chapters/journal/, topics/, patterns/, queries/people/, projects/, customers/, decisions/, meetings/competitors/, features/, market/, comparisons/raw/articles/, raw/papers/, raw/transcripts/, raw/external/, raw/imported/, raw/assets/SCHEMA.md with everything chosen above — mark it "user-editable"index.md (one section per category)log.md with init entrygit init — the wiki is a git repo, version history for freeTrigger: "ingest this", "add this source", "read this and update the wiki"
Arguments: <url|file|text>, --batch, --no-discuss, --no-images
raw/articles/ or raw/papers/ — never modify raw filesraw/assets/{source-stem}-{name} and
rewrite the saved raw source to use local paths (URLs rot; local copies persist)--no-discuss) — surface 3–5 insights
and ask what to emphasize## [YYYY-MM-DD] ingest | Source TitleA single ingest usually touches 10–15 pages. That's the compounding effect.
Trigger: "find pages about X", "search for Y", "what do we have on Z"
Three-pass scan. No orientation needed — runs cold.
Arguments: <query>, --tag=<tag>, --type=<category>, --limit=10
index.md (Karpathy: "reads the index first to find relevant pages")raw/)Scaling up:
wiki-lintqmd (hybrid BM25/vector
with LLM re-rank); wiki-search auto-shells-out when qmd is in PATHqmd in MCP server mode; agent calls directly, bypassing built-in scanKarpathy: "You could also build something simpler yourself — the LLM can help you vibe-code a naive search script as the need arises."
Trigger: "neighbors of X", "shortest path between A and B", "which pages are hubs", "orphans", "Dataview-style filter"
Arguments: [--query=<expr>], [--neighbors=<page> --depth=N], [--shortest-path=<a>,<b>], [--hubs], [--orphans], [--cluster=<tag>], [--limit=20], [--format=text|json|mermaid]
wiki-search handles ranked text relevance. wiki-graph handles structural
queries over frontmatter and the [[wikilink]] graph — with no persistent store.
Every call scans the markdown files fresh, builds an in-memory graph, runs the
query, and exits. The filesystem remains the only source of truth.
Modes:
--query — Dataview-style frontmatter filter. Fields, comparisons on
dates/numbers, AND/OR/NOT, parentheses.
Example: --query "type: entity AND tags contains transformer AND updated > 2025-01"--neighbors <page> — BFS over [[wikilinks]] up to --depth (cap 4).
Best way to see everything adjacent to a topic.--shortest-path <a>,<b> — surface bridge concepts between two
entities. Often the highest-signal mode — finds connections the user didn't
realize were there.--hubs — nodes ranked by |in_edges| + 0.5·|out_edges|. Inbound > outbound.--orphans — isolated nodes (no links at all) vs. leaves (inbound only).
True orphans are usually ingest bugs; leaves are often intentional stubs.--cluster=<tag> — members, anchor, intra- vs. external edges, density.
Answers "is this tag a coherent topic or just a shelf?"Output formats: text (default), json (pipe-friendly), mermaid (paste
into Obsidian or a GitHub README — great for --neighbors, --shortest-path,
--cluster).
Why no persistent graph: a second data store drifts silently out of sync the moment the agent updates a page and forgets to update the graph. Files-as- truth is Karpathy's core discipline. Scanning several hundred pages takes milliseconds — cheaper than keeping an index coherent.
Trigger: "show tier distribution", "push active window to 2 years", "pin this page as active", "which pages just aged out"
Arguments: [--show], [--set-active=Nd], [--set-archived=Nd],
[--set-field=published_at|ingested_at], [--preview],
[--pin=<page>:<tier>], [--unpin=<page>], [--list=<tier>],
[--disable], [--enable]
Manages the three-tier aging system configured in SCHEMA.md memory_tiers:.
--show (default) — print config + distribution table + recently aged-out
pages + pinned overrides--set-active=Nd / --set-archived=Nd — stage a threshold change, show
the delta ("31 pages move archived→active"), ask before writing SCHEMA.md--preview — dry-run a threshold change without writing--pin=<page>:<tier> / --unpin=<page> — write or clear tier_override:
frontmatter on a specific page (for canonical references that shouldn't age out)--list=<tier> — list every page in a given tier--disable / --enable — toggle the tier system in SCHEMA.mdTier changes are safe — they re-interpret existing dates, not mutate content. Auto-dreaming for frozen content is planned for v2 but not implemented yet.
Trigger: "what's in the wiki?", "summarize what we know", "give me an overview"
Arguments: --since=7d, --focus=<topic>, --format=brief|full
Trigger: any question about the wiki's domain
Arguments: <question>, --file, --format=markdown|table|slides|chart|canvas
Like ingestion, substantive query results compound back into the wiki. When an answer is worth keeping, it's filed as a wiki page and joins the knowledge base just like sources do.
wiki-search[[wikilink]] citations; be explicit about gaps; present
contradictions honestlymarp: true frontmatter, --- slide breaks)queries/.canvas JSON (nodes = pages, edges = connections)queries/{name}.md if:
--file flag was passedlog.md: ## [YYYY-MM-DD] query | QuestionTrigger: "lint", "audit", "health-check", "clean up"
Two layers of checks. The content layer is where the LLM earns its keep.
Structural checks (the "well-formed?" layer):
| Severity | Check | Description |
|---|---|---|
| HIGH | Broken wikilinks | [[links]] pointing to non-existent pages |
| HIGH | Index gaps | Pages on filesystem not in index.md |
| MEDIUM | Orphan pages | Pages with no inbound links |
| MEDIUM | Contradictions | Unresolved contradictions: frontmatter flags |
| MEDIUM | Frontmatter | Missing fields per SCHEMA.md required list |
| LOW | Tag drift | Tags not in SCHEMA.md taxonomy |
| LOW | Stale content | updated older than newer sources mentioning same entities |
| LOW | Oversized pages | Over SCHEMA.md limit (default: no limit) |
| AUTO | Log rotation | Per SCHEMA.md threshold (default: no rotation) |
Content checks (the LLM layer — this is what Karpathy emphasized):
wiki-ingest.--fix
or on user confirmation.--fix applies safe automated fixes (index gaps, log rotation, approved
schema updates). --report-only is report-only (default).
Trigger: "import my existing docs", "migrate my vault", "bring in my whole folder"
Arguments: <source-path>, --format=folder|obsidian|notion|confluence,
--map=<file>, --dry-run, --resume, --priority=recency|links|manual
Five-phase import: Discovery → Mapping → Deduplication → Processing
in waves of 20 → Navigation update. Checkpoint to
.wiki-import-checkpoint.json every 20 files; resume on interruption. Handles
Obsidian vaults, Notion/Confluence exports, or plain folders. For large imports
(hundreds of files).
See the full plugin skill for all details.
Trigger: "watch raw/", "auto-detect new files", "what's pending to ingest", "drain the queue"
Arguments: [--start [--poll N --debounce N]], [--stop], [--status],
[--drain [--pages 1,2,3]], [--remove <id>]
Closes the "I dropped a file in raw/articles/ and forgot to ingest it" gap.
A polling daemon (default 5 s) detects new files in
raw/{articles,papers,transcripts,external}/, debounces against in-progress
writes (default 5 s), and writes them to .wiki-ingest-queue.json with a
pending status. The watcher never auto-runs wiki-ingest — drain is
always explicit, so a misconfigured watcher cannot silently mutate wiki pages.
--start — spawn the daemon for the resolved wiki. PID and log are
per-project (~/.ak-wiki/watcher-{slug}.pid, ….log), so multiple
project wikis (~/.llm-wiki/necall, ~/.llm-wiki/rtc, …) can each run
their own watcher concurrently.--status (default when no flag passed) — daemon liveness + queue
summary (pending / processed / failed / removed). Works whether or not
the daemon is running.--drain — for each pending entry in detection order: invoke
wiki-ingest <path>, mark processed on success or failed on error.
After the loop, append a single ## [date] watch | drained N files log
entry. --drain --pages 1,3 processes only the listed indices.--remove <id|index> — drop an entry from the queue without ingesting.
Use for WIP files you don't want in the wiki yet.--stop — clean SIGTERM to the daemon (TerminateProcess on Windows).When NOT to use: for one-off ingestion of a known file → just run
wiki-ingest <path>. The watcher is for the steady-state drip workflow.
For bulk migration of an existing folder use wiki-import.
Trigger: "what frozen pages just got relevant again", "dream", "re-promote old pages"
Arguments: [--since YYYY-MM-DD], [--strategy co-occurrence],
[--apply --pages 1,2,3], [--explain <page>], [--out <file>]
The only ak-wiki feature that runs without you. While you sleep, the dreamer
re-evaluates which frozen and archived pages have become relevant again
based on this period's ingests. Reads only the wiki filesystem (log.md +
page frontmatter dates ingested_at / updated) — never reads chat
sessions, file mtimes, or any external state. Frontmatter dates are
checked into git, so git clone reproduces dreamer behavior on any
machine; using mtimes would not (mtimes reset on checkout).
How it scores (v1.6 strategy: co-occurrence):
min_count times in fresh
pages after being dormant for dormancy_window_days[[wikilinks]] to
the old pagePages above confidence_threshold (default 0.6) become candidates —
surfaced for review, never auto-promoted. --apply --pages 1,2,3 writes
tier_override: active plus a reason and timestamp to selected candidates.
dreaming/{YYYY-MM-DD}.md for human review next morning. Caps at
max_repromote_per_run (default 10).--explain <page> — score breakdown for one specific page (entity /
tag / citation components). Useful for "why didn't X get promoted?"--apply — promotes named candidates and advances the dream |
watermark in log.md.Schedule weekly via Claude Code: claude /schedule "0 23 * * 0" "/ak-wiki:wiki-dream". Tune via wiki-config --set dreaming.<key> <value>.
Trigger: "show all config", "set dreaming threshold", "what does X mean", "validate SCHEMA.md"
Arguments: [--show], [--get <path>], [--set <path> <value>],
[--explain <path>], [--validate]
Generic path-based interface to SCHEMA.md tunables. Use this when there's no
domain-specific shortcut (e.g. wiki-tier --set-active=540d is better UX
for tier thresholds; wiki-config is for the long tail like dreaming
weights and resurgence params).
--show — all config blocks at a glance (memory_tiers, dreaming,
custom_dimensions, tag_taxonomy_count, categories)--get memory_tiers.active_days — read one dotted-path value--set dreaming.confidence_threshold 0.55 — surgical line replacement
that preserves comments, blank lines, and ordering. Validates after every
write; reverts SCHEMA.md if validation fails, surfaces the validation
errors verbatim.--explain dreaming.weights.entity — read the docstring for any
documented key (kept in sync with TRD additions).--validate — run the full schema_validate pipeline (structural +
cross-field rules).v1.6 limitations (deliberate trade for safety):
custom_dimensions[0].required is unreachable).--set is independent.For introducing a dreaming: block to a wiki initialized before v1.6, edit
SCHEMA.md by hand or re-run wiki-init with the dreaming flag.
Trigger: "sync the wiki", "pull from origin", "push my changes", "are my machines in sync"
Arguments: [--auto], [--dry-run]
Pull-merge-push your wiki across machines. Custom merge driver
(merge_log.py) auto-merges log.md as a union+sort, deduplicating by
canonical hash and preserving body data on same-triple-different-body
divergence. Per-machine sync reports go to
~/.ak-wiki/sync-reports/{slug}/ — never inside the wiki repo, so
they don't self-conflict on the next sync.
--auto (cron mode) — any non-clean outcome exits non-zero so a
chained && wiki-dream doesn't run. Recommended cron line:
0 23 * * 0 wiki-sync --auto && wiki-dream--dry-run — read-only preview; forks BEFORE side effects (no
lock, stash, driver registration, or log mutation). Only git fetch
runs (writes only to .git/refs/remotes/).Hard stops (exit 1):
wiki_id ≠ remote wiki_id from
git show origin/main:SCHEMA.md.wiki-import-lock fresh OR
.wiki-import-checkpoint.json present (wiki-import was interrupted)Conflicts that the driver can't auto-resolve land as standard
unmerged paths (<<<<<<< / >>>>>>>); the conflict report includes
the recovery commands using stash SHA (not stash@{0} which can drift).
v1.8 MVP limitations:
merge_log.py driver registered. merge_index.py is v1.8-full
per PRD §13 phasing — until then index.md uses git's default 3-way.race-exhausted.{wiki_path}/
├── SCHEMA.md # Conventions, tag taxonomy, policies (USER-EDITABLE)
├── index.md # Sectioned content catalog with one-line summaries
├── log.md # Append-only chronological action log
├── raw/ # IMMUTABLE — source material, never modified
│ ├── articles/
│ ├── papers/
│ ├── transcripts/
│ ├── external/
│ ├── imported/
│ └── assets/ # Downloaded images and attachments
└── {categories}/ # Category directories — defined by SCHEMA.md
# e.g. entities/, concepts/, comparisons/, queries/
# or: characters/, themes/, plot/, timeline/
# or: people/, projects/, decisions/, meetings/
Starter structure ≠ law. Categories match the domain.
The wiki is a drop-in Obsidian vault:
[[wikilinks]] render as clickablewiki-digest for a text equivalent.raw/articles/. Fastest way to get sources in.raw/assets/; reference as
![[image.png]]--format=slides output inside ObsidianThe wiki is a git repo by default. wiki-init suggests git init as the final
step. Every wiki-ingest and wiki-query --file produces clean, reviewable
diffs. Team wikis can use branches for proposed changes before merging. log.md
gives the timeline; git log gives the diffs behind it.
raw/ is read-only. Write corrections to wiki pages, not sources.wiki-lint as a pure formatter — it should suggest content gaps
and schema evolution, not just structural fixes