LLM-maintained wikis rot in predictable ways (Karpathy gist comments cite 4 failure modes after 1 month of use):
-
Identity scan + dedup scoring (duplicate concepts):
grep -rh "^id:" .llmwiki/wiki/ | sort | uniq -d
grep -rh "^aliases:" .llmwiki/wiki/ | tr ',' '\n' | sort | uniq -d
dup_ids=$(LC_ALL=C.UTF-8 grep -rhoP '^id:\s*\K\S+' .llmwiki/wiki/ | sort | uniq -d)
dup_aliases=$(LC_ALL=C.UTF-8 grep -rhoP '^aliases:\s*\[\K[^\]]+' .llmwiki/wiki/ \
| tr ',' '\n' | sed 's/^[[:space:]]*//; s/[[:space:]]*$//' \
| grep -v '^$' | sort | uniq -d)
for tok in $dup_ids; do
printf '== id cluster: %s ==\n' "$tok"
LC_ALL=C.UTF-8 grep -rlP "^id:\s*\Q$tok\E\b" .llmwiki/wiki/
done
for tok in $dup_aliases; do
printf '== alias cluster: %s ==\n' "$tok"
LC_ALL=C.UTF-8 grep -rlP "^aliases:.*\Q$tok\E" .llmwiki/wiki/
done
For each duplicate cluster, score the overlap and propose one concrete
remedy — borrowing mem0's memory-reviewer triage pattern (the pattern, not
its data; no mem0 call). The score is a coarse band, never a fabricated float
(consistent with the wiki's provenance-over-confidence rule):
| Overlap | Signal | Suggest |
|---|
| High | same id, or near-identical concept + overlapping body claims | merge — fold into the stronger page, redirect the other's aliases |
| Medium | same concept, different facet / partial claim overlap | supersede — keep both, mark the older status: stale + > Superseded-by: the newer |
| Low | shared alias but genuinely distinct concepts | alias — disambiguate the colliding alias (rename/scope), keep both pages |
Report-only — surface <cluster> — overlap: <band> — suggest: <remedy> for the
user; never merge/supersede/alias automatically (you may delete load-bearing
content — see Anti-patterns).
-
Level scan (pages too big):
find .llmwiki/wiki -name '*.md' -size +5k -print
For pages > 5KB, propose split.
-
Relationship scan (untyped cross-refs):
rg -nP '^\[\[' .llmwiki/wiki/ || echo "OK"
Any hits → must convert to typed > Refines: / > Contradicts: / > Evidence: / > See-also: / > Supersedes: / > Superseded-by: / > Uses: / > Depends-on: / > Caused-by: / > Fixed-by: (per-token meanings: ${PLUGIN_ROOT}/references/wiki-conventions.md § Cross-reference grammar). Only a bare line starting with [[ is flagged; typed refs are never flagged.
-
Staleness scan (per-page volatility window — volatile 30d / stable or absent 180d; covers the promoted .llmwiki/insight/ layer too, matching the stale-check hook):
today=$(date +%s)
while IFS= read -r f; do
d=$(LC_ALL=C.UTF-8 grep -oP '^last_verified:\s*\K\d{4}-\d{2}-\d{2}' "$f" || true)
[[ -z "$d" ]] && continue
vol=$(LC_ALL=C.UTF-8 grep -oP '^volatility:\s*\K\S+' "$f" || true)
[[ "$vol" == "volatile" ]] && window=30 || window=180
age_days=$(( (today - $(date -d "$d" +%s)) / 86400 ))
[[ $age_days -gt $window ]] && printf '%s (%d days, %s window %dd)\n' "$f" "$age_days" "${vol:-stable}" "$window"
done < <(find .llmwiki/wiki .llmwiki/insight -name '*.md' -not -name 'index.md' -not -name 'log.md' -not -name 'log-[0-9][0-9][0-9][0-9].md' 2>/dev/null)
For each stale page, either re-verify against current code and bump the date, or mark for review.
-
Orphan scan (pages not in index, indexed pages that don't exist):
diff <(find .llmwiki/wiki -name '*.md' -not -name 'index.md' -not -name 'log.md' -not -name 'log-[0-9][0-9][0-9][0-9].md' | sort) \
<(LC_ALL=C.UTF-8 grep -oP '\(\K[^)]+\.md' .llmwiki/wiki/index.md | sed 's|^|.llmwiki/wiki/|' | sort)
-
MOC integrity (cross-refs to non-existent pages):
rg -oP '\[\[\K[^\]]+' .llmwiki/wiki/ | sort -u
-
Contradictions: any page with a > Contradicts: link is a flag — those should be resolved (one of the two pages updated or merged), not left standing.
-
Status/supersession integrity (report-only, no auto-fix):
while IFS= read -r f; do
LC_ALL=C.UTF-8 grep -q '^status:\s*stale' "$f" || continue
LC_ALL=C.UTF-8 grep -q '^> Superseded-by:' "$f" || printf 'stale without Superseded-by: %s\n' "$f"
done < <(find .llmwiki/wiki -name '*.md' -not -name 'index.md' -not -name 'log.md' -not -name 'log-[0-9][0-9][0-9][0-9].md')
LC_ALL=C.UTF-8 grep -rhoP '^> Supersedes:\s*\[\[\K[^\]]+' .llmwiki/wiki/ | sort -u | while IFS= read -r id; do
[[ -z "$id" ]] && continue
tgt=$(LC_ALL=C.UTF-8 grep -rl "^id:\s*$id\b" .llmwiki/wiki/ | head -1)
[[ -z "$tgt" ]] && { printf 'Supersedes target missing: %s\n' "$id"; continue; }
LC_ALL=C.UTF-8 grep -q '^status:\s*stale' "$tgt" || printf 'Supersedes target not stale: %s (%s)\n' "$id" "$tgt"
done
Flag mismatches for the user — every status: stale page should have a > Superseded-by:, and every > Supersedes: should point at a page now status: stale.
-
Sources sanity (soft, report-only): the sources: N count should roughly match the number of entries under each page's ## Sources section. Large divergence (e.g. sources: 3 with one bullet under ## Sources) is a flag, not a fail.
-
Insight layer integrity (the promoted .llmwiki/insight/ layer; skip if absent):
[ -d .llmwiki/insight ] || echo "no insight layer (skip)"
LC_ALL=C.UTF-8 grep -rhoP '^promoted_from:\s*\[\[\K[^\]]+' .llmwiki/insight/ 2>/dev/null | sort -u | while IFS= read -r id; do
[[ -z "$id" ]] && continue
tgt=$(LC_ALL=C.UTF-8 grep -rl "^id:\s*$id\b" .llmwiki/wiki/ | head -1)
[[ -z "$tgt" ]] && { printf 'promoted_from target missing: %s\n' "$id"; continue; }
LC_ALL=C.UTF-8 grep -q '^status:\s*stale' "$tgt" && printf 'promoted_from target is stale: %s (%s)\n' "$id" "$tgt"
done
find .llmwiki/insight -name '*.md' -not -name 'index.md' -size +2k -print 2>/dev/null
while IFS= read -r f; do
LC_ALL=C.UTF-8 grep -q '^tier:\s*insight' "$f" || printf 'insight entry missing tier: %s\n' "$f"
LC_ALL=C.UTF-8 grep -q '^promoted_from:' "$f" || printf 'insight entry missing promoted_from: %s\n' "$f"
done < <(find .llmwiki/insight -name '*.md' -not -name 'index.md' -not -name '_insight-template.md' 2>/dev/null)
Report-only. Beyond the mechanical checks, eyeball each insight entry against its promoted_from: wiki page: the insight must condense the page, not contradict it, and must not restate the page's full body (dedup — the long story stays in the wiki). Flag any insight whose rule conflicts with its now-active source page, or that has grown into a second copy of the wiki page.
-
Source-drift scan (raw evidence integrity — the raw layer is immutable, so a body-hash that no longer matches the stored sha256: means the file was edited or the source URL's content moved):
LC_ALL=C.UTF-8
_body_sha256() {
awk 'NR==1&&$0=="---"{fm=1;next} fm&&$0=="---"{fm=0;next} !fm{print}' "$1" \
| { sha256sum 2>/dev/null || shasum -a 256; } | awk '{print $1}'
}
_fm_sha256() {
awk 'NR==1&&$0=="---"{f=1;next} f&&$0=="---"{exit} f&&/^sha256:[[:space:]]*[^[:space:]]/{v=$0;sub(/^sha256:[[:space:]]*/,"",v);print v;exit}' "$1" 2>/dev/null
}
while IFS= read -r f; do
stored=$(_fm_sha256 "$f")
[[ -z "$stored" ]] && continue
actual=$(_body_sha256 "$f")
[[ "$stored" != "$actual" ]] && printf 'DRIFT: %s (stored %s.. != actual %s..)\n' "$f" "${stored:0:12}" "${actual:0:12}"
done < <(find .llmwiki/raw -type f -not -name '*.pdf' 2>/dev/null)
find recurses into the raw source-type buckets (external/ research/ transcripts/ audits/) and scans any-extension text raw (.md, .txt, .html, ...), not just .md — a transcript or external doc can carry sha256: frontmatter too; the sha256:-presence guard (not the extension) is the real filter. Binary raw (.pdf) is excluded since it can't carry text frontmatter. Files with no sha256: field are skipped (frontmatter is prospective-only — existing raw is never backfilled, per raw-immutability). A DRIFT hit means either the immutable raw file was edited (a discipline break) or the same source_url now yields different bytes (re-ingest -> write a new dated snapshot, don't overwrite). Report-only.
-
Link-poverty scan (graph-isolated pages — Step 5's orphan scan catches index omissions, but a page can be in the index yet carry zero typed cross-refs, leaving it invisible to graph traversal):
LC_ALL=C.UTF-8
while IFS= read -r f; do
n=$(LC_ALL=C.UTF-8 grep -cP '^> (Refines|Contradicts|Evidence|See-also|Supersedes|Superseded-by|Uses|Depends-on|Caused-by|Fixed-by):' "$f")
[[ "$n" -eq 0 ]] && printf 'link-poverty: %s (0 typed cross-refs)\n' "$f"
done < <(find .llmwiki/wiki -name '*.md' -not -name 'index.md' -not -name 'log.md' -not -name 'log-[0-9][0-9][0-9][0-9].md' 2>/dev/null)
Flags wiki pages with no typed cross-ref line (> Refines: / > See-also: / > Evidence: / ...). Report-only — a genuinely standalone page (a domain's first page, a leaf citing only raw evidence) can be legitimately ref-poor; the human decides whether it should be wired into the graph.
-
Log-rotation due (bounded hot log — log.md grows monotonically; at year-turnover it should shed the prior year):
LC_ALL=C.UTF-8
cur_year=$(date +%Y)
LC_ALL=C.UTF-8 grep -oP '^## \K\d{4}' .llmwiki/wiki/log.md 2>/dev/null | sort -u \
| awk -v y="$cur_year" '$1 < y {printf "log-rotation due: %s entries in log.md -> migrate to log-%s.md\n", $1, $1}'
If any ## YYYY-... entry predates the current year, suggest migrating that year's block into a sibling log-YYYY.md (newest-first preserved; grep '## ' log*.md still recovers the full time-series). Report-only — the migration itself is a manual / ingest-finding op, logged like any other event. (Convention: ${PLUGIN_ROOT}/references/wiki-conventions.md § log.md discipline.)
Report only — do not auto-fix. User reviews and triggers /llm-wiki:ingest-finding for each remediation.
