wiki-sync

name

wiki-sync

description

Build and refresh the company wiki from source repos. Use when the user runs /wiki-sync, asks to "sync the wiki", "update the wiki", "scan the repos", "refresh docs from code", or after onboarding new repos in repos.yml. Walks the repos one at a time — a fast skim that drops light concept pages, then a deep pass that enriches them into real, well-linked pages — merging into existing pages when a concept already exists and growing the cross-link graph as it goes. Writes concept-first, cross-linked Markdown (concepts, standards, decisions, business logic, cross-repo scenarios) — NOT implementation details, and NEVER a home/index.md. Reads changed files for the BEHAVIOUR they encode, not just literal name matches: recognizes conditional business rules (country/market/currency/tier gates, feature-eligibility checks) as page-worthy concepts even when no named term exists yet. Honors --all, --force <repo>, --repo <name>, --dry-run.

/wiki-sync — grow & refresh the wiki, repo by repo

You orchestrate one sync. Teamwork is the model: the scripts do the deterministic plumbing (fetch, diff, mention-index, backlinks, validate, advance state); YOU do every piece of judgment. The scripts can only match literal strings — they are blind to what a file means, to whether a concept is the same one under a new name, to who really owns it. That is your half. Read CLAUDE.md (the operating manual) first — it is the contract. Never break its §8 invariants.

Three things make this sync different from a dumb scan — keep them in mind throughout:

1. Repo by repo, two passes each. Don't try to understand five repos at once. Take ONE repo: skim it and drop light pages fast — of every category the vault has (concepts, standards, decisions, scenarios, glossary), each classified into its right type/folder (title
   • a line + source) — then go deep on that same repo and enrich those pages into real ones. Be inclusive — capture a lot; the deep pass and the curation bar prune. Then next repo.
2. Merge, never duplicate. Before creating any page, resolve the name. If the thing already has a page — of any type, from an earlier repo this run or a past sync — you enrich that page with what this repo adds; you do not make a second file. One concept, one page, across all repos and all categories.
3. Link as you go. Don't leave linking to the end. Every page you touch should link out to the concepts/repos/standards it relates to, and you fix dead links continuously, so the graph thickens with every page instead of in one big pass.

And one hard rule about layout: there is NO index.md / home / dashboard page. The wiki is a flat peer-to-peer graph — pages link to each other, never to a central hub (a hub becomes a false node every page edges to, which distorts the Obsidian graph view). The maintenance signal that a dashboard used to show — needs-verification, orphans, possibly-retired, unclear-ownership — now comes to you as the sync report on stderr from render-mocs.py. You act on it; you never write it to a page. Navigation is the link graph + the per-domain _moc/ hubs.

Arguments

• (none) — sync repos that are due.
• --all — force every repo.
• --force <repo> / --repo <repo> — sync just that repo.
• --dry-run — show what's due and the plan; write nothing.

Procedure

0. Dry run first if asked

bash scripts/sync.sh dry-run [flags]

Show the table, stop. Otherwise continue.

1. Prepare (deterministic — one command)

RUN=$(bash scripts/sync.sh prepare [flags])

This computes due repos → fetches them → extracts concept candidates → records the proposed scan state (NOT applied yet), and writes a run file. $RUN is its path. Read it:

cat "$RUN"

It contains: due, fetched (each with changed_files and new_sha), candidates (literal name matches with context), ambiguous_terms, failed. If note: "nothing due", report and stop.

2. Walk the repos ONE AT A TIME (the heart of the skill — pure judgment)

Process the fetched repos sequentially, finishing one repo's two passes before starting the next. This is deliberate: by the time you reach repo #2, repo #1's pages already exist, so #2's mentions of the same things become enrichment + links, not duplicates. The graph grows in dependency order and stays merged.

"Concept" here is shorthand for a page of ANY vault category — not just concepts/. The two-pass / merge / link-as-you-go method applies identically to every page type the vault has, and Pass 1's first job for each candidate is to classify it into the right type (which fixes its folder, template, and required frontmatter):

type	folder	what earns it
concept	concepts/	a named project / initiative / service / domain thing
standard	standards/	a documented convention — "how we do X"
decision	decisions/	an architectural/technical decision worth remembering (ADR-ish; adds date, supersedes, superseded_by, consequences)
scenario	scenarios/	a cross-repo flow no single repo fully explains
glossary	glossary/	a business term with one agreed meaning
repo	repos/	auto-managed by finalize from repos.yml — never hand-create
moc	_moc/	a per-domain hub; created sparingly, lists auto-generate
A deliberate conditional business rule is usually a standard or glossary (cross-repo →
scenario). When two types fit, prefer the more specific: a remembered choice → decision;
a rule we follow → standard; a term → glossary; a flow → scenario; otherwise
concept. Pick the type per page; don't default everything to .

For each repo R in fetched, in order:

2a. Pass 1 — fast skim, drop LIGHT pages of EVERY type (be inclusive)

Skim R's surface to enumerate page-worthy things quickly — across all categories above, not just concepts. Sources, cheapest first, and what each tends to surface:

• the run's candidates for R (literal hits against existing titles/aliases — any type),
• R's README + top-level dirs → concepts (services, domains) and glossary terms,
• any docs/ tree under .state/sources/<R>/ → standards ("how we do X"), decisions (an adr/ or decisions/ dir, RFC/ADR files), scenarios (cross-repo flows/runbooks),
• R's changed_files list → which of the above this run actually touched.

For each thing that plausibly clears the curation bar (CLAUDE.md §6), do the classify-then-resolve-then-act step:

0. Classify the type using the table above (concept / standard / decision / scenario / glossary). This decides the folder, the .templates/<type>.md to follow, and the required frontmatter. Never lump everything into concept.
1. Resolve first (this is how duplicates are prevented): python3 scripts/wiki-grep.py --resolve "<name>"
   • Returns a page → it exists (in whatever folder). Leave it for Pass 2 enrichment; if <name> is a new spelling, add it to that page's aliases now (cheap, keeps mentions linking). If it exists but you think the type is wrong, note it for Pass 2 — don't make a second file in another folder.
   • Returns nothing → also confirm vault/<folder>/<slug>.md doesn't exist, where <slug> = python3 -c "import sys;sys.path.insert(0,'scripts');from _lib import slugify;print(slugify(sys.argv[1]))" "<title>". Then create a light page with new-page.py, which reads ONE JSON spec on stdin (it slugs the filename, seeds title-as-alias, fills all required frontmatter + GENERATED markers, and refuses to overwrite an existing slug). Light spec — just enough to be valid:

     echo '{"title":"<Title>","type":"<concept|standard|decision|scenario|glossary>",
            "owner":"<best-guess>","summary":"<one-line what-this-is + who-for>",
            "source_refs":["<R>:<path>@<new_sha>"],"source_repos":["<R>"],
            "verified_refs":["<R>@<new_sha>"],"tags":["confidence/inferred"],
            "today":"<today>"}' | python3 scripts/new-page.py
     

     (For type:"decision" also pass "date":"<today>" — the decision template requires it.)
2. A light page = the JSON above → a valid page (of whatever type) with a one-liner + the source ref you already know. It does not need to be complete yet — Pass 2 fills it by editing the file directly (new-page.py refuses to overwrite, so enrichment is a normal file edit, not a re-create). The point of Pass 1 is coverage: get a page (and therefore a link target) of the right type to exist for everything real in R, fast, so Pass 2 and later repos can link to it.

Inclusive in Pass 1, strict in Pass 2 — capture generously now; the deep pass and the curation bar trim what doesn't earn its keep. (When you're unsure something is page-worthy, still make the light page but tag it confidence/inferred; Pass 2 either promotes or you drop it.)

2b. Pass 2 — go DEEP on the same repo, enrich into real pages

Now read the changed files of R that matter (only the changed ones — token discipline) under .state/sources/<R>/, and turn the light pages — of every type, not just concepts — into real ones. Each type fills out per its own .templates/<type>.md (a standard gets its convention + rationale; a decision gets context/options/consequences + date/supersedes; a scenario gets the step-by-step cross-repo flow, ideally a Mermaid sequenceDiagram; a glossary term gets the one agreed meaning + where it's enforced). If Pass 1 misclassified a page's type, fix it now: move the file to the correct folder, swap to the right template's frontmatter/sections, and add an alias so existing [[links]] still resolve — don't leave a second copy behind. For each page touched by R:

• Read for BEHAVIOUR, not strings. candidates is a floor, not a ceiling — it only finds literal name matches, so it is blind to logic with no named term yet. Your deep-pass job is the part no script can do: read the changed files for what they encode.
  • Recognize conditional business logic as a concept. When a changed file branches on a business condition — a country/market/currency/tier/segment check, a feature-eligibility gate, a special-case rule — ask "would a teammate be misled without knowing this?" If yes, it earns a page (a standard/glossary; cross-repo → scenario). Strongest signals, in order: a named rule/strategy/policy/eligibility type or a docs/features/<x>.md; an enum/constant branch on a market or segment; a comment/doc/test stating the why. Title it for the business meaning, not the code symbol; alias the symbol + condition keywords; capture condition → behaviour → why. Don't paste the branch — describe it.
  • Curation still applies — don't page every if. Generic guards, validation, retries, one-off hacks are plumbing. Bar: surprising/misleading to omit, and still true in 6 months. Unsure → leave a proposal in your report instead of minting a page.
• Enrich, merge, never fork. If the concept already has a page (it usually will after Pass 1, or from a past sync / earlier repo this run), open that page and fold in what R adds — the real behaviour, this repo's role, a new source_ref, a cross-repo relationship. Add R's spelling to aliases if new. Do not create a second file. (If you discover a true duplicate from a past run — order-hold.md and order hold.md — merge them: keep the kebab slug, move the other's content + aliases onto it, delete the loser. validate.py will fail the build on same-folder slug / case-insensitive title collisions, so fix it here.)
• Learn from the source across repos, then write. When a concept also lives elsewhere (same term in another repo, a shared client/SDK, a sibling service), read how it's actually used there in .state/sources/<other>/ (call sites, the contract, the event/schema, the README) until you understand it concretely — THEN fold only what a reader needs into our page. The page must reflect verified cross-repo usage, not one repo's local guess.

2c. Link as you go (every page you touch, every repo)

Don't defer linking to the end. As you write/enrich each page:

• Forward-link generously with [[Canonical Title]] (and [[repo-name]] for repos — repo pages auto-exist after finalize). Leave the <!-- … --> generated markers in place; never hand-write backlinks/ownership blocks.
• If your prose links a concept with no page yet, prefer to create its light page right then (so the link resolves immediately) rather than queueing a dead link. A new page can spawn new links — that's fine, the cascade is the point.
• Periodically (after finishing a repo) run bash scripts/sync.sh dead-links to see what your new prose left dangling, and close it before moving on. This keeps the invariant: every [[link]] resolves to a file true throughout the sync, not just at the gate.
• You do NOT need to re-scan earlier repos for the pages you just created — finalize's term sweep finds the new pages' mentions across every already-scanned clone automatically. Never force extra sync runs just to "pick up" cross-repo backlinks.

2d. Handle ambiguity & proposals for this repo

• For each candidate with ambiguous: true, use its context to decide if it really refers to the concept. If not, don't force a link — it simply won't resolve, and that's correct.
• For a real concept you're not confident enough to page, add a proposal to your report rather than minting a junk page.

→ Repo R done. Move to the next fetched repo and repeat 2a–2d.

3. Resolve any remaining dead links (delegate to /wiki-fix-links)

You linked as you went, so this should be small. Before finalize, sweep the rest:

Invoke the wiki-fix-links skill now (its steps 1–4: find dead [[links]] → research in the clones under .state/sources/ → create the missing page or a flagged draft stub → cascade until bash scripts/sync.sh dead-links reports dead_count: 0). The clones are already fetched from this run, so it researches without re-fetching. Don't duplicate its logic here.

Invariant before finalize: zero empty [[links]].

4. Finalize (deterministic — one command)

bash scripts/sync.sh finalize

This: ensures a page exists in repos/ for every repo (so [[repo-name]] always links); sweeps every already-scanned clone for the terms you registered this run (pages created in Pass 1/2 or by fix-links weren't in the registry when prepare extracted candidates — the sweep finds their mentions across ALL scanned repos now, so the graph converges in THIS sync instead of needing repeated re-runs); rebuilds .backlinks.json + every page's backlink block from the aggregate candidate index; writes each page's OWNERSHIP block (Python's best mechanical guess — declared vs CODEOWNERS vs repo-owner, with confidence + evidence); regenerates the per-domain _moc/ lists and each repo's DEFINED-HERE; prints the WIKI SYNC REPORT to stderr (needs-verification / orphans / possibly-retired / unclear-ownership — this replaces the old index.md dashboard); runs validate.py; and — only if validate passes — advances .sync-state.json.

There is no index.md to write and you must not create one. If validate exits non-zero, fix the reported errors (dead links, missing owner, secrets, bad frontmatter, malformed source_refs/verified_refs, duplicate slug/title) and run finalize again. Scan state is NOT advanced until validate is clean.

Backlinks come from the aggregate candidate index (.candidates/<repo>.json, gitignored, clone-local). On a fresh clone, prepare must run before finalize can reproduce backlinks from scratch. A normal /wiki-sync keeps both in sync.

5. Act on the SYNC REPORT (your judgment — the report is input, not output)

finalize printed the report to stderr. Don't file it away — work it. This is the teamwork hand-back: the scripts surfaced the signal, you decide what's true.

• Needs verification. For any listed page whose source you fetched this run, re-check it vs the source and bump last_verified / verified_refs, rewriting if reality moved.
• Orphans. A page with no inbound [[link]] is either a missing-link bug or genuine noise. Weave it into the graph by linking it from a related page (preferred), or — if it's truly not a concept — retire it. Don't leave orphans; an unreachable page is invisible.
• Unlinked mentions. finalize also lists prose that names a concept without [[linking]] it (typically: an older page mentioning a concept whose page was only created this run). Get the full list with bash scripts/sync.sh unlinked. For each genuine reference, edit the prose to [[Canonical Title]] (or [[Canonical Title|as written]]); skip coincidental word matches — a mention that isn't really the concept should stay plain text. This is how page↔page links converge in one sync instead of accreting over several.
• Possibly retired (source-of-truth file gone). Decide per entry:
  • Moved/renamed → find the new path in the clone, update source_refs + verified_refs, bump dates. Not retired.
  • Genuinely gone (confirm across the relevant repos) → mark retired, don't delete: set status: deprecated (or superseded + superseded_by: [[Replacement]]), prepend > ⚠ **Deprecated <date>:** <why> — kept for history., and shrink the body to a tombstone (what it was, why gone, what replaced it). Deprecated pages stay linkable so old links don't break.
• Shrink concepts that got smaller, not gone. If the source still has it but it lost scope, trim the page to current reality — delete paragraphs that no longer hold. A page that only ever grows drifts.

5b. Ownership is a JUDGMENT step — Python proposes, YOU confirm (teamwork)

ownership.py writes a mechanical owner guess into each page's OWNERSHIP block (CODEOWNERS → repo-owner → weak recent-contributor hint). It is deliberately conservative and often wrong or "unconfirmed" — that's where you come in. The Python is the floor; your read of the code is the authority. For every page on the report's Unclear ownership list (and any page whose declared owner the report says the code contradicts):

1. Look at who actually works on this concept — the defining file's directory, its CODEOWNERS, the surrounding service, the team named in nearby docs/READMEs in .state/sources/<repo>/. You can infer an owner the mechanical resolver can't (e.g. "all of src/pricing/** is the Pricing team even though there's no CODEOWNERS line").
2. Set the page's owner to the team you concluded — that declared owner takes precedence and clears the "unconfirmed" flag on the next finalize. If the right fix is upstream (the source repo needs a CODEOWNERS rule), say so in your report.
3. Never hand-edit the generated OWNERSHIP block — change the owner frontmatter and let finalize regenerate it.

Re-run finalize after the §5 / §5b edits so backlinks, ownership blocks, and the report reflect your changes (and scan state advances on a clean validate).

6. Report

Summarize for the human: repos synced (+ any failed); pages created (light) vs enriched (deep) per repo; merges you performed; new-term proposals; dead links resolved (full pages vs draft stubs needing detail, + any you unlinked as noise); concepts retired/shrunk; ownership you corrected and any source repos that need a CODEOWNERS rule; anything still needing verification. Keep it scannable. Do not mention or create an index.md — there isn't one.

The browsable site rebuilds itself — don't build it here

The committed vault/ markdown is the only artifact you produce. The Quartz static site (site/) is a derived view: site/public is gitignored and never committed. Do NOT run make site during a sync. Because there is no vault/index.md, Quartz's enabled FolderPage emitter auto-generates the site's root landing page from the folder listing — the site needs no hand-made home page, and you must not add one to the vault to "give it a homepage". The hosted site regenerates in CI when this sync's vault lands on main (publish-site.yml, if WIKI_SITE_ENABLED). A human previews locally with make serve.

Token discipline

Read only the changed files the run surfaced, never whole repos — except the cheap surface (README/dirs/docs tree) in Pass 1, which is what makes the fast skim fast. Prefer rtk-proxied commands for noisy git/grep. The scripts already pre-filter for you.

Hard rules (see CLAUDE.md §8)

Never commit .state/. Never hand-edit generated blocks or .backlinks.json. Every page has an owner. No secrets. No implementation dumps. Forward links only. One concept, one page. No index.md / home / dashboard page — ever. No dead links in either resolver. Scan state advances only after a clean validate. Date everything you touch.