migration-docs-audit

name	migration-docs-audit
version	1.0.0
description	Use when auditing the hand-authored AsciiDoc prose under docs/ for accuracy and staleness, verifying a doc claim against source of truth, or before a release/cutover when the docs must be trustworthy. The repo moved many facts into generated partials and a verify-doc-facts net, but hand-authored prose still drifts silently because nothing checks it. Covers the source-of-truth audit method, the recurring staleness patterns in these docs, and how to report and fix findings.

Migration docs audit

docs/ is a multi-page Diataxis wiki (operator, developer, sme, spec, shared, decks, plus README.adoc/toc.adoc). A lot of the facts that used to rot in prose are now emitted as generated partials and guarded by verify_doc_facts.py. What is left is hand-authored prose that names a command, path, phase, count, schema, gate, or env var, and nothing checks it. That prose drifts as the code moves. An audit means re-deriving every such claim from the code and asserting the page still agrees.

When to use

• Auditing docs/ prose for accuracy or staleness, in whole or per folder
• Verifying a specific doc claim against the code that backs it
• After a code change that could invalidate prose (a phase renamed, a command added, a table moved between schemas, a count changed)
• Before a release or cutover, when an operator will follow these pages
• When asked to produce a per-file accuracy/staleness report and fix list

First: run the safety net and read its output

Do not start a manual sweep cold. The repo already enforces nine fact classes; run them first and treat their failures as known, so you do not re-report what tooling already catches.

# Whole net (from docs/): schema-refs, crosswalks, counts, doc-facts,
# prisma-analysis, plus the generated-partial freshness checks.
cd docs && make install && mise install && make verify 2>&1 | tail -40

# The hand-authored-fact checker on its own (from repo root):
uv run python docs/tools/verify_doc_facts.py

verify_doc_facts.py re-derives nine facts from code and asserts the prose agrees: the PMDA project-number regex, the gate graph, pgloader template markers, the sql/ stage-dir map, Env variables, the lib.py package surface, Makefile targets, human-input files, and the JSONB schema registry. If a make verify step fails, confirm it is not pre-existing before blaming your changes:

git stash && (cd docs && make verify-schema-refs 2>&1 | tail -8); git stash pop

Scope: hand-authored prose only

In scope (audit the prose)	Out of scope for prose staleness
operator/, developer/, sme/ pages	shared/generated/* partials
shared/*.adoc hand-authored pages	operator/reference-cli.adoc (from cli_to_adoc.py)
spec/canonical-spec.adoc, spec/migration-plan.adoc	reports/generated/source_target_columns_table.adoc and the source_target_columns_* fragments
decks/*.adoc	schema diagrams, reference-data-dictionary.adoc
README.adoc, toc.adoc	the prisma-analysis partial

Generated partials are regenerated by make and guarded by a git diff --exit-code drift check in CI. Do not audit their prose. The only thing to flag is a partial that is stale on disk versus its generator, which you check by regenerating and diffing:

cd docs && make schema-diagrams data-dictionary prisma-analysis live-partials
git diff --stat -- docs/shared/generated/ docs/operator/reference-cli.adoc

Source-of-truth precedence

When prose, a sibling page, and the code disagree, the code wins.

1. Code: migration/ (especially lib.py, cli.py, phases/*.py), sql/, pgloader/, scripts/
2. Tests: tests/, tests/sql/, tests/integration/
3. Generated artifacts and working files: reports/, runbooks/, state/
4. Live output: make help, migrate --help, Makefile recipes

A page is wrong even when it is internally consistent and even when it matches another prose page. The developer pages and canonical-spec.adoc were the correct siblings in past audits; stale SME pages contradicted them and lost.

The method, file by file

For each page, read it once, then for every claim that names a concrete artifact, re-derive the fact and assert agreement:

• Commands and flags: does the command exist in cli.py / the Makefile? Are the flags real? (e.g. migrate ddl runs a prod-schema guard before the DROP that the prose omitted.)
• Paths: does the file exist? Cross-check docs/tools/doc_absent_paths.txt for paths already known not to ship (e.g. scripts/dev_pg.sh).
• Phase / gate semantics: does the phase do what the prose says? Read the phase module, not its name.
• Counts: count the real files/tables/checks; do not trust a number.
• Schema location: which schema actually owns the object (migration vs mysql_raw vs demos_app)?
• Controlled vocabulary: transform verbs are tracked in docs/tools/doc_vocab.txt and the generators; prose lists drift from the live set.

Look for cross-cutting roots: one wrong claim copied across several pages. Grep the wrong phrase across docs/ so you fix every copy, and fix the root claim once rather than each symptom.

rg -n "row.count delta is 0|delta = 0" docs/

Recurring staleness patterns in these docs

These are the actual errors prior audits found. Check them first; they tend to recur and they propagate across pages.

Pattern	The wrong claim	The truth
Delta gate semantics	"Gate 2: row-count delta = 0 for every table"	load_delta.py only runs assert_pgloader_ok; row-count drift is the separate, non-gating migrate load-fidelity
Parity engine	"parity runs DuckDB checks"	parity is pure Postgres (parity.py: "intentionally NOT a DuckDB job")
_delta_log schema	"_delta_log lives in the migration schema"	it lives in mysql_raw (sql/00_init/04_delta_log.sql)
BN aggregate location	"rich BN aggregate + trigger on demos_app.budget_neutrality_workbook"	the aggregate is the migration-private parity oracle migration.bn_workbook_detail; the live column is loaded empty, no trigger wired
Check counts	"preflight = 5 checks", "parity = 6 checks"	preflight runs 7 (P0.1-P0.7), parity emits 7; recount from the modules
Table/file counts	"~40 mdcd_*_pgm_dtl tables", "10_stg has 13 files"	recount from the schema snapshot / sql/ dir (was ~70 and 15)
Dead paths	cites scripts/dev_pg.sh, cmt_orgn_cd_rfrnc.csv, mmd_sql_compare/	none exist; see doc_absent_paths.txt; real ephemeral PG is make test-db-up
Inverted I/O prose	"constraints reads state/fk_violations.csv, dies if non-empty"	it writes the file from the in-memory remaining list, then dies; never reads it
Phantom defaults	"healthz defaults to https://demos.example/healthz"	new_app_healthz_url defaults to "" and die()s if unset
Stale status dates	"status as of <2 weeks ago>" in README.adoc, canonical-spec.adoc §12	bump to today; move the timeline today-marker
Silent table options	options="sortable" on a table	Ruby Asciidoctor ignores it (Asciidoctor.js only); use the .sortable role + docinfo JS

Report format

Deliver a per-file report (the prior one lives at, with:

• A per-subfolder verdict table: file, verdict, worst severity.
• Severity tiers: Safety (cutover/rollback-breaking), High (wrong command/path/phase that breaks a how-to), Medium (stale but non-breaking), Low (cosmetic/date/wording).
• Per finding: file + section anchor + the exact wrong text + a copy-pasteable proposed replacement.
• A cross-cutting-roots section: which wrong claim propagated where, so one root fix clears many pages.
• For canonical-spec.adoc (~1,500 lines), summarize per == section rather than line by line.
• A prioritized fix list: Safety, then the cross-cutting roots, then High, then Medium/Low batches.

When you find a class of error the fact net could catch automatically, recommend a new verify_doc_facts.py check for it (for example, a check that no parity page calls parity "DuckDB", or that the delta-gate prose does not claim row-count verification).

Editing the fixes

Use the asciidoc-wiki skill for the house style (one H1 per file, xref pattern, no content duplication, runnable commands). Edit separate files in parallel. The :docinfo: shared mechanism is wired through the Makefile ASCII_ATTRS, so client-side features (sortable tables) live in docs/docinfo.html.

Verify it worked

cd docs && make verify 2>&1 | tail -25   # the fact net still passes
cd docs && make html 2>&1 | tail -20     # builds clean (failure-level=WARN)

• Re-run verify_doc_facts.py and confirm any env/package fixes cleared the corresponding failures.
• Grep the wrong phrase again across docs/ to confirm no copy survived.
• If you touched a generated partial's generator, regenerate and confirm git diff is intentional, since CI fails on drift.

What not to do

• Don't trust prose over code. When a page disagrees with a sibling page or the code, the code is the source of truth. Read the phase module, not its name.
• Don't re-report what the fact net already catches. Run verify_doc_facts.py first and dedupe against its failures.
• Don't audit generated partials for prose staleness. They are regenerated by make; only flag a partial that is stale on disk versus its generator.
• Don't fix a propagated claim in one place. Grep for the wrong phrase and fix every copy, then consider a fact-net check so it cannot recur.
• Don't assert against memory of the repo. Re-derive each fact from the actual file at audit time; counts and paths change between sessions.
• Don't hand-diff the huge generated catalogs (the data dictionary, the column map). Regenerate them and git diff instead.
• Don't conflate the delta gate with load-fidelity, or parity with DuckDB. These two are the most-propagated errors in the repo.