check-docs-drift

name	check-docs-drift
description	Identify drift between docs/ folder and specification/schemas/code
allowed-tools	["Read","Grep","Glob","Write","Bash(mktemp -d /tmp/glx-drift-*:*)","Bash(./bin/glx validate:*)","Bash(rm -rf /tmp/glx-drift-*:*)","Bash(git rev-parse:*)","Bash(gh issue list:*)"]
model	claude-opus-4-8

You are tasked with identifying any drift between the GLX user documentation and the source of truth (specification, schemas, and code).

Note: Example archives (docs/examples/) are checked separately by the check-examples skill. Auto-generated CLI pages (docs/cli/) are validated by .github/workflows/docs-drift.yml via make docs-cli — do not review them here. External GEDCOM PDFs (docs/gedcom-spec/) and historical Architecture Decision Records (docs/decisions/) are not in scope. This skill focuses only on hand-authored, user-facing documentation.

Source of Truth Flow

IMPORTANT: Documentation is derived from the source of truth:

Specification (*.md) → Schema (*.schema.json) → Go Code (types.go)
         ↓
    User Docs
  (quickstart,
   guides/)

This means:

• Specification/Schema/Code are the source of truth
• Documentation is derived from these sources
• Any drift detected means the documentation needs to be updated
• When reporting drift, frame it as "Documentation X needs to be updated because source says Y"

Output contract

All findings — including a count of clean files — MUST be emitted in a single fenced findings-json block at the end of the run. See the schema at .claude/skills/check-suite/findings.schema.json and the shared severity rubric at .claude/skills/check-suite/severity-rubric.md. The eval harness greps for the info-string findings-json to extract this block; do not omit it.

The block must be one JSON object (not per-file fragments) with top-level keys:

• command: "check-docs-drift" (literal string, required)
• commit: the HEAD SHA from git rev-parse HEAD (required)
• checked_files: the actual post-exclusion list of files read this run (required; lets the eval harness detect under-scoping)
• findings: array of finding objects — empty array when no drift detected
• positive_notes: array of strings for things confirmed correct (clean files go here, not in findings as info items)
• summary: object with critical, major, minor, info counts, and suppressed_as_duplicate_of_known_issue count from the cross-reference step

Each finding object requires:

• file: repo-relative path
• severity: one of critical | major | minor | info (per the shared rubric — prefer info when uncertain)
• category: one of the values defined in the category table below
• message: what drifted and why it matters
• validator_caught: true if glx validate or CI already flags this
• llm_only: true if only the LLM semantic pass surfaced it — must be the logical inverse of validator_caught
• line: 1-based line number, or null when not line-addressable
• fix (optional): concrete suggested remediation

Category → default severity table (skill-specific, anchors to the shared rubric at .claude/skills/check-suite/severity-rubric.md):

Category	Default severity	Notes
example_validation	critical	glx validate non-zero exit on a doc code block — see fragment caveat below
required_optional	major	Doc says required; spec says optional, or vice versa
field_type	major	Field type mismatch between doc and spec
field_presence	major	Field documented but not in spec, or in spec but undocumented with additionalProperties: false context
cli_example	major	Documented CLI command/flag does not exist in code; or example no longer runs
vocabulary	major	Referenced vocabulary type does not exist in specification/5-standard-vocabularies/
cross_reference	minor	Stale internal link; link uses .md extension in URL form
validator_unavailable	info	./bin/glx could not be found; build with make build-cli or go build -o bin/glx ./glx
pedagogical	info	Minor wording mismatch where technical accuracy is preserved

Step 0 — Capture run provenance

Before any file reading:

git rev-parse HEAD

Record the printed SHA; substitute it for <HEAD_SHA> in the final JSON block.

Step 1 — Enumerate files to check

Use the Glob tool to enumerate docs/**/*.md. Exclude the following paths — they are either owned by other skills, auto-generated, or not derived from the GLX spec:

Excluded path pattern	Reason
docs/examples/**	Owned by check-examples
docs/cli/**	Auto-generated by make docs-cli; drift validated by CI
docs/decisions/**	Architecture Decision Records — historical records, not derived from current spec
docs/gedcom-spec/**	External GEDCOM reference PDFs and ancillary files

Additionally include these files that live outside docs/ but are user-facing:

• (The glossary specification/6-glossary.md is owned by check-spec, which has the dedicated glossary-consistency checks — it is intentionally not checked here. Specification entity-type files are also check-spec's responsibility.)

Also read these cross-reference surfaces when checking CLI examples and sidebar coverage. Do NOT flag these as drifting documentation — they are source-of-truth surfaces, not documentation files:

• glx/cli_commands.go — Cobra Use/Short/Long/Example declarations (the canonical CLI registry, per project CLAUDE.md)
• glx/*_runner.go — handler implementations where flags and runtime behaviour are defined
• website/.vitepress/config.js — sidebar navigation; cross-check that every published doc appears

The checked_files array is the complete post-exclusion glob result plus every cross-reference surface you actually open above (glx/cli_commands.go, the glx/*_runner.go files, website/.vitepress/config.js) — list what you read, so the provenance is complete and the eval harness can detect under-scoping.

This dynamic enumeration means new guides added to docs/ are automatically in scope on the next run without any prompt edits (resolving the recurring defect class tracked in GitHub issue #829).

Step 2 — Entity field documentation (check each file)

Compare documented fields with specification/4-entity-types/*.md:

• Check that field names match exactly (e.g., state_province not state)
• Verify required vs optional matches the specification
• Check field types are accurately described
• Use categories field_presence, field_type, required_optional as appropriate

Step 3 — Example code blocks (deterministic validator pass)

For each YAML-tagged fenced code block in a documentation markdown file:

1. Extract the YAML body from the markdown source.

2. Classify the block by intent — only blocks that present a complete archive or entity get deterministically validated; partial illustrations must NOT be forced through validation (they would false-fail). glx validate <file> validates archive-shape input; glx validate --stdin --entity-type <type> (#910) validates a complete bare entity/vocabulary-entry body:

   • Archive-shaped — top-level keys are entity-collection names (persons:, events:, …), vocabulary-collection keys, or metadata:. This also covers a single entity keyed by its ID under its collection (e.g. persons: → person-jane: → body). Validate as a whole archive (step 4a).
   • Complete single entity / vocabulary entry — the block shows the full body of one entity or vocabulary entry (the fields a real one carries; type clear from context). If it is keyed by a bare entity ID with no collection wrapper (e.g. person-jane: → body), use the value under that ID key as the body — do not feed the person-jane: wrapper to --stdin (the ID would become an unknown top-level property). Validate via --stdin --entity-type <type> (step 4b).
   • Focused / partial illustration — the block deliberately shows only part of an entity to illustrate one field or sub-shape (e.g. a lone participants: list, a properties: excerpt, a single date: field), omitting other required fields on purpose. This is the dominant false-positive source. Do not run structural validation — record a positive_notes note ("illustrative partial fragment — not structurally validated") and rely on the narrative checks in Steps 2 and 4.
   • Non-object snippet — a scalar or bare sequence that is not an entity/vocabulary entry at all. Skip structural validation the same way.

3. Create a unique scratch directory and capture its path:

mktemp -d /tmp/glx-drift-XXXXXX

Read the printed path (e.g. /tmp/glx-drift-aB3xYz); substitute it for <TMPDIR> in the remaining steps. Bash sessions don't persist between tool calls, so each subsequent command must use the literal path you captured.

4. Use the Write tool to save the extracted YAML body to <TMPDIR>/snippet.glx. Using Write rather than shelling out via cat/printf/heredoc avoids every shell-quoting concern around YAML bodies that contain $, backticks, or quotes — for both paths below. Then run the validator (do NOT mentally simulate schema validation):

   4a — archive-shaped block:

./bin/glx validate <TMPDIR>/snippet.glx

4b — entity / vocabulary fragment (redirect the file into stdin so the body is never re-quoted on a pipe):

./bin/glx validate --stdin --entity-type <type> < <TMPDIR>/snippet.glx

<type> is the entity singular (person, event, …) or vocabulary-collection key (event_types, place_types, …) inferred from the block's context. --stdin runs Pass-1 structural validation only (no cross-reference resolution), which is exactly right for a context-free fragment.

5. Clean up this snippet's scratch directory before moving to the next one:

rm -rf <TMPDIR>

Per-snippet cleanup keeps concurrent runs from racing on the /tmp/glx-drift-* namespace.

6. Classify the result deterministically by exit code alone:

   • Exit 0 → snippet passed validation. Record this as a positive_notes entry — do not emit a finding (a clean validation is not drift, and validator_caught: false + llm_only: false is not a valid finding combination). Still apply the narrative checks in steps 2 and 4 — the validator does not catch specification-prose drift.
   • Exit 1 (structural/semantic failure) → normally category: example_validation, default severity critical (validator_caught: true, llm_only: false); put the validator's stderr verbatim in message. Carve-out for partial illustrations: if the only errors are missing property '<field>' / missing properties … and the block is a focused/partial illustration (step 2), it was mis-classified as complete — this is not drift. Record a positive_notes "illustrative partial fragment" note instead of a finding. Keep critical when the failure shows additional properties … not allowed, a type mismatch, a pattern or array-length (minItems) violation, or a missing-required field on a block the page presents as its complete canonical example.
   • Exit 2 (bad invocation — only reachable on the --stdin path: unknown/missing --entity-type, or an empty body) → this means the skill picked the wrong --entity-type or mis-classified the block, not documentation drift. Re-classify (the block was usually a sub-object excerpt that step 2 should skip) and re-run, or record one category: validator_unavailable, severity: info note (with validator_caught: false, llm_only: true — see below). Never emit an exit-2 result as a critical example finding.

   If ./bin/glx is unavailable, emit one category: validator_unavailable, severity: info finding and build with make build-cli (or go build -o bin/glx ./glx) if needed before proceeding. Per the suite convention (.claude/skills/check-suite/README.md → Cross-finding conventions), a validator_unavailable finding carries validator_caught: false, llm_only: true — the schema's oneOf forbids false/false, and it is the LLM run, not the validator, that surfaced the gap.

Step 4 — CLI command examples

Cross-reference against glx/cli_commands.go (Use/Short/Long/Example declarations) and glx/*_runner.go (handler implementations, where flags are registered):

• Verify every documented CLI command appears in glx/cli_commands.go
• Verify flags and arguments documented in prose match flags registered in the corresponding *_runner.go
• Verify output examples match the command's actual documented behavior
• Use category cli_example for any mismatch

Note: the glob pattern is glx/cli_commands.go plus glx/*_runner.go, NOT glx/cmd_*.go — the cmd_*.go pattern mostly matches test files and misses the 29 production command implementations (GitHub issue #800).

Step 5 — Vocabulary references

Check referenced vocabulary types against specification/5-standard-vocabularies/:

• Referenced vocabulary types must exist
• Example values must be drawn from the standard vocabularies

Use category vocabulary.

Step 6 — Internal links

• Verify internal cross-doc links resolve to actual files
• Internal links must use the URL form with no .md extension, per the project's documentation URL convention

Use category cross_reference.

Step 7 — Cross-Reference with Known Issues

Before finalizing the report, suppress findings that are already tracked:

gh issue list -R genealogix/glx --state open --limit 100 --json number,title --jq '.[] | "#\(.number) \(.title)"'

For each finding, scan the issue title list for overlap on the doc file path, field name, or command name. If the finding is already tracked in an open issue, omit it from findings but increment suppressed_as_duplicate_of_known_issue in summary. If the finding is not tracked, include it in full.

This prevents duplicate triage on every run.

Step 8 — Emit the findings-json block

After all checks and the cross-reference step, emit the human-readable summary followed by the machine-readable block.

Human-readable summary format:

# Documentation Drift Report

## Scope
- Commit: <HEAD_SHA>
- Files checked: N

### docs/quickstart.md
No drift detected.

### docs/guides/hands-on-cli-guide.md
Drift detected — documentation needs updates:
- Line 42: ...

Then emit the machine-readable block (the eval harness greps for the info-string):

{
  "command": "check-docs-drift",
  "commit": "<HEAD_SHA>",
  "checked_files": ["docs/quickstart.md", "..."],
  "findings": [
    {
      "file": "docs/quickstart.md",
      "line": 42,
      "severity": "major",
      "category": "required_optional",
      "message": "Documentation says `name` is required, specification says optional",
      "fix": "Change 'required' to 'optional' in the field description at line 42",
      "validator_caught": false,
      "llm_only": true
    }
  ],
  "positive_notes": [
    "docs/guides/best-practices.md: all field names and types match specification"
  ],
  "summary": {
    "critical": 0,
    "major": 1,
    "minor": 0,
    "info": 0,
    "suppressed_as_duplicate_of_known_issue": 2
  }
}

Common issues to look for

• Outdated CLI commands or flags (cross-reference glx/cli_commands.go and glx/*_runner.go)
• Incorrect required/optional field documentation
• Examples in prose with invalid YAML syntax
• References to deprecated or renamed fields (e.g., state vs state_province — the latter is the current field name)
• Wrong vocabulary type names
• Internal links using .md extension in URL form (should be omitted per the project's URL convention)

Notes

• Source of truth hierarchy: Specification → Schema → Code → Documentation
• User documentation errors are high severity — users rely on these
• Minor wording differences are acceptable if technical accuracy is preserved
• Focus on technical accuracy, not writing style
• CLI examples should be copy-paste ready
• The glossary (specification/6-glossary.md) is owned by check-spec, not this skill — do not check it here
• See .claude/skills/check-suite/severity-rubric.md for the full shared severity scale; do not redefine it here
• See .claude/skills/check-suite/findings.schema.json for the complete output schema