| name | check-harness-docs |
| description | Use when adding or editing a skill, script, schema, or pipeline stage in the ai-security-harness repo, or before committing any change to its docs — runs scripts/check_docs_consistency.py to verify the harness's own documentation (README.md, AGENTS.md, PROCESS.md, docs/) has not drifted from the tree. Checks that "N skills / N commands / N Python scripts / N-stage" count claims match the actual tree counts, that every repo-relative Markdown link and in-repo backtick path still resolves, and — when the gitlab-profile sibling is checked out — that the GitLab group README's harness-version and skill/command-count claims match too. Use when asked to "check the harness docs", "did I break the docs", "verify doc consistency", or "is anything stale after this change". |
| argument-hint | [--root DIR] |
| user-invocable | true |
| allowed-tools | ["Bash(python3:*)","Read","Edit","Glob","Grep"] |
Check harness docs for drift
A guardrail for the harness's own documentation. The docs repeatedly state
countable facts about the tree — "26 active skills", "31 slash command
wrappers", "Twelve Python scripts", "8-stage pipeline" — and cross-reference
files by path. Both rot silently as the harness grows. This skill runs the
deterministic checker and, if it finds drift, fixes the docs to match the
tree.
When to use
Run it whenever you:
- add / rename / remove a skill under
harnessing/,
- add / remove a script under
scripts/, a schema under schema/, or a
command wrapper under .claude/commands/,
- add or remove a
## Stage N in PROCESS.md,
- bump
VERSION (the group README states the harness version), or
- edit any of
README.md, AGENTS.md, PROCESS.md, or docs/*.md.
Run it
python3 ai-security-harness/scripts/check_docs_consistency.py
Exit 0 = docs consistent. Exit 1 = drift; each issue is printed. The same
logic runs in the test suite as tests/test_docs_consistency.py.
What it checks
-
Count assertions. Global inventory claims in README.md / AGENTS.md /
PROCESS.md / docs/*.md must equal the live tree count:
| Claim | Source of truth |
|---|
N skills / N active skills / all N skills | harnessing/*/SKILL.md |
N slash command | .claude/commands/*.md |
N Python scripts (digit or word form) | scripts/*.py |
N-stage | ## Stage N headings in PROCESS.md |
| Expected numbers are computed from the tree, so the checker never itself | |
| goes stale. | |
-
Dead paths. Every Markdown link with a repo-relative target and every
backtick path under a known in-repo dir (harnessing/, scripts/,
schema/, docs/, .claude/, .crush/, config/, tests/) must resolve.
Sibling-repo paths (analysis-results/, …), externals (http(s)),
glob/placeholder patterns (*, <repo>, **), and historical records
(CHANGELOG.md) are intentionally skipped.
-
Group README. The GitLab group profile page
(../gitlab-profile/README.md, the sibling checkout of
https://gitlab.cee.redhat.com/hybrid-platforms-sec/gitlab-profile) also
states harness facts. When the sibling is present, the checker vets:
| Claim | Source of truth |
|---|
Harness version: vX.Y.Z / harness **vX.Y.Z** (any vX.Y.Z on a line mentioning "harness") | the harness VERSION file |
N skills / Skills (N), by function | harnessing/*/SKILL.md |
N slash-command wrappers / N commands | .claude/commands/*.md |
| When the sibling is not checked out, the check is skipped (and says so). | |
How to fix drift
Update the docs, not the checker. The counts are computed from the tree, so
a failure means a doc is wrong, not the script. For a count failure, correct the
number in the named doc (and, for a new skill, add its row + section to
docs/skills.md). For a dead path, repoint or remove the reference. Re-run
until it exits 0. Only touch scripts/check_docs_consistency.py itself to
change what is checked (e.g. add a new count claim or an allowlist entry).
Group-README drift is fixed in the sibling repo: edit
../gitlab-profile/README.md (version banner, repository-map row, key-contents
table, skills-by-function table — a new skill needs a row in its function
group), re-run the checker, then commit and push that repo separately — its
page is the group's public face and does not ship with the harness.
