| name | review-docs |
| description | Review agentic documentation — verify claims locally against source code first, then use chai-bot for cross-repo and cross-functional verification |
| trigger | explicit |
Documentation Review & Verification
Reviews agentic documentation using two-tier verification: local codebase checks first, then cross-repo verification via chai-bot MCP server for claims that can't be resolved locally. Verify locally when possible, escalate to chai-bot when necessary.
Prerequisites
Recommended (for cross-repo verification): chai-bot MCP server configured with bearer token and Red Hat VPN connection. Without chai-bot, cross-repo claims (enhancements, platform terminology, convention compliance, non-vendored API types) are flagged as "unverified" but local verification still runs fully.
Setup instructions: See plugin README for chai-bot configuration steps.
When to Use
- After
/component-docs or /update-platform-docs
- Before documentation PRs
- When docs may contain outdated/incorrect information
Limitations
This skill verifies factual accuracy of what's documented. It does NOT detect:
- Missing documentation — incomplete coverage, absent sections
- Low information density — generic placeholders instead of repo-specific details
- Misleading by omission — technically correct but useless (e.g., "uses controllers" without specifying which framework)
For completeness and specificity, rely on the Implementation Pattern Discovery checklist in /component-docs and SME review.
Claim Classification
Classify each claim by what it asserts. A single doc sentence may contain both a local fact and a cross-repo assertion — extract these as separate claims.
Local: "Can this be verified from the current repo?"
Verifiable by reading the repo's own source code, including vendor/:
- File/directory paths exist
- Makefile targets exist
- Go version — check both the
go directive (minimum version) and toolchain directive (actual toolchain) in go.mod
- Default branch name
- Import statements present (which frameworks a controller imports)
- Code patterns present (apply/update/patch calls, webhook registration code, feature gate checks)
- Go symbols referenced in examples exist
- YAML examples parse without errors
- Internal doc links resolve
- External links return HTTP 200 (with caveats — see Phase 3)
- Cross-file consistency (same concept not contradicted across doc files)
- Test directories and framework imports exist
- API/CRD field names, types, defaults — if
vendor/github.com/openshift/api exists, verify struct definitions from vendored types.go files
- Feature gate definitions — if
vendor/github.com/openshift/library-go or vendor/github.com/openshift/api exists, verify gate names and stages from vendored code
- API group/version — if vendored, check
register.go for GroupName and SchemeGroupVersion
Cross-repo: "Is this correct against external sources?"
Claims that cannot be resolved from local code or vendor/:
- API/CRD fields, feature gates, API group/version when not vendored, or when checking whether the vendored version is current
- Enhancement existence and status (in
openshift/enhancements)
- Official terminology (must match
openshift-docs)
- Cross-component interactions (how other operators behave)
- Platform convention compliance (whether a locally-verified pattern follows platform norms)
- Platform pattern references (whether linked platform docs are accurate)
- Historical context (design decisions in Slack/Jira, not code)
Examples
| Documentation Says | Local Claim | Cross-Repo Claim (chai-bot) |
|---|
| "Uses admission webhooks" | Webhook registration code exists in repo | Webhooks follow platform conventions |
| "Applies resources via SSA" | SSA apply calls exist in controller code | Field manager names match platform norms |
| "Feature gate TechPreviewNoUpgrade controls X" | Code checks for this gate name | Gate exists in openshift/api with claimed stage (if not vendored) |
Always run the local check first. If a claim fails locally, report it — no need to query chai-bot.
Execution Workflow
Phase 1: Document Discovery
Phase 2: Extract & Classify Claims
For each file, systematically extract every verifiable claim and track it internally as a claims inventory — the running list of assertions that drives all subsequent verification. For each claim, note: the source file, line number or range, what is being asserted, and whether it is local or cross-repo.
Extraction rules:
- A "claim" is any assertion that can be confirmed or refuted against source code, APIs, or external references. This includes prose statements, code snippet comments, numeric values, symbol names in instructions, enum constraints, and command/target names.
- Treat every line of documentation content as potentially containing one or more claims. If a line contains no verifiable assertion, skip it — but err on the side of extraction.
- A doc file with N lines of substantive content should typically yield claims proportional to its density. If a 40-line section yields only 3 claims, re-read it — something was missed.
Cross-file consistency: After extracting claims from all files, check for internal contradictions — the same concept described differently across files (e.g., AGENTS.md says "uses SSA" while components.md says "strategic merge"). Flag these before any verification queries.
The claims inventory is internal — it is not written to a file or shown to the user. But every claim in it must receive a verification status (verified, failed, or skipped) before the review can proceed, and the final report must include coverage totals derived from it.
Phase 3: Local Codebase Verification
Verify all local claims from the Phase 2 claims inventory against the current repo's source code. Every local claim must receive a status: verified, failed, or skipped (with justification). Do not proceed to Phase 4 until all local claims have a status.
Build & toolchain:
Directory & file structure:
Framework & pattern claims:
Vendored API types & feature gates (if vendor/github.com/openshift/api or vendor/github.com/openshift/library-go exists):
Code examples:
Naming conventions:
Test organization:
Links:
Cross-file consistency:
Phase 3.5: Self-Audit Re-Read
Re-read each documentation file and compare it against the claims inventory from Phase 2. The goal is to catch claims that were missed during initial extraction — lines that contain verifiable assertions but were not included in the inventory.
For each missed claim found: add it to the inventory, verify it immediately (local or cross-repo classification), and update its status. This step addresses the pattern where category-level verification feels complete but individual claims within those categories were never extracted.
Phase 4: Cross-Repo Verification via chai-bot
Verify all cross-repo claims that could not be resolved locally.
Step 1 — Check chai-bot availability:
Step 2 — Batch verification via chai-bot:
Batch related claims into grouped queries to reduce round-trips — each call takes ~15-25 seconds. Group all field claims for the same API type into one query, all convention claims for the same repo into one query, all enhancement references into one query, etc.
Question construction templates (substitute {component}, {api-type}, etc.):
API fields (batch all fields for one type): "In github.com/openshift/api,
what fields are defined in the {api-type}Spec struct? Please list field
names, types, and any documented default values from the actual Go type
definition."
Feature gates (batch per operator): "What feature gates are defined for
{component} in openshift/api? List gate names, stages, and version
information."
Enhancements (batch related): "Do the following enhancements exist in
openshift/enhancements, and do they match the claimed descriptions?
1. enhancements/{area}/{enhancement-file-1}.md
2. enhancements/{area}/{enhancement-file-2}.md"
Terminology: "In the official OpenShift documentation, how is {component}
described? What is the correct terminology?"
Cross-component: "How does {other-component} interact with {component}
during {operation}?"
Convention check: "What are the platform conventions for {pattern} in
OpenShift operators? Does the pattern used by {component} match?"
Phase 5: Report Findings
Severity guide:
- Critical — factually wrong; would cause an agent to produce incorrect code (wrong fields, wrong methods, wrong framework)
- Warning — outdated, imprecise, or missing reference; won't cause broken code but degrades trust
- Cross-file inconsistency — same concept described differently across files
- Unverified — couldn't confirm or deny (chai-bot uncertain or unavailable for cross-repo claims)
Summarize findings directly to the user with:
- Coverage metrics: Total claims extracted, total verified, total failed, total skipped — broken down by local vs cross-repo. This gives the user a concrete signal of review thoroughness.
- Verification source breakdown: Local codebase vs chai-bot vs unverified.
- Issues by severity: Listed per the severity guide above.
Phase 6: Offer Fixes
Success Criteria
- All scoped documentation files reviewed (not sampled)
- Claims inventory tracked internally with every verifiable assertion extracted — coverage metrics reported
- Every local claim in the inventory has a verification status (verified, failed, or skipped with justification)
- Self-audit re-read completed — no unextracted claims remain
- Links valid, cross-file consistency confirmed
- All cross-repo claims verified via chai-bot or flagged as unverified
Arguments
/review-docs [--path <docs-path>] [--auto-fix] [--local-only]
--path: Path to documentation file(s) to review (defaults to current directory)--auto-fix: Automatically fix issues found (default: prompt user)--local-only: Skip chai-bot verification entirely, only run local checks
Auto-discovery: Infers component repo from git remote or current working directory
See Also
/component-docs - Create component documentation/update-platform-docs - Update platform documentation- openshift/api - OpenShift API types
- openshift-docs - Official documentation (terminology cross-check)
- chai-bot - OpenShift AI helpdesk (MCP server)
