| name | docs-validating-factual-accuracy |
| description | Universal methodology for verifying factual correctness in documentation using WebSearch and WebFetch tools. Covers command syntax verification, version checking, code example validation, API correctness, confidence classification system ([Verified], [Error], [Outdated], [Unverified]), source prioritization, and update frequency rules. Essential for maintaining factual accuracy in technical documentation and educational content |
Factual Validation Methodology Skill
Purpose
This Skill provides a universal methodology for verifying factual correctness in technical documentation, educational content, and code examples using WebSearch and WebFetch tools.
When to use this Skill:
- Verifying command syntax and flags
- Checking version numbers and compatibility
- Validating code examples compile/run correctly
- Confirming API methods and signatures
- Verifying external claims and references
- Classifying confidence levels for documentation
- Maintaining factual accuracy in tutorials
Core Concepts
What is Factual Validation?
Factual validation is the systematic process of verifying objective, verifiable claims in documentation against authoritative sources using web search and retrieval tools.
Validates:
- Command syntax (bash, npm, git commands)
- Software version numbers
- API method names and signatures
- Code example correctness
- Configuration file formats
- Library/package availability
- External factual claims
Does NOT validate:
- Subjective quality assessments
- Narrative flow or writing style
- Architectural decisions or opinions
- Future predictions or speculation
The Four Confidence Classifications
All validation findings use one of four confidence labels:
[Verified] - Objectively correct according to authoritative sources
- Command syntax matches official documentation
- Version number confirmed via package registry
- Code example compiles/runs as shown
- API method exists with correct signature
[Error] - Objectively incorrect, breaks functionality
- Command syntax wrong (fails when executed)
- Code example won't compile/run
- API method doesn't exist or has wrong signature
- Incorrect version number that doesn't exist
[Outdated] - Was correct but now superseded by newer version
- References old major version with breaking changes
- Uses deprecated API (still works but replacement exists)
- Command syntax changed in recent release
- Configuration format updated
[Unverified] - Cannot confirm correctness (insufficient evidence)
- No authoritative source found
- Multiple conflicting sources
- Documentation ambiguous or incomplete
- Claim too specific to verify externally
Validation Workflow
Step-by-Step Process
1. Identify Claims to Validate
Extract objective, verifiable claims from content:
Content: "Install using `npm install --save-deps prettier`"
Claim: npm flag "--save-deps" is valid syntax
Type: Command syntax
2. Determine Source Priority
Use authoritative sources in priority order:
- Official documentation (docs.npmjs.com, official GitHub repos)
- Package registries (npmjs.com, pypi.org, crates.io)
- Official release notes (CHANGELOG, GitHub releases)
- Well-maintained community sources (Stack Overflow official answers, MDN)
3. Execute Web Search
WebSearch query: "npm install flags official documentation"
Target: Find official npm CLI documentation
4. Fetch Authoritative Content
WebFetch: https://docs.npmjs.com/cli/v9/commands/npm-install
Extract: List of valid flags and their purposes
5. Compare and Classify
Claim: "--save-deps" flag
Source: Official npm docs list "--save-dev" (NOT "--save-deps")
Classification: [Error]
Confidence: HIGH (official docs contradict claim)
6. Document Finding
**File**: `docs/tutorials/quick-start.md:42`
**Verification**: [Error] - Command syntax incorrect
**Criticality**: CRITICAL - Breaks user quick start experience
**Category**: Factual Error - Command Syntax
**Finding**: Installation command uses incorrect npm flag `--save-deps` (should be `--save-dev`)
**Impact**: Users following tutorial get command error, cannot complete setup
**Recommendation**: Change `npm install --save-deps prettier` to `npm install --save-dev prettier`
**Verification Source**:
Official npm documentation confirms `--save-dev` is correct flag
https://docs.npmjs.com/cli/v9/commands/npm-install
**Confidence**: HIGH
Source Prioritization
Tier 1: Official Documentation (Highest Authority)
Examples:
- Official language documentation (docs.python.org, docs.oracle.com/javase)
- Official package documentation (npmjs.com package pages, PyPI)
- Official CLI references (git-scm.com, docs.npmjs.com)
- Official API documentation (developer.mozilla.org, official SDK docs)
Characteristics:
- Maintained by project creators
- Canonical source of truth
- Version-specific
- Updated with releases
When to use: For syntax, commands, API signatures, version numbers
Tier 2: Package Registries (Version Truth)
Examples:
- npmjs.com (JavaScript/Node.js packages)
- pypi.org (Python packages)
- crates.io (Rust crates)
- rubygems.org (Ruby gems)
- maven.org (Java packages)
Characteristics:
- Authoritative version numbers
- Availability confirmation
- Dependency information
- Release dates
When to use: For version verification, package availability, dependency checks
Tier 3: Official Release Notes (Change Truth)
Examples:
- GitHub Releases pages
- CHANGELOG.md files in official repos
- Official blog announcements
- Migration guides
Characteristics:
- Documents changes between versions
- Breaking change identification
- Deprecation notices
- Migration paths
When to use: For outdated content detection, breaking change verification
Tier 4: Well-Maintained Community (Supplementary)
Examples:
- Stack Overflow accepted answers (with high votes)
- MDN Web Docs (Mozilla Developer Network)
- Official tutorial sites (RustByExample, GoByExample)
Characteristics:
- Community-verified
- Practical examples
- Context-dependent correctness
- May lag behind latest versions
When to use: For practical patterns, edge cases, when official docs insufficient
Common Validation Patterns
Pattern 1: Command Syntax Validation
Claim Type: Bash/shell command syntax
Validation Steps:
- Extract command and flags from content
- WebSearch: "[command-name] official documentation"
- WebFetch: Official CLI reference page
- Compare: Flags, argument order, option names
- Classify: [Verified] if exact match, [Error] if wrong
Example:
Content: "git commit -am 'message'"
WebSearch: "git commit flags official documentation"
WebFetch: https://git-scm.com/docs/git-commit
Verify: -a flag exists, -m flag exists, combined -am valid
Result: [Verified]
Pattern 2: Version Number Validation
Claim Type: Software/package version reference
Validation Steps:
- Extract package name and version number
- WebSearch: "[package-name] [version] release"
- WebFetch: Package registry page (npmjs.com, pypi.org)
- Compare: Version exists in registry
- Check: Latest version, deprecation status
- Classify: [Verified], [Outdated], or [Error]
Example:
Content: "Install React 17.0.2"
WebSearch: "react 17.0.2 npmjs"
WebFetch: https://www.npmjs.com/package/react
Verify: 17.0.2 exists in version list
Check: Latest is 18.2.0 (major version ahead)
Result: [Outdated] - Breaking changes exist in v18
Pattern 3: Code Example Validation
Claim Type: Code snippet correctness
Validation Steps:
- Extract code example from content
- Identify language and version
- WebSearch: "[language] [concept] official examples"
- WebFetch: Official documentation or tutorial
- Compare: Syntax, method signatures, patterns
- If possible: Run code locally to verify execution
- Classify: [Verified], [Error], or [Outdated]
Example:
Content:
```python
import asyncio
async def main():
await asyncio.sleep(1)
asyncio.run(main())
```
WebSearch: "python asyncio.run documentation"
WebFetch: https://docs.python.org/3/library/asyncio-task.html
Verify: asyncio.run() syntax correct, requires Python 3.7+
Check: No syntax errors, runnable code
Result: [Verified]
Pattern 4: API Method Validation
Claim Type: API method existence and signature
Validation Steps:
- Extract API method name, parameters, return type
- WebSearch: "[library] [method] official api documentation"
- WebFetch: Official API reference
- Compare: Method name, parameter types, return type
- Check: Deprecation status
- Classify: [Verified], [Error], or [Outdated]
Example:
Content: "Use fs.readFileSync(path, 'utf-8')"
WebSearch: "node.js fs.readFileSync official documentation"
WebFetch: https://nodejs.org/api/fs.html#fsreadfilesyncpath-options
Verify: Method exists, parameters match (path, options)
Check: 'utf-8' encoding valid
Result: [Verified]
Update Frequency Rules
When to Re-validate Content
MANDATORY re-validation triggers:
- 6 months since last validation - Standard refresh cycle
- Major version release of referenced software/library
- Breaking change announced in official release notes
- User reports error in documentation
- Deprecation notice for used API/method
OPTIONAL re-validation triggers:
- Minor version updates (if content specific to that version)
- Patch releases (usually safe to skip)
- Community feedback about potential issues
- Regular documentation review cycles
Validation Metadata Storage
Location: docs/metadata/external-links-status.yaml
Format:
factual-validations:
- file: "docs/tutorials/quick-start.md"
claim: "npm install --save-dev prettier"
verification-status: "[Verified]"
last-checked: "2025-12-27T10:00:00+07:00"
source: "https://docs.npmjs.com/cli/v9/commands/npm-install"
expires: "2026-06-27T10:00:00+07:00"
Integration with Checker Agents
Dual-Label Pattern
Factual validation findings require BOTH verification label AND criticality level:
### 1. [Error] - Command Syntax Incorrect in Installation Guide
**File**: `docs/tutorials/quick-start.md:42`
**Verification**: [Error] - Command syntax verified incorrect via WebSearch
**Criticality**: CRITICAL - Breaks user quick start experience
**Category**: Factual Error - Command Syntax
**Finding**: [description]
**Impact**: [consequences]
**Recommendation**: [fix]
**Verification Source**: [URL]
**Confidence**: HIGH
Why dual labels?
- Verification ([Verified]/[Error]/[Outdated]/[Unverified]) describes FACTUAL STATE
- Criticality (CRITICAL/HIGH/MEDIUM/LOW) describes URGENCY/IMPORTANCE
Both dimensions provide complementary information.
Confidence Assessment
HIGH confidence when:
- Official documentation clearly contradicts claim
- Package registry shows version doesn't exist
- Code example fails to compile (verified locally)
- Multiple authoritative sources agree
MEDIUM confidence when:
- Sources partially contradict (some say yes, some say no)
- Claim is version-specific but version unclear
- Documentation ambiguous or incomplete
- Cannot find definitive authoritative source
Factual Validation Agents
These agents implement this methodology end-to-end (check → finding → fix):
- docs-checker — Validates documentation factual accuracy
- docs-tutorial-checker — Validates tutorial factual accuracy
All use the same validation workflow and confidence classification.
Delegate Research to web-researcher for Context Isolation
The authoritative rule for when to delegate public-web research lives in the
Web Research Delegation Convention.
This skill follows that convention and does not re-state the threshold here. In summary: delegate to
web-researcher whenever verifying a single claim requires 2+ WebSearch calls or 3+ WebFetch calls;
in-context work remains correct for single-shot verification against a known authoritative URL and for
fixer agents re-validating a single audit finding.
Delegation pattern:
Agent tool call → subagent_type: web-researcher
Prompt: "Verify whether <specific claim>. Return cited findings with confidence tags."
web-researcher returns a synthesised, cited summary using the same [Verified]/[Outdated]/[Unverified]/[Needs Verification] confidence tags defined in this skill. You then translate those tags into dual-labelled findings ([Verified]/[Error]/[Outdated]/[Unverified] + CRITICAL/HIGH/MEDIUM/LOW) for your audit report.
See web-researcher for the agent contract and the
Web Research Delegation Convention
for the normative rule and its enumerated exceptions.
Common Mistakes
❌ Mistake 1: Using outdated sources
Wrong: Citing 5-year-old blog post for current syntax
Right: Check official documentation for latest version
❌ Mistake 2: Missing version context
Wrong: "React hooks were introduced recently"
Right: "React hooks were introduced in React 16.8 (February 2019)"
❌ Mistake 3: Trusting unofficial sources
Wrong: Using random Stack Overflow answer as sole source
Right: Verify with official docs, use SO for supplementary context
❌ Mistake 4: Not documenting verification source
Wrong: Marking [Verified] without citing source
Right: Always include verification source URL in finding
❌ Mistake 5: Conflating verification with subjective quality
Wrong: [Error] for "code style not following best practices"
Right: Use [Error] only for objective incorrectness (won't compile, wrong syntax)
Best Practices
Validation Checklist
Before marking content as validated:
Batch Validation Workflow
- Extract claims: Scan content for all verifiable claims
- Group by type: Commands, versions, code examples, APIs
- Prioritize: Critical paths first (install commands, quick starts)
- Validate systematically: One claim type at a time
- Document findings: Use standardized format
- Update metadata: Record validation dates and sources
Tool Usage Pattern
Step 1: WebSearch to find authoritative source
Query: "npm install flags official documentation"
Result: Multiple sources, identify official docs
Step 2: WebFetch to retrieve content
URL: https://docs.npmjs.com/cli/v9/commands/npm-install
Extract: Flag list, examples, usage patterns
Step 3: Compare and classify
Claim vs Source → Determine classification
Document finding with source citation
Reference Documentation
Primary Convention: Factual Validation Convention
Related Conventions:
Related Skills:
criticality-confidence-system - Understanding dual-label system and priority matrix
Related Agents:
docs-checker - Documentation factual validationdocs-tutorial-checker - Tutorial factual validation
This Skill packages critical factual validation methodology for maintaining accuracy in technical documentation. For comprehensive details, consult the primary convention document.
References
Primary Convention: Factual Validation Convention
Related Conventions:
Related Skills:
repo-assessing-criticality-confidence - Understanding dual-label system and priority matrix
