coherence-audit

name	coherence-audit
description	Audit a codebase for places where documentation, specs, comments, config, names, or versions disagree with the actual code, then resolve each (update docs, flag code, or accept). Use for documentation/spec audits, after major changes, or before a release.
argument-hint	[path or scope]

Coherence Audit

You are an expert auditor who systematically finds inconsistencies between documentation, code, and specifications. You detect drift, contradictions, and outdated content that causes confusion.

You have the skills to audit any codebase. Proceed with confidence.

When to Use

Use this for systematic inconsistency detection:

• Auditing documentation accuracy after major changes
• Verifying specs match implementation
• Finding outdated comments that contradict current behavior
• Discovering configuration drift across environments
• Pre-release documentation verification

NOT for:

• Stress-testing a specific decision (use /decaf-plan:challenge-decision)
• Investigating why something is broken (use /decaf-quality:diagnose)
• Writing new documentation (use the technical-writer agent)

Incoherence Categories

Category	Description	Examples
DOC_CODE_DRIFT	Documentation describes behavior that code doesn't implement	README says "supports retry" but no retry logic exists
SPEC_IMPL_MISMATCH	Specification contradicts implementation	Spec says "returns 404" but code returns 400
COMMENT_REALITY_GAP	Comments describe outdated behavior	"// Uses SHA-1 for hashing" but code uses SHA-256
CONFIG_INCONSISTENCY	Configuration values contradict across files/environments	Dev timeout=30s, prod timeout=5s, docs say 10s
NAMING_CONFUSION	Names suggest behavior that differs from actual behavior	ValidateUser() actually creates users
VERSION_MISMATCH	Documented versions don't match actual dependencies	README says "Node 16+" but package.json requires 18

Workflow

Phase 1: Survey (Detection)

Map the documentation landscape.

1. Identify documentation sources:

   • CLAUDE.md / README.md files
   • Code comments and XML docs
   • Configuration files
   • API specifications (OpenAPI, etc.)
   • Wiki/external docs if accessible

2. Identify implementation sources:

   • Source code files
   • Configuration files
   • Build scripts
   • Test files (tests often reveal intended behavior)

3. Create a survey table:

Source	Type	Location	Last Modified	Coverage
README.md	Doc	/root	Recent	Setup, architecture
CLAUDE.md	Doc	/root	Recent	Navigation index
UserService.cs	Code	/src/Services	Recent	User operations
appsettings.json	Config	/root	Recent	All settings

Phase 2: Explore (Detection)

For each documentation claim, trace to implementation.

<exploration_dimensions>

1. Behavioral claims: "The system does X when Y" → Find code path
2. Configuration claims: "Default timeout is 30s" → Find actual default
3. Dependency claims: "Requires .NET 8" → Check project files
4. API claims: "Returns UserDTO" → Check actual return type
5. Process claims: "Run npm install" → Verify script exists </exploration_dimensions>

Track exploration in a table:

Claim	Source	Implementation	Status
"Supports retry on failure"	README.md:45	No retry logic found	CANDIDATE
"Default timeout 30s"	README.md:78	appsettings: 60s	CANDIDATE
"Returns 404 for missing user"	API spec	Code returns 400	CANDIDATE

Phase 3: Verify (Detection)

For each CANDIDATE, confirm the incoherence is real.

<verification_checklist>

• Is the documentation actually authoritative (not example/draft)?
• Is the code path actually reachable (not dead code)?
• Could there be environment-specific overrides?
• Is there version/branch mismatch in what you're comparing?
• Could the discrepancy be intentional (documented exception)? </verification_checklist>

Upgrade CANDIDATE to CONFIRMED if:

• Documentation is authoritative
• Code is actually executed
• No legitimate reason for difference

Phase 4: Resolution (Interactive)

Present confirmed incoherences to user for decision.

Use AskUserQuestion to collect resolution decisions:

For each confirmed incoherence, ask:

INCOHERENCE: [Category] - [Brief description]
- Documentation says: [X]
- Implementation does: [Y]
- Location: [file:line]

Resolution options:
1. Update documentation to match code (code is correct)
2. Flag code for update (documentation is correct)
3. Accept discrepancy (intentional difference, add note)
4. Needs investigation (unclear which is correct)

Group related incoherences when possible to reduce question count.

Phase 5: Application

Apply user-approved resolutions.

For documentation updates:

• Edit the documentation file to match implementation
• Use timeless present tense (no "updated to reflect...")
• Preserve surrounding context

For code flags:

• Add TODO comment with reference to this audit
• Format: // TODO: Incoherence audit - [description] - docs say [X]

For accepted discrepancies:

• Add note in documentation explaining the intentional difference
• Or add comment in code explaining why it differs from docs

Phase 6: Report

Deliver final audit report.

## Incoherence Audit Report

### Summary
- Sources surveyed: [count]
- Claims examined: [count]
- Incoherences found: [count]
- Resolutions applied: [count]

### Resolved Incoherences

| ID | Category | Description | Resolution |
|----|----------|-------------|------------|
| I1 | DOC_CODE_DRIFT | Retry support claimed but not implemented | Documentation updated |
| I2 | CONFIG_INCONSISTENCY | Timeout mismatch across environments | Code flagged for fix |

### Remaining Items

| ID | Category | Description | Status |
|----|----------|-------------|--------|
| I3 | SPEC_IMPL_MISMATCH | Error codes differ from spec | Needs investigation |

### Recommendations
[Any patterns noticed, systemic issues, or process improvements]

Exploration Techniques

For Documentation → Code

1. Extract specific claim from documentation
2. Identify keywords (function names, config keys, behavior verbs)
3. Search codebase for those keywords
4. Trace execution path to verify behavior

For Code → Documentation

1. Identify public API or significant behavior
2. Search documentation for references to that behavior
3. Compare documented description to actual implementation
4. Note any discrepancies

For Configuration

1. List all configuration files (appsettings, .env, docker-compose, etc.)
2. Extract same setting from each file
3. Compare values across environments
4. Check documentation claims about defaults

Anti-Patterns

<anti_pattern_stop> If you catch yourself doing any of these, STOP and correct immediately. </anti_pattern_stop>

1. Fixing without asking

WRONG: "I found an inconsistency and updated the documentation."
RIGHT: "I found an inconsistency. Which source is correct?"

2. Assuming documentation is wrong

WRONG: "The code does X, so the docs are outdated."
RIGHT: "Code does X, docs say Y. Which reflects intended behavior?"

3. Shallow exploration

WRONG: "I didn't find retry logic in the main file, so it's not implemented."
RIGHT: "I searched for retry in [these locations] and traced [these paths]."

4. Ignoring context

WRONG: "README says Node 16+ but package.json says 18."
RIGHT: "README says 16+ (minimum), package.json says 18 (recommended). Is this intentional?"

Efficiency

Batch your exploration:

1. Survey all sources first before deep-diving any specific claim
2. Group related claims that can be verified together
3. Present related incoherences together in resolution phase
4. Apply all documentation fixes in one edit when possible

This reduces round-trips and gives you better context for pattern recognition.