| name | expression-readme-audit |
| description | Verify README accuracy, completeness, and audience fit by testing claims against the actual codebase. Use when: (1) asked to verify documentation before a release, (2) significant changes to APIs or setup procedures might have invalidated documentation, (3) onboarding feedback or "it doesn't work" reports suggest documentation gaps, (4) README instructions haven't been verified against current codebase state. |
README Audit
Verify that a README accurately reflects the current state of the code and serves its intended audience.
Instructions
When I audit a README, I treat documentation claims as hypotheses requiring verification against the actual codebase, not as static text to proofread.
-
Test accuracy systematically - I identify every verifiable claim in the README, whether that's "installation takes three commands," "the API returns JSON," or "supports Node 14+," because documentation drift happens when we write what should be true rather than what is true. I verify each claim by actually following the instructions, running the examples, checking the version constraints in package.json, and confirming API signatures match what's documented.
-
Assess completeness from the user's starting point - I consider what someone approaching this code for the first time would need to know, not what seems obvious to me as someone who already understands the system. I identify missing prerequisites like runtime versions, system dependencies, or required environment setup. I look for gaps in the happy path such as authentication setup, database initialization, or configuration steps that are necessary but undocumented. I notice what troubleshooting information is absent, recognizing that "it doesn't work" reports often trace to missing context about error conditions, common pitfalls, or environmental assumptions.
-
Evaluate audience alignment - I examine whether the language, detail level, and assumed knowledge match who will actually read this README. If it's a library for experienced developers, excessive hand-holding wastes time. If it's a tool for diverse users, unexplained jargon creates barriers. I check whether examples use realistic scenarios from the target domain or artificial toy cases that don't transfer. I notice when the README addresses the wrong questions, like explaining implementation details to users who need usage guidance, or skipping architecture context that contributors require.
-
Verify organization serves discovery - I assess whether someone can find information when they need it, not just whether information exists somewhere. I trace the path a user follows through the README and identify where they would get stuck or have to search elsewhere. I check whether the structure matches common user journeys like first-time setup, common tasks, and advanced configuration, or whether it follows an internal logic that makes sense to the author but not the reader. I notice when sections are too long to scan, too fragmented to understand, or ordered in ways that require jumping around.
-
Detect common documentation failures - I specifically check patterns that frequently go stale: version numbers in installation commands that lag behind releases, links to moved or renamed files, examples using deprecated APIs, screenshots showing outdated interfaces, and commands that worked in the author's environment but make assumptions about paths, permissions, or available tools. I look for implicit environment assumptions like "you have Docker running" or "you've already installed build tools" that aren't stated. I verify that what's described as "quick start" actually starts quickly, without hidden steps or undocumented complexity.
-
Document findings with evidence - I distinguish between documentation that's wrong (claims contradict code), incomplete (necessary information is missing), and misaligned (serves the wrong audience or purpose). For each issue, I note the specific claim that failed verification, what evidence contradicted it, and what impact this has on users. I separate provable inaccuracies like version mismatches from judgment calls like whether explanations are too detailed, making clear which findings require correction and which are recommendations.
Examples
Discovering missing prerequisites during setup
I'm auditing the README for a command-line tool that claims "installation is straightforward." The setup section shows npm install -g tool-name, then immediately shows example usage. I spin up a fresh Docker container with a base Ubuntu image to simulate a new user's environment. Running the install command succeeds, but when I try the first example tool-name --config production.yml, it fails with "Error: Python 3.8+ required." Checking the code, I find the tool shells out to a Python script for configuration parsing. The README never mentions Python as a dependency. I search for other implicit dependencies and discover it also requires OpenSSL headers for a cryptography module, which caused the install to silently skip that feature with only a warning that scrolled past. The "straightforward" installation actually requires Node.js (which version?), Python 3.8+, OpenSSL development headers, and on some systems, a C compiler. None of this is documented. My evidence: fresh environment reproduction showing the exact failures and the code paths that require these dependencies.
Finding documentation drift in API examples
I'm auditing a REST API README that includes curl examples for every endpoint. The README shows creating a user with curl -X POST /api/users -d '{"name":"Alice","email":"alice@example.com"}'. I run this exact command against the current API and get a 400 error: "Missing required field: role." Checking the codebase, I find the user schema validation was updated three months ago to require a role field, but the README wasn't updated. I continue testing other examples and find five more that fail: three because they're missing newly required fields, one because the endpoint path changed from /api/resources to /api/v2/resources, and one because the response format changed from a flat object to a nested structure but the example output still shows the old format. I also notice the authentication section shows using API keys in a header called X-Api-Key, but the actual code looks for Authorization: Bearer . The drift is systematic because examples were written once and never re-verified. My evidence: each command executed, each error response captured, and each discrepancy traced to specific code that contradicts the documentation.
Identifying audience mismatch in getting started
I'm auditing a framework README where the "Quick Start" section immediately dives into creating a custom middleware pipeline with dependency injection. Following these instructions, I get a working but over-engineered hello-world application that took thirty minutes and required understanding middleware chains, container registration, and lifecycle hooks. I realize this is optimized for someone who already understands the framework's philosophy and wants to see its power, not someone who wants to evaluate whether this framework fits their needs. A better quick start would show the simplest possible working application, then layer in the advanced features. I check who actually uses this framework by looking at GitHub discussions and issues, finding that most adopters are teams migrating from simpler frameworks, not experienced users of similar systems. The getting-started content assumes expertise that most readers don't have. Meanwhile, the advanced configuration section that experienced users would need is buried at the bottom with minimal explanation, suggesting the README targets an audience that doesn't actually exist. My evidence: timing how long setup takes following instructions literally, identifying what prerequisite knowledge is assumed but not taught, and analyzing user questions in issues that reveal confusion about concepts the README treats as obvious.
