| name | qa-spec-auditor |
| description | Living documentation auditor that compares requirements vs UI/behavior vs API contract vs test results to detect spec drift, outdated docs, and undocumented features. |
| output_dir | reports/audit |
| dependencies | {"recommended":["qa-requirements-generator","qa-spec-writer"]} |
QA Spec Auditor
Purpose
Compare "how it should be" vs "how it is" after each release or sprint. Detect spec drift, outdated documentation, undocumented features, and regression set optimization opportunities. Produce actionable drift reports with task suggestions for qa-task-creator.
Trigger Phrases
- "Spec audit" / "Documentation audit for [release/sprint]"
- "Spec drift" / "Detect drift between docs and implementation"
- "Outdated documentation" / "Docs vs implementation comparison"
- "Undocumented features" / "Find features not in specs"
- "Requirements vs UI" / "Requirements vs API contract"
- "Living documentation audit" / "Spec compliance check"
- "Regression set optimization" / "Worn out tests" / "Test rotation"
Comparison Dimensions
| Dimension | Source A | Source B | Detects |
|---|
| Requirements ↔ UI/behavior | Requirements, specs | Live UI (Playwright MCP) | UI doesn't match spec; missing flows |
| Requirements ↔ API contract | Requirements, specs | OpenAPI, actual endpoints | API drift; undocumented endpoints |
| Requirements ↔ Test results | Requirements | Test cases, RTM, execution | Untested requirements; orphan tests |
| Documentation ↔ Implementation | Docs, README, wikis | Code, UI, API | Outdated docs; undocumented behavior |
See references/audit-checklist.md for checklists per dimension.
Detection Categories
| Category | Description | Output Action |
|---|
| Outdated documentation | Docs describe old behavior | Update docs |
| Undocumented features | Implementation has features not in specs | Add to specs or create tests |
| Spec drift | Behavior changed but docs not updated | Update docs or fix implementation |
| Regression set optimization | "Worn out" tests (always pass, low value); suggest rotation | Suggest test rotation; prioritize high-value tests |
See references/drift-patterns.md for common patterns and remediation.
Output Deliverables
- Drift Report — Per dimension: findings, severity, affected artifacts
- Action Items — Update docs / fix implementation / create tests
- Task Suggestions — Structured for qa-task-creator (doc updates, test creation, spec fixes)
- Regression Set Recommendations — Tests to rotate, prioritize, or deprioritize
Drift Report Template
# Spec Audit — [Release/Sprint]
## Summary
| Dimension | Findings | Critical | Action Items |
|-----------|----------|----------|--------------|
| Req ↔ UI | N | X | [Count] |
| Req ↔ API | N | X | [Count] |
| Req ↔ Tests | N | X | [Count] |
| Docs ↔ Impl | N | X | [Count] |
## Findings by Dimension
### Requirements ↔ UI
- [Finding] — [Severity] — [Action]
### Requirements ↔ API
- [Finding] — [Severity] — [Action]
### Requirements ↔ Tests
- [Finding] — [Severity] — [Action]
### Documentation ↔ Implementation
- [Finding] — [Severity] — [Action]
## Task Suggestions (for qa-task-creator)
1. [Task type] — [Description] — [Owner]
2. ...
## Regression Set Recommendations
- Rotate: [Tests that are "worn out"]
- Prioritize: [High-value tests]
- Deprioritize: [Low-value, stable tests]
Workflow
- Input: Requirements, specs, OpenAPI/API docs, test cases/RTM, optional UI URL
- Requirements ↔ UI: Use Playwright MCP to explore UI; compare flows, elements, validation to spec
- Requirements ↔ API: Compare OpenAPI/spec to requirements; detect undocumented endpoints, schema drift
- Requirements ↔ Tests: Validate RTM; find untested requirements, orphan tests
- Documentation ↔ Implementation: Compare docs to code/UI/API; flag outdated sections
- Regression optimization: Analyze test history; identify always-pass, low-value tests; suggest rotation
- Output: Drift report, action items, task suggestions for qa-task-creator
MCP Tools Used
- Playwright MCP (cursor-ide-browser): UI exploration, compare live behavior to spec
- GitHub MCP: Change tracking, PR/commit history for docs and code
Integration with Other Skills
| Need | Skill | Usage |
|---|
| Task creation | qa-task-creator | Convert action items to structured tasks |
| Requirements source | qa-requirements-generator | Normalized requirements for comparison |
| API contract | qa-api-contract-curator | OpenAPI for API dimension |
| Test cases/RTM | qa-testcase-from-docs, qa-coverage-analyzer | Test mapping, coverage |
| Spec writing | qa-spec-writer | Update specs when drift found |
Scope
Can do (autonomous):
- Compare requirements vs UI (via Playwright MCP)
- Compare requirements vs API contract (OpenAPI)
- Compare requirements vs test results (RTM, coverage)
- Compare documentation vs implementation
- Detect outdated docs, undocumented features, spec drift
- Produce drift report with action items
- Suggest tasks for qa-task-creator
- Recommend regression set optimization (test rotation)
- Use GitHub MCP for change tracking
Cannot do (requires confirmation):
- Modify documentation or code directly
- Change requirements without stakeholder approval
- Override organizational doc/spec policies
Will not do (out of scope):
- Execute tests (consume results only)
- Write or modify test automation
- Deploy or change production systems
Quality Checklist
Troubleshooting
| Symptom | Likely Cause | Fix |
|---|
| UI comparison fails | Playwright MCP not available; auth required | Use API/docs comparison only; note "UI audit skipped" |
| API contract missing | No OpenAPI; endpoints undocumented | Use qa-api-contract-curator first; manual endpoint list |
| RTM incomplete | Requirements lack IDs; tests not traced | Use qa-requirements-generator; qa-testcase-from-docs for traceability |
| Too many findings | Broad scope; noisy comparison | Focus on critical modules; prioritize by business impact |
| Regression data missing | No test history | Use execution logs; flag "insufficient data" |
| GitHub MCP errors | Token, permissions | Verify GitHub token; check repo access |
Reference Files
| Topic | Reference |
|---|
| Audit checklist per dimension | references/audit-checklist.md |
| Drift patterns and remediation | references/drift-patterns.md |
