| name | tracey-verify |
| description | Audit spec coverage using tracey MCP tools. Finds uncovered, untested, and stale rules, then semantically verifies that test code actually exercises the spec requirements. Use when the user asks to verify tracey coverage, audit specs, check spec fulfillment, or run a tracey verification.
|
Tracey Verification
Perform a two-phase audit: (1) identify coverage gaps, (2) semantically verify
that tests marked as verified actually match their spec text.
Scope restriction: NEVER read implementation code. Only read test files and the spec (plan/spec.md).
All tracey calls use CallMcpTool.
Phase 1: Coverage Gap Report
Run these MCP calls in parallel:
| Tool | Arguments | Purpose |
|---|
tracey_status | {} | Overall coverage percentages |
tracey_validate | {} | Structural errors (circular deps, unknown refs) |
tracey_uncovered | {} | Rules with no [verify ...] |
tracey_untested | {} | Rules implemented but not tested |
tracey_stale | {} | References pointing at old rule versions |
Present the results as a summary table:
## Coverage Summary
- Total rules: N
- Verified (tested): N
- Uncovered (no verification): N
- Untested (impl but no test): N
- Stale references: N
- Validation errors: N
List each gap with its rule ID. For uncovered rules, include the spec text
(available from tracey_rule).
Phase 2: Semantic Audit
For every rule that IS marked as verified, confirm the test actually exercises
the spec requirement. This catches superficial annotations.
Step 1: Collect verified rules
From the Phase 1 status output, identify all rules that have verification
references. These are the rules NOT listed in the uncovered/untested outputs.
Step 2: Fetch rule details
For each verified rule, call tracey_rule with { "rule_id": "<id>" }.
This returns:
- The spec text
- Verification reference locations (file path + line number)
Batch these calls — issue multiple tracey_rule calls in parallel (up to ~10
at a time).
Step 3: Read and audit test code
For each verification reference location:
- Read the test file at the referenced location. Read enough context to
capture the full test function (typically 50-100 lines from the annotation).
- Compare the test against the spec text. Check:
- Preconditions: Does the test set up the scenario the spec describes?
- Assertions: Does the test assert the behavior the spec requires?
- Substance: Is the test actually exercising the rule, or is the
annotation placed on an unrelated test?
Use parallel subagents (Task tool with subagent_type="explore") to read and
audit multiple test files concurrently when there are many rules. Each subagent
should receive the rule ID, spec text, and file location, and return a verdict.
Step 4: Classify each rule
- PASS: Test clearly exercises the spec requirement.
- WEAK: Test partially covers the requirement or assertions are
insufficient.
- MISMATCH: Test does not match the spec text — annotation appears wrong.
Output Format
Present the final report to the user:
## Tracey Verification Report
### Coverage Summary
| Metric | Count |
|--------|-------|
| Total rules | N |
| Verified | N |
| Uncovered | N |
| Untested | N |
| Stale | N |
### Gaps
#### Uncovered Rules (no verification)
| Rule ID | Spec Text |
|---------|-----------|
| ... | ... |
#### Untested Rules (impl only)
| Rule ID |
|---------|
| ... |
#### Stale References
| Rule ID | Location | Issue |
|---------|----------|-------|
| ... | ... | ... |
### Audit Findings
#### Issues Found
| Rule ID | Verdict | Test Location | Notes |
|---------|---------|---------------|-------|
| ... | WEAK | tests/tests/foo.rs:42 | Missing assertion for X |
| ... | MISMATCH | tests/tests/bar.rs:10 | Test checks Y, spec requires Z |
#### Passed (N rules verified correctly)
<collapsed list or omitted for brevity>
If all verified rules pass, say so and skip the detailed pass list.
Focus the user's attention on gaps and issues.
