Diagnostic-only test suite auditor. Classifies repo type, maps against 7-layer testing taxonomy (git hooks → static → unit → integration → system → E2E → acceptance), runs deterministic quality gates (coverage, mutation, CRAP, architecture, escape-scan), builds RTM / personas / journeys traceability, produces TEST_AUDIT.md, updates tests/TESTING.md, and mandatorily hands off to implement-tests when gaps are found. Use when auditing test quality, finding test gaps, or running the full 7-layer sweep. Trigger with "audit tests", "find gaps", "test audit", "check test quality", "full sweep", "7-layer audit", "rtm check".
Diagnostic-only test suite auditor. Classifies repo type, maps against 7-layer testing taxonomy (git hooks → static → unit → integration → system → E2E → acceptance), runs deterministic quality gates (coverage, mutation, CRAP, architecture, escape-scan), builds RTM / personas / journeys traceability, produces TEST_AUDIT.md, updates tests/TESTING.md, and mandatorily hands off to implement-tests when gaps are found. Use when auditing test quality, finding test gaps, or running the full 7-layer sweep. Trigger with "audit tests", "find gaps", "test audit", "check test quality", "full sweep", "7-layer audit", "rtm check".
Enforcement harness: this skill delegates all deterministic checks (hash-pin, escape-scan, CRAP, architecture, bias, Gherkin lint) to the in-repo installation of @intentsolutions/audit-harness. Node repos install via pnpm add -D @intentsolutions/audit-harness; other languages vendor via the package's install.sh. Never reference ~/.claude/... paths from a target repo's hooks or CI — the enforcement must travel with the code.
Read-only diagnostic skill. Classifies the repo, maps applicable layers from the 7-layer testing taxonomy, runs deterministic gates, builds requirements traceability (RTM / personas / journeys), writes TEST_AUDIT.md + updates tests/TESTING.md, and mandatorily hands off to implement-tests when P0/P1 gaps exist.
No filesystem mutations other than TEST_AUDIT.md (transient) and the observational sections of tests/TESTING.md, tests/RTM.md, tests/PERSONAS.md, tests/JOURNEYS.md.
Overview
Diagnostic-only test suite auditor for any Intent Solutions (or compatible) repo.
Classifies the repository, maps the 7-layer testing taxonomy, runs deterministic
gates via in-repo @intentsolutions/audit-harness, builds RTM/personas/journeys
traceability, and writes TEST_AUDIT.md plus observational sections of
tests/TESTING.md. When P0/P1 gaps exist it hands off to implement-tests
instead of implementing tests itself.
Prerequisites
A git working tree for the target repository
Ability to run Read/Glob/Grep/Bash against that tree
For deterministic gates: in-repo audit-harness (or install path documented
for the language) — never wire target CI/hooks to ~/.claude/...
You are the engineer
Deterministic tools do the grading. The engineer owns the walls (features/*.feature, coverage thresholds, architecture rules, policy sections, MoSCoW tags). The AI operates inside them and never grades itself — every gate is a tool that returns exit 0.
Drift detected → surfaced in TEST_AUDIT.md under ## Freshness
Not blocking — audit continues regardless
Step 1 — Read context
Check for tests/TESTING.md. If absent → proceed to discovery. If present → read ## Classification, ## Thresholds, ## Waived layers. Engineer policy wins.
Verify the target repo has @intentsolutions/audit-harness installed (Node: package.json devDep; other langs: .audit-harness/ directory exists). If absent, add to the handoff payload as L0 install. Hooks and CI reference the in-repo harness, never ~/.claude/ paths.
Verify hash manifest via pnpm exec audit-harness verify (Node) or scripts/audit-harness verify (vendored). Interpret exit code:
0 (OK) — pinned hashes match; continue.
2 (HARNESS_TAMPERED) — pinned artifact changed; halt the pipeline; flag in TEST_AUDIT.md and do not proceed.
3 (no manifest) — fresh repo or never initialized; continue. Log "no hash manifest — fresh repo" in the report. This is NOT a halt condition.
Check current branch. On main / develop / protected branches, any handoff at Step 8 will prompt via AskUserQuestion. On feat/* branches, handoff runs autonomously.
Step 2 — Classify
Deterministic floor first (the harness brain). Run pnpm exec audit-harness classify (Node) / scripts/audit-harness classify (vendored) / audit-harness classify to emit the read-only audit-profile/v1 value — the UNION of detected classifications (service / api / library / cli / frontend / embedded / monorepo + skill / agent / hook / mcp / plugin / marketplace / action), the resolved gate set (a projection of registry.v1.json, recorded by registry_hash), and an explicit unresolved[].
Then dispatch test-discovery-agent (via Task tool) only to refine the unresolved[] residue (ambiguous/novel repos) — it proposes classification patches a human pins in .audit-harness.yml/TESTING.md; it never overrides the deterministic classification value (keeps CI reproducible). Matrix + signals: {baseDir}/references/layer-applicability.md (itself generated from the registry datum via audit-harness gen-layer-applicability).
Monorepo: the profile carries per-package-classify; classify each package independently and run the full pipeline per-package.
Step 3 — Map applicable layers
The applicable gates are already resolved in the audit-profile/v1 from Step 2 (dimension + applicability + enforcement per gate). Dispatch taxonomy-mapper-agent to map presence/absence/configured/enforced on top of the profile's gate set + the TESTING.md compliance overlay.
Output: gap list tagged P0/P1/P2 per layer-applicability.md severity symbols.
Step 4 — Run deterministic quality gates
Run the harness gate-runners — each is read-only and emits gate-result/v1 Evidence Bundle rows the quality-gate-runner-agent consumes:
crap-score + per-layer presence (does NOT execute the repo's suite)
audit-harness scan
security / hygiene / skill-quality
shells out to gitleaks/osv/semgrep/syft/markdownlint/lychee; consumes j-rig verdict
audit-harness currency
upstream currency
advisory report only (no exit authority)
New gates ship enforcement: advisory; blocking is engineer-pinned in TESTING.md, FP-rate-gated (audit-harness fp-rate, ≤ 5% bar — see docs/gate-promotion.md). Then dispatch quality-gate-runner-agent for the L3 measurement tools and capture exit-code + report:
Gate
Command
Coverage
pytest --cov --cov-fail-under=$(TESTING.md coverage.line) or framework equivalent
Mutation
mutmut run / npx stryker run / cargo mutants run
CRAP & complexity
python3 {baseDir}/scripts/crap-score.py --target both --out reports/crap/
Architecture
bash {baseDir}/scripts/arch-check.sh
Bias count
bash {baseDir}/scripts/bias-count.sh
Gherkin lint
bash {baseDir}/scripts/gherkin-lint.sh
Gate failure on prod CRAP > 30, test CRAP > 15, project average > 10, architecture violation > 0, or coverage below policy floor → halt the pipeline; record in TEST_AUDIT.md.
Step 5 — Meta-audit (RTM / personas / journeys)
Dispatch three specialists in parallel (Task tool):
rtm-builder-agent — rebuild tests/RTM.md from Gherkin scenarios, ADRs, product docs, engineer-declared REQs, commit tags. Apply MoSCoW precedence (explicit tag → source default → engineer override → SHOULD fallback). Preserve engineer overrides. See {baseDir}/references/rtm-personas-journeys.md.
persona-coverage-agent — rebuild tests/PERSONAS.md; compute per-persona flow coverage; flag personas below threshold.
journey-mapper-agent — rebuild tests/JOURNEYS.md; per-step layer mapping; flag untested steps.
Classify findings:
Finding
Severity
Uncovered MUST requirement
P0 (blocks audit — triggers handoff)
Uncovered SHOULD requirement
P1 (advisory; regulated overlay escalates to P0)
Uncovered COULD requirement
P2 (logged only)
WON'T requirement
excluded from coverage math
Orphaned test (no linked REQ)
P1 advisory (regulated: P1; normal: P1)
Persona below flow-coverage threshold
P1 advisory
Journey step untested (step linked to a MUST REQ)
P0; (linked to SHOULD) P1; (COULD) P2
Step 6 — Escape-scan on any pending diff
Dispatch escape-detection-agent:
bash {baseDir}/scripts/escape-scan.sh --staged # reads policy floors from tests/TESTING.md
Exit grammar: 0 clean · 1 CHALLENGE · 2 REFUSE. Any REFUSE halts the audit with the specific pattern recorded in TEST_AUDIT.md.
Step 7 — Write reports
TEST_AUDIT.md at repo root (transient; short-lived):
Run escape-scan.sh + harness-hash.sh --verify; report REFUSE / CHALLENGE / FLAG
rtm-builder-agent
Build/update RTM.md; apply MoSCoW precedence; flag orphans + uncovered
persona-coverage-agent
Compute per-persona flow coverage in PERSONAS.md
journey-mapper-agent
Map journey steps to layer-tests in JOURNEYS.md
Phase 2 (deferred): each agent may be upgraded to a full skill bundle with its own references/ scripts/ evals/ — see the implementation plan's "Phase 2 Roadmap".
Discovery and pre-flight
Full discovery logic (config-file scan, language detection, package-manager detection, monorepo detection): {baseDir}/references/discovery-preflight.md.
Quality gate deep-dives
Catalog of every gate tool across 10 sweep categories (aligned to 7-layer taxonomy): {baseDir}/references/quality-gates-sweep.md.
AUDIT COMPLETE — {repo-name}
Grade: C (68/100)
P0 gaps: 2
P1 gaps: 5
Branch: main (protected) — prompting for handoff confirmation...
Examples
Example 1 — Fresh Python library, no tests yet.
Run audit tests → test-discovery-agent classifies as library, no TESTING.md present → taxonomy-mapper-agent reports L1, L2, L3 all absent (P0), L4–L7 waived per library applicability → rtm-scaffolder schedules skeleton creation → handoff payload: install_order: [L1, L2, L3] → autonomous handoff fires (feat branch) → implement-tests scaffolds pytest + coverage + mutmut + ruff + pre-commit + starter tests.
Example 2 — Mature service, previously audited.
Run full sweep. All layers report installed; coverage 82% (above 80 floor); mutation kill rate 74% (above 70 floor); CRAP max 22 (below 30). Grade: A (92/100). TESTING.md#Last audit updated with today's date. No handoff.
Example 3 — Monorepo with three packages.
Run audit tests. test-discovery-agent returns monorepo: true, packages: [a, b, c]. Each package classified independently (packages/a = service, packages/b = library, packages/c = cli). Each gets a per-package TESTING.md. Aggregate TEST_AUDIT.md at repo root has three sections. Handoff payload includes per-package install_order arrays.
Example 4 — PR lowers coverage threshold to bypass gate.
A PR proposes [tool.coverage.report] fail_under = 60 in pyproject.toml. TESTING.md#Thresholds has coverage.line: 80. escape-detection-agent runs escape-scan.sh --staged. The script reads floor 80 from TESTING.md, sees fail_under dropped to 60, emits [REFUSE] coverage fail_under lowered below policy floor (80), exits 2. Audit halts. No TEST_AUDIT.md produced. No handoff. Engineer must revert the threshold edit or run harness-hash.sh --init to legitimize a policy change.
Example 5 — HIPAA-compliant service with uncovered SHOULD.
Repo has Compliance overlay: HIPAA in TESTING.md. RTM.md has one uncovered SHOULD (error-message retry logic). Normally P1 advisory, but the regulated overlay promotes it to P0. Handoff fires with p0_gaps: [{layer: L3, gap: REQ-034 retry logic uncovered (SHOULD, HIPAA overlay)}]. On main branch → AskUserQuestion prompts; engineer approves; implement-tests proceeds.
Troubleshooting
Symptom
Diagnosis
Fix
rg: error parsing flag -E: grep config error: unknown encoding
Shell has grep aliased to rg (ripgrep); flag grammar differs.
Use /usr/bin/grep explicitly, or command grep, or call the native Grep tool. All detection examples in this SKILL assume GNU grep.
error: unexpected argument '-m' found (from find)
Shell has find aliased to fd; flag grammar differs.
Use /usr/bin/find explicitly, or command find, or call the native Glob tool. All detection examples assume GNU find.
gh label create appears to succeed but label already existed
gh label create is a silent no-op without --force.
Use gh label create <name> --description "..." --color <hex> --force when creating labels idempotently in automation.
HARNESS_TAMPERED at Step 1
A hash-pinned file was modified since last --init.
Review the diff; if legitimate, run pnpm exec audit-harness init (Node) / scripts/audit-harness init (vendored) to re-pin; if unexpected, investigate the diff before re-init.
escape-scan REFUSES on a legitimate threshold change
Thresholds are policy — lowering triggers REFUSE.
Engineer edits tests/TESTING.md#Thresholds, then runs harness-hash.sh --init. AI cannot do this step.
test-discovery-agent returns repo_type: unknown
No recognizable manifest (package.json, pyproject.toml, etc.).
Add the correct manifest for your language; or set Repo type: manually in tests/TESTING.md#Classification.
Handoff does not fire on feat branch
Branch name does not match feat/* pattern, or no P0/P1 gaps found.
Check branch name; if gaps exist and handoff still does not fire, check that Skill tool is in the allowed-tools frontmatter.
Decline is logged; TEST_AUDIT.md still produced. To force handoff, merge to a feat branch first.
rtm-builder-agent keeps re-promoting a MUST
Engineer override was not preserved because RTM.md was rewritten (Write instead of Edit).
Ensure the agent uses Edit tool on RTM.md; engineer override is detected by row-level MoSCoW value diff.
Missing coverage.py / mutmut / other tool
Measurement tool not installed.
Handoff to implement-tests with the specific layer; it installs the measurement tool per the L3 playbook.
Monorepo produces only root-level audit
pnpm-workspace.yaml / turbo.json / nx.json not present or unrecognized.
Confirm workspace config is at repo root; or manually declare per-package TESTING.md files.
Edge cases
Missing framework: record gap; skip the measurement; emit install_order: [framework] in handoff.
Docker unavailable for integration tests: skip L4 integration measurement; still report the gap as "unmeasured — Docker missing".
Hash mismatch: HARNESS_TAMPERED — halt before any other work; do not write TEST_AUDIT.md.
Engineer-declared waiver: any layer listed in TESTING.md#Waived layers is skipped silently; appears in report under "Waived (per engineer policy)".
Regulated overlay: HIPAA/SOX/PCI/SOC2/GDPR/FedRAMP tag in TESTING.md promotes SHOULD uncovered to P0 — see rtm-personas-journeys.md § "Regulated overlay".
