| name | canary-test-pipeline |
| description | Multi-phase orchestrator that composes canary-ci-ready, canary-critical-areas, canary-edge-case-discovery, canary-failure-impact, and canary-write-test into a sequential pipeline with a convergence loop and health report.
|
Canary: Test Pipeline
Runs the full test intelligence pipeline from risk assessment to verified CI
readiness. Follows the same pattern as harness:docs-pipeline and
harness:knowledge-pipeline — sequential phases, convergence loop, qualitative
health report on exit.
When to Use
-
Starting a new test suite from scratch
-
Improving a suite that is not yet CI-ready
-
After a major feature lands and test coverage needs updating
-
When asked to "bring this suite to CI-ready"
Phases
Phase 0 — Gate (/canary-ci-ready)
Run canary-ci-ready as a baseline.
-
If CI-READY: inform the user and offer to exit or run an improvement sweep
anyway.
-
If NOT CI-READY: show the score, continue to Phase 1.
Phase 1 — Assess (/canary-critical-areas)
Run canary-critical-areas on the repo (or --diff if provided). Save
critical-areas.json automatically (no --save required when run inside the
pipeline).
Present the top 5 areas and ask the user to confirm or trim the list before
continuing.
Phase 2 — Discover (/canary-edge-cases)
For each confirmed critical area from Phase 1, run canary-edge-case-discovery
with the area as context (passed via critical-areas.json).
Group edge cases by critical area. Present the full list before Phase 3 so the
user can review.
Phase 3 — Impact (/canary-failure-impact)
For each high-risk path (risk_score ≥ 0.7 from Phase 1), run
canary-failure-impact. Use critical-areas.json as context.
Annotate each gap with its severity (Critical / High / Medium / Low).
Phase 4 — Generate (/canary-write-test)
Present the consolidated gap list sorted by: severity (Critical first) then
depth (0 before 1 before 2). Include edge cases from Phase 2 as test variant
suggestions for each gap.
For each gap, ask the user to confirm before invoking canary-write-test.
Do not generate tests silently.
After generation, show a summary of files written before proceeding to Phase 5.
Phase 5 — Verify (/canary-ci-ready)
Re-run canary-ci-ready. Show the delta — which checks improved, which remain
failing.
If any check still fails, investigate using the same user-catalog logic defined
in canary-ci-ready: auth/config failures get a catalog lookup before being
declared blockers.
Convergence
After Phase 5:
-
ci-ready passes → emit health report and exit
-
no new gaps found (Phase 1 produces empty list) → emit health report,
note remaining gaps are outside current signal, suggest manual review
-
gaps remain and user confirms → loop back to Phase 1
-
user stops → emit health report with current state
Health Report
Emit on every exit (convergence or user stop):
Test Pipeline — run complete
Areas assessed: N → N now at depth 3+ (+N this run)
Tests written: N → N critical paths now covered
Gaps remaining: N → <top gap names>
CI-Ready: CI-READY or NOT CI-READY (N/5 checks)
<if not ready> Next: /canary-test-pipeline --continue, or address manually.
```text
## Flags
- `--continue` — skip Phase 0 baseline (resume a prior run)
- `--diff <ref>` — pass to `canary-critical-areas` to scope assessment to a diff
- `--threshold <n>` — pass to `canary-ci-ready` (default: 2)
## Related skills
- `/canary-ci-ready` — Phase 0 and convergence gate
- `/canary-critical-areas` — Phase 1
- `/canary-edge-cases` — Phase 2
- `/canary-failure-impact` — Phase 3
- `/canary-write-test` — Phase 4
