메뉴
Multi-phase orchestrator that composes canary-ci-ready, canary-critical-areas, canary-edge-case-discovery, canary-failure-impact, and canary-write-test into a sequential pipeline with a convergence loop and health report.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Instrument a Playwright run with OpenTelemetry and emit a run.json artifact correlating every test to the outbound HTTP requests it made — "which test made which request?" — with zero manual bookkeeping in test code. Trace-only v1 contract, additive-safe for future pytest/k6/node producers. Self-contained (bundles its own dataclasses and span reader).
Playwright JSON results → Markdown + JSON test report. Summarises passed, failed, flaky, and skipped counts with a per-failure error block and a summary table. Exits non-zero when any test failed so the CI step fails on real failures. Complements canary-fail-fast (which aborts early); this skill summarises the full run at the end.
Surface test failures fast and loud — audit a Playwright config for fail-fast knobs (maxFailures/forbidOnly/retries) and print a loud, categorized failure digest to the CI log + ::error annotations at run end, failing the step so the signal can't be missed. Self-contained (bundles its own Playwright JSON parser and failure categorizer).
Analyses a test suite for CI readiness: coverage depth, flakiness, assertion quality, critical path coverage, and suite runtime. Accepts documented failures (quarantined tests with linked open issues count as verified). Investigates config/auth failures using the consuming repo's declared user_catalog_skill.
Risk-based test prioritisation. Given a codebase or diff, identifies which areas carry the most risk using git churn, downstream dependents, business-critical signals, and existing coverage depth. Produces a ranked list with recommended test types per area.
Given a feature description, function signature, or existing test suite, surfaces edge cases worth testing across six categories. Explanation depth scales to user skill level. Optionally focuses on critical areas when critical-areas.json is present.
| name | canary-test-pipeline |
| description | Multi-phase orchestrator that composes canary-ci-ready, canary-critical-areas, canary-edge-case-discovery, canary-failure-impact, and canary-write-test into a sequential pipeline with a convergence loop and health report. |
Runs the full test intelligence pipeline from risk assessment to verified CI
readiness. Follows the same pattern as harness:docs-pipeline and
harness:knowledge-pipeline — sequential phases, convergence loop, qualitative
health report on exit.
Starting a new test suite from scratch
Improving a suite that is not yet CI-ready
After a major feature lands and test coverage needs updating
When asked to "bring this suite to CI-ready"
/canary-ci-ready)Run canary-ci-ready as a baseline.
If CI-READY: inform the user and offer to exit or run an improvement sweep anyway.
If NOT CI-READY: show the score, continue to Phase 1.
/canary-critical-areas)Run canary-critical-areas on the repo (or --diff if provided). Save
critical-areas.json automatically (no --save required when run inside the
pipeline).
Present the top 5 areas and ask the user to confirm or trim the list before continuing.
/canary-edge-cases)For each confirmed critical area from Phase 1, run canary-edge-case-discovery
with the area as context (passed via critical-areas.json).
Group edge cases by critical area. Present the full list before Phase 3 so the user can review.
/canary-failure-impact)For each high-risk path (risk_score ≥ 0.7 from Phase 1), run
canary-failure-impact. Use critical-areas.json as context.
Annotate each gap with its severity (Critical / High / Medium / Low).
/canary-write-test)Present the consolidated gap list sorted by: severity (Critical first) then depth (0 before 1 before 2). Include edge cases from Phase 2 as test variant suggestions for each gap.
For each gap, ask the user to confirm before invoking canary-write-test.
Do not generate tests silently.
After generation, show a summary of files written before proceeding to Phase 5.
/canary-ci-ready)Re-run canary-ci-ready. Show the delta — which checks improved, which remain
failing.
If any check still fails, investigate using the same user-catalog logic defined
in canary-ci-ready: auth/config failures get a catalog lookup before being
declared blockers.
After Phase 5:
ci-ready passes → emit health report and exit
no new gaps found (Phase 1 produces empty list) → emit health report, note remaining gaps are outside current signal, suggest manual review
gaps remain and user confirms → loop back to Phase 1
user stops → emit health report with current state
Emit on every exit (convergence or user stop):
Test Pipeline — run complete
Areas assessed: N → N now at depth 3+ (+N this run)
Tests written: N → N critical paths now covered
Gaps remaining: N → <top gap names>
CI-Ready: CI-READY or NOT CI-READY (N/5 checks)
<if not ready> Next: /canary-test-pipeline --continue, or address manually.
```text
## Flags
- `--continue` — skip Phase 0 baseline (resume a prior run)
- `--diff <ref>` — pass to `canary-critical-areas` to scope assessment to a diff
- `--threshold <n>` — pass to `canary-ci-ready` (default: 2)
## Related skills
- `/canary-ci-ready` — Phase 0 and convergence gate
- `/canary-critical-areas` — Phase 1
- `/canary-edge-cases` — Phase 2
- `/canary-failure-impact` — Phase 3
- `/canary-write-test` — Phase 4