| name | test-runner |
| description | Execute the appropriate test suite (unit, Gherkin, e2e, smoke) and return structured results. Use during Phase 3 (red baseline verification), Phase 4 (implementation inner/middle/outer loops), Phase 5 (smoke tests against deployment), and on resume (re-validate test state). Trigger when running tests, checking test status, or verifying test baselines. |
Test Runner
Execute tests and return structured results for the orchestrator.
Test Commands
| Type | Command |
|---|
| Unit (.NET) | cd src/api && dotnet test |
| Gherkin | npx cucumber-js |
| E2E | npx playwright test --config=e2e/playwright.config.ts |
| Smoke | npx playwright test --grep @smoke |
| All | npm run test:all |
Steps
- Determine test type — Select the test suite based on current phase and task
- Run tests — Execute the command, capture stdout and stderr
- Parse results — Extract pass/fail counts, failure details, and test names
- Detect flaky tests — If a test failed, re-run it once; if it passes on retry, flag as flaky
- Structure output — Format results for the orchestrator
Output Format
Type: unit | gherkin | e2e | smoke | all
Pass: <count>
Fail: <count>
Flaky: <count>
Verdict: GREEN | RED | FLAKY
Failed tests:
- <test name>: <error message>
Edge Cases
- Test runner itself fails (not assertions) → report as infrastructure failure
- Tests exceed 5 minutes → check for hung processes
- Always capture both stdout and stderr
