| name | run-test-all |
| description | Runs ALL Vitest tests across /features, /tests, and /supabase directories. Executes tests via npx vitest run, collects results, and generates test.result.json (replaced on every run). Uses qa-specialist to analyze failures and provide actionable feedback. Triggers on: run test all, run all tests, test all, vitest all, run tests. |
| user-invocable | true |
Run Test All Skill
Run every Vitest test file in the project, collect results, and generate test.result.json.
When to Use
Use /run-test-all when:
- You want to run every test in the project
- You want a test.result.json report of all pass/fail results
- You want to verify the full test suite after changes
Do NOT use when:
- You want to run only a specific phase → use
/run-test-phase
- You want to generate tests (not run them) → use
/generate-test
How It Works
/run-test-all
|
v
+----------------------------------------------------------+
| STEP 1: Discover all test files |
| - tests/**/*.test.ts |
| - supabase/**/*.test.ts |
+----------------------------------------------------------+
|
v
+----------------------------------------------------------+
| STEP 2: Read test-plan.md for context |
| - Map test files to phases/features |
| - Understand expected behavior per test |
+----------------------------------------------------------+
|
v
+----------------------------------------------------------+
| STEP 3: Run vitest |
| - npx vitest run --reporter=json --outputFile=... |
| - Capture all results (pass, fail, skip, duration) |
+----------------------------------------------------------+
|
v
+----------------------------------------------------------+
| STEP 4: Parse results & generate test.result.json |
| - Replaces previous test.result.json entirely |
| - Structured by feature/phase with pass/fail counts |
+----------------------------------------------------------+
|
v
+----------------------------------------------------------+
| STEP 5: Analyze failures (if any) |
| - Spawn qa-specialist to analyze failing tests |
| - Provide actionable fix suggestions |
+----------------------------------------------------------+
|
v
+----------------------------------------------------------+
| STEP 6: Report results |
+----------------------------------------------------------+
Step 1: Discover All Test Files
Find every test file in the project:
Glob: tests/**/*.test.ts
Glob: supabase/**/*.test.ts
If no test files found:
No test files found. Run /generate-test first to generate test files from test-plan.md.
STOP. Do NOT proceed.
Report discovery:
### Test Files Found
| Directory | Files |
|-----------|-------|
| tests/features/ | {count} |
| tests/rls/ | {count} |
| tests/constraints/ | {count} |
| supabase/ | {count} |
| **Total** | **{total}** |
Step 2: Read test-plan.md for Context
Glob: **/test-plan.md OR **/test.plan.md
Read: [found path]
Extract the phase-to-feature mapping so results can be grouped by phase in test.result.json.
If test-plan.md does not exist, warn but proceed:
test-plan.md not found. Results will be grouped by directory only, not by phase.
Step 3: Run Vitest
3.1 Check Supabase local is running
npx supabase status
If Supabase is NOT running, warn:
Warning: Supabase local does not appear to be running. Tests that connect to the database will fail.
Start it with: npx supabase start
Ask the user:
AskUserQuestion:
question: "Supabase local doesn't seem to be running. Proceed anyway?"
header: "Supabase"
options:
- label: "Run tests anyway"
description: "Some tests may fail due to missing database connection"
- label: "Stop — I'll start Supabase first"
description: "I'll run npx supabase start and come back"
3.2 Execute vitest
Run ALL tests with JSON reporter:
npx vitest run --reporter=json --reporter=default --outputFile=docs/vitest-raw-output.json 2>&1
Capture:
- Exit code (0 = all pass, 1 = some failures)
- JSON output for structured parsing
- Console output for human-readable summary
3.3 If vitest is not installed
STOP. Tell user:
"Vitest is not installed. Run: npm install -D vitest"
Step 4: Parse Results & Generate test.result.json
4.1 Read the raw vitest JSON output
Read: docs/vitest-raw-output.json
4.2 Parse and structure results
Build test.result.json with this structure:
{
"generated": "2026-02-10T12:00:00.000Z",
"command": "run-test-all",
"scope": "all",
"duration_ms": 12345,
"summary": {
"total_suites": 10,
"passed_suites": 8,
"failed_suites": 2,
"total_tests": 85,
"passed": 78,
"failed": 5,
"skipped": 2,
"pass_rate": "91.8%"
},
"by_directory": {
"tests/features": {
"suites": 6,
"tests": 50,
"passed": 47,
"failed": 3,
"skipped": 0
},
"tests/rls+constraints": {
"suites": 3,
"tests": 25,
"passed": 23,
"failed": 2,
"skipped": 0
},
"supabase": {
"suites": 1,
"tests": 10,
"passed": 8,
"failed": 0,
"skipped": 2
}
},
"by_phase": {
"Phase 1: Auth": {
"suites": 2,
"tests": 20,
"passed": 18,
"failed": 2,
"features": ["auth"]
},
"Phase 2: Orders": {
"suites": 3,
"tests": 30,
"passed": 28,
"failed": 2,
"features": ["orders", "payments"]
}
},
"suites": [
{
"file": "tests/features/auth/auth.test.ts",
"status": "fail",
"duration_ms": 1234,
"tests": {
"total": 15,
"passed": 13,
"failed": 2,
"skipped": 0
},
"failures": [
{
"name": "Auth CRUD > should create user with valid data",
"error": "Expected null, received { code: '23502', message: '...' }",
"line": 45
},
{
"name": "Auth RLS > should deny other user access",
"error": "Expected 0, received 1",
"line": 78
}
]
},
{
"file": "tests/features/orders/orders.test.ts",
"status": "pass",
"duration_ms": 890,
"tests": {
"total": 20,
"passed": 20,
"failed": 0,
"skipped": 0
},
"failures": []
}
]
}
4.3 Write test.result.json
ALWAYS replace the file entirely — never append.
Write: docs/test.result.json
4.4 Clean up raw output
rm docs/vitest-raw-output.json
Step 5: Analyze Failures (if any)
If ALL tests passed → skip this step.
If there are failures, spawn a qa-specialist agent to analyze:
Task:
subagent_type: qa-specialist
description: "Analyze test failures"
prompt: |
Analyze the following test failures and provide actionable fixes.
Read context:
- docs/test.result.json (test results with failure details)
- references/vitest-best-practices.md (testing rules)
- The failing test files (read each one)
- The source files being tested (read each one)
For each failure:
1. Read the failing test file and the line that failed
2. Read the source code being tested
3. Determine root cause:
- Is the test wrong? (assertion mismatch, wrong expectation)
- Is the source code wrong? (bug in implementation)
- Is the schema wrong? (missing column, wrong constraint)
- Is RLS wrong? (policy too permissive or too restrictive)
- Is test data wrong? (missing seed data, wrong setup)
4. Provide a specific fix recommendation
Return a structured analysis:
FAILURE ANALYSIS:
---
Test: {test name}
File: {file path}:{line}
Root Cause: {test_bug | source_bug | schema_issue | rls_issue | data_issue}
Explanation: {what went wrong}
Fix: {specific action to fix it}
---
[repeat for each failure]
SUMMARY:
- {N} test bugs (fix the test)
- {N} source bugs (fix the implementation)
- {N} schema issues (fix the migration)
- {N} RLS issues (fix the policy)
- {N} data issues (fix seed/setup)
Do NOT modify any files. Analysis only.
Step 6: Report Results
## Test Results — All Tests
**Run at:** {timestamp}
**Duration:** {duration_ms}ms
**Command:** `npx vitest run`
### Summary
| Metric | Count |
|--------|-------|
| Test Suites | {total_suites} |
| Passed Suites | {passed_suites} |
| Failed Suites | {failed_suites} |
| Total Tests | {total_tests} |
| Passed | {passed} |
| Failed | {failed} |
| Skipped | {skipped} |
| **Pass Rate** | **{pass_rate}** |
### Results by Directory
| Directory | Tests | Passed | Failed | Status |
|-----------|-------|--------|--------|--------|
| tests/features/ | {tests} | {passed} | {failed} | {pass/fail} |
| tests/rls/ | {tests} | {passed} | {failed} | {pass/fail} |
| tests/constraints/ | {tests} | {passed} | {failed} | {pass/fail} |
| supabase/ | {tests} | {passed} | {failed} | {pass/fail} |
### Results by Phase
| Phase | Features | Tests | Passed | Failed |
|-------|----------|-------|--------|--------|
| Phase 1: Auth | auth | {tests} | {passed} | {failed} |
| Phase 2: Orders | orders, payments | {tests} | {passed} | {failed} |
### Failures (if any)
| Test | File | Root Cause | Fix |
|------|------|------------|-----|
| {test name} | {file}:{line} | {cause} | {fix recommendation} |
### Output
- **test.result.json:** `docs/test.result.json` (replaced)
### Next Steps
1. Fix failing tests using the failure analysis above
2. Re-run: `/run-test-all` to verify fixes
3. Run specific phase: `/run-test-phase` to focus on one area
4. Generate more tests: `/generate-test {feature}`
Execution Rules
DO:
- ALWAYS discover test files before running
- ALWAYS run vitest with JSON reporter for structured output
- ALWAYS replace test.result.json entirely (never append)
- ALWAYS check if Supabase local is running before executing
- ALWAYS group results by directory AND by phase (if test-plan.md exists)
- ALWAYS spawn qa-specialist for failure analysis when tests fail
- ALWAYS clean up raw vitest output after parsing
- ALWAYS report pass rate prominently
DO NOT:
- NEVER append to test.result.json — always replace
- NEVER skip running tests and generate fake results
- NEVER modify test files during a test run
- NEVER modify source code during a test run
- NEVER ignore test failures — always analyze them
- NEVER run tests without checking for test files first
Checklist
