| name | regression |
| description | Koru regression test harness workflow. Use when checking test status, investigating failures, or coordinating test runs with the user. |
Regression Test Harness
The regression suite is the hub for all Koru compiler progress. ~600 tests, 40+ minutes for a full run.
Our Workflow
The user runs the full suite (it takes too long for me to run):
./run_regression.sh
./run_regression.sh --parallel 8
I check what broke and investigate:
./run_regression.sh --regressions
./run_regression.sh --status
./run_regression.sh --diff
We debug specific tests together:
./run_regression.sh 501
./run_regression.sh 330
./run_regression.sh --history 501
After fixing, verify:
./run_regression.sh 501 502 503
./run_regression.sh smoke
Key Commands
| Command | Purpose |
|---|
--status | Show current test state from disk markers |
--regressions | List failing tests + when they last passed |
--diff | Compare current run vs last snapshot |
--history <id> | Show test history across all snapshots |
--list | List all tests with descriptions |
--priority | Show tests marked as PRIORITY |
--last-run | Show results from last full run |
Test Markers
Tests are marked with files in their directory:
SUCCESS / FAILURE - Result of last run
TODO - Not yet implemented
SKIP - Intentionally skipped
BROKEN - Known broken, needs fix
BENCHMARK - Performance test, not run normally
PRIORITY - Urgent, needs attention
Snapshots
After each full run, a snapshot is saved to test-results/ with:
- Timestamp and git commit
- Complete test status
- Used for regression detection via
--diff and --history
Important Notes
- Don't run the full suite - it takes 40+ minutes. Use targeted commands.
- User runs full suite, I check regressions - that's our division of labor.
- Tests that MUST_ERROR are negative tests - they're supposed to fail compilation.
- Check
--regressions first when the user reports failures.