一键导入
stress-hunt
Automated bug-hunting loop using vole-stress + vole-reduce. Generates random codebases, tests them, reduces failures, fixes bugs, verifies, and repeats.
用 Codex 或 Claude 帮你安装 复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它检查 Skill 页面并帮你完成安装。
菜单
Automated bug-hunting loop using vole-stress + vole-reduce. Generates random codebases, tests them, reduces failures, fixes bugs, verifies, and repeats.
用 Codex 或 Claude 帮你安装 复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Automated performance optimization loop. Generates large benchmark workloads with vole-stress, profiles with perf, identifies hotspots in our code, fixes them, verifies improvement, repeats.
Use when adding new diagnostic error codes to the compiler
Use when you are asked to implement tickets (tasks/epics) created in `tk`
Incremental Rust code quality loop. Scans the compiler codebase for one refactoring opportunity per round — duplicated logic, poor factoring, mechanical lint fixes — applies it, verifies, and repeats.
Use when you have a design or requirements and need to create tickets (epics/tasks) for implementation
You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.
| name | stress-hunt |
| description | Automated bug-hunting loop using vole-stress + vole-reduce. Generates random codebases, tests them, reduces failures, fixes bugs, verifies, and repeats. |
Automated bug-hunting skill that ties together vole-stress (synthetic codebase
generation) and vole-reduce (test case minimization) in an iterative loop.
Launch via ralph-loop with a completion promise:
/ralph-loop "/stress-hunt 5 20" --completion-promise STRESS_HUNT_DONE
/ralph-loop "/stress-hunt 10 10 full,many-modules" --completion-promise STRESS_HUNT_DONE
Or invoke directly (single iteration):
/stress-hunt 5 20 # 5 seeds per round, 20 max rounds
/stress-hunt 10 10 full,many-modules # 10 seeds, 10 rounds, specific profiles
Parse $ARGUMENTS:
Default profiles if none specified:
full, many-modules, deep-nesting, wide-types, generics-heavy.
If new profiles are added during Generator Evolution, they are appended to
the profiles list in the state file and used in subsequent rounds.
Each iteration processes one round of K seeds through the workflow. When all K seeds in a round pass or are verified, the round is completed: passing seed directories are cleaned up, and K fresh seeds are generated for the next round.
The skill tracks iterations internally via round in the state file. When
round > Z (max rounds), output <promise>STRESS_HUNT_DONE</promise> to
signal the ralph loop to stop.
Iteration cost: a single bug takes at minimum 5 ralph-loop iterations to fully process (generate → triage → reduce → fix → verify). Plan accordingly when setting K and Z.
.claude/stress-hunt-state.json{
"k": 10,
"max_rounds": 20,
"profiles": ["full", "many-modules", "deep-nesting", "wide-types", "generics-heavy"],
"known_bugs": [
{
"fingerprint": "panicked at frontend.rs:509",
"ticket_id": "vol-xxxx",
"error_category": "codegen",
"description": "Cranelift frontend panic on complex types"
}
],
"seeds": [
{
"seed": 12345,
"profile": "full",
"dir": "/tmp/vole-stress/clever-badger",
"status": "pending",
"error_category": null,
"error_summary": null,
"error_fingerprint": null,
"reduced_dir": null,
"fix_commit": null,
"ticket_id": null,
"dupe_of": null
}
],
"round": 1,
"consecutive_clean": 0,
"base_k": 10,
"history": [
{ "round": 1, "passed": 3, "failed": 2, "bugs_fixed": 2, "skipped": 0, "dupes": 0 }
]
}
Seed statuses: pending -> pass | fail -> reducing -> reduced -> fixing -> verified
-> skipped
-> dupe (matched known bug)
.claude/stress-hunt-journal.mdA persistent log of facts learned across runs — reserved words, syntax gotchas, generator pitfalls, process lessons. This file is gitignored and survives across sessions.
- **Read the journal** at the start of every stress-hunt session (step 1) - **Append to the journal** whenever you discover something that would save a future run from wasting time — wrong syntax assumptions, reserved words, codegen quirks, process mistakes. Keep entries terse (one line each).Read .claude/stress-hunt-state.json. Based on seed statuses, perform the
first applicable action from the list below, then update state and finish.
.claude/stress-hunt-journal.md to load lessons from previous runs.claude/stress-hunt-state.json exists from a previous session (i.e. we
did NOT create it this session), delete it and start freshjust check to verify the repo compiles. If it
fails, stop and report — do not generate seeds against a broken repo.shuf -i 0-4294967295 -n K for full u32 range)pending, known_bugs as []round to 1, max_rounds to Z, history to []pending seeds exist)Step-by-step:
pending seeds from the state file into seed:profile pairsbash .claude/skills/stress-hunt/run-round.sh [flags] /tmp/vole-stress \
<seed1>:<profile1> <seed2>:<profile2> ...
For example, if the state has seeds 12345 (profile=full), 67890 (profile=many-modules), and 11111 (profile=deep-nesting):
bash .claude/skills/stress-hunt/run-round.sh /tmp/vole-stress \
12345:full 67890:many-modules 11111:deep-nesting
Flags:
--release — Build and run with the release-local cargo profile
(optimized with debug symbols). Use when recent rounds have had no failures
— it iterates much faster. Omit for debug builds (better error messages,
faster compilation).--diff — Differential testing: builds both debug and release-local, runs
each seed under both, and reports mismatches (different exit codes or stdout
output). Catches optimization-related bugs. Output includes a diff_mismatch
field. Use this periodically (e.g. every 3-5 rounds) to catch bugs that only
manifest under optimization.--asan — Build vole with AddressSanitizer (nightly toolchain, into
target-asan/). Catches heap corruption, double-free, and use-after-free
in the compiler runtime. Mutually exclusive with --release and --diff.
Use every 5-10 rounds with moderate K (10-20) — ASan adds ~2-3x overhead.
ASan errors appear in error_summary with AddressSanitizer: prefixed
messages and should be categorized as codegen errors during triage.Scaling K: When using --release and recent rounds have zero failures,
increase K to 50-100 seeds per round. The expanded name pool (94k unique
names) supports this without collisions, and release-local builds execute
significantly faster than debug.
Script output format: The script builds once, then for each seed:profile pair it generates, tests (60s timeout), and runs main.vole (30s timeout). It outputs JSONL to stdout — one JSON object per line, one per seed:
{"seed":12345,"profile":"full","dir_name":"soaring-swift-penguin","dir":"/tmp/vole-stress/soaring-swift-penguin","test_status":"pass","run_status":"pass","error_summary":""}
In --diff mode, an additional diff_mismatch field is included (empty string
if both builds agree, otherwise describes the difference).
Status values: pass, fail, timeout, skip. Progress goes to stderr.
Parsing the JSONL output: For each line, update the corresponding seed in the state file:
test_status and run_status both pass → mark seed passfail or timeout → mark seed fail, record error_summarydiff_mismatch non-empty → mark seed fail even if both builds "pass"
(the mismatch itself is the bug)dir and dir_name fields from the JSONL outputProcess all pending seeds in one iteration before moving on.
fail seeds without error_category)For each fail seed:
Extract a fingerprint from the error: the panic location (e.g.
panicked at frontend.rs:509), the error code + message (e.g.
[E2023]: unknown method 'foo'), the signal (e.g. signal 11), or the
ASan error type and location (e.g.
AddressSanitizer: heap-use-after-free on address 0x... at pc 0x...).
Store in error_fingerprint.
Compare the fingerprint against known_bugs in the state file. If it
matches an existing known bug:
dupe_of to the matching ticket IDdupetk add-note <id> "Also hit by seed <S> (round N)"If multiple seeds in the same round hit the same new error (same
fingerprint, no matching known bug), pick ONE as the primary and mark
the rest as dupe of that seed's (future) ticket. Only reduce/fix the
primary.
Generator error (vole-stress bug):
[E0xxx] lexer errors, [E1xxx] parser errors[E2xxx] sema errors where generated code is clearly structurally wrong
(impossible types, undefined names that should have been declared)filter().collect())error_category: "generator"src/tools/vole-stress/ before proceeding to the next round.Sema / VIR lowering error (type checker or VIR bug):
[E2xxx] errors where generated code looks valid but type checker rejects itcargo run -- inspect vir <file> shows
incorrect VIR output (wrong types, missing coercions, missing RC ops)error_category: "sema"Codegen error (code generation / runtime bug):
error_category: "codegen"fail seeds with error_category set)For each non-dupe fail seed, run vole-reduce with oracle flags based on
error type:
| Error Pattern | Oracle Flags |
|---|---|
| Parse/lex error | --stderr "E0|E1" --exit-code 1 |
| Sema error | --stderr "E2" --exit-code 1 |
| Segfault | --signal 11 |
| Timeout | --timeout 60 |
| Assertion failure | --stderr "assertion failed" --exit-code 1 |
| ASan error | --stderr "AddressSanitizer" --exit-code 1 |
| Generic crash | --exit-code 1 --stderr "<specific error text>" |
Always use --command with --manifest-path since vole-reduce runs from the
result/ directory:
cargo run -p vole-reduce -- <dir>/ \
--command 'timeout 90 cargo run --manifest-path /home/phil/code/personal/vole/Cargo.toml -- test {file}' \
--timeout 60 \
<oracle-flags>
For ASan-detected bugs, use the ASan binary in the oracle command instead:
cargo run -p vole-reduce -- <dir>/ \
--command 'timeout 90 env ASAN_OPTIONS=detect_leaks=0 /home/phil/code/personal/vole/target-asan/x86_64-unknown-linux-gnu/debug/vole test {file}' \
--timeout 60 \
--stderr "AddressSanitizer" --exit-code 1
After reduction completes:
cp -r <reduced_dir> .claude/stress-hunt-repros/<seed-name>/
This is the canonical repro — survives /tmp cleanup and generator
evolution. The reducer may leave multiple files if the bug requires
cross-module interaction.reduced_dir and mark reducing -> reduced.reduced seeds exist)For each reduced seed, actually attempt to fix the bug. The goal is to
fix it, not just file it. The ticket exists to track your investigation data
— a running log of what you tried, what you found, and what's left.
Fix ONE seed per iteration to keep changes focused and reviewable.
error_category: "generator")Generator bugs are straightforward vole-stress fixes and do NOT get tickets.
Read the reduced test case, fix the generator code in src/tools/vole-stress/,
run just pre-commit, commit, and mark fixing.
For sema and codegen bugs, dispatch a sequential sub-agent (using the Task tool) to investigate and attempt the fix.
Before dispatching the sub-agent:
reduced_dirtk ticket for the bug (see Ticket Tracking below)ticket_idknown_bugs in the state file with its fingerprint
and ticket ID, so future duplicates are detectedThe sub-agent's task:
Give the sub-agent a detailed prompt including:
.claude/stress-hunt-repros/<name>/)sema: src/crates/vole-sema/codegen: src/crates/vole-codegen/ or src/crates/vole-runtime/The sub-agent MUST:
cargo run -- inspect vir <file>
to understand what sema/VIR lowering produces. If VIR looks wrong, fix
in sema or VIR lowering. If VIR looks correct, fix in codegen.tk add-note <id> "..." as it goestest/unit/ covering the patternjust pre-commit to verifytk close <id>After the sub-agent completes:
fix_commit hash, mark fixingskipped, move on
fixing seeds exist)For each fixing seed:
Generator fixes: must regenerate the seed (generated code changes):
cargo run -p vole-stress -- --seed <S> --profile <P> --output /tmp/vole-stress --name <same-name>
Then re-test.
Sema/codegen fixes: re-test the same files (no regeneration needed):
timeout 60 cargo run -- test <dir>/
If passes: mark verified.
If new failure: mark fail again with new error, clear error_category.
pass, verified, skipped, or dupe)A round is NOT complete if any generator bugs were observed but not fixed.
Generator bugs must be fixed before moving to the next round. Only sema/codegen
bugs that hit the 15-minute time limit may be skipped.
When all K seeds in the current round are terminal (pass, verified,
skipped, or dupe):
history:
{ "round": N, "passed": X, "failed": Y, "bugs_fixed": Z, "skipped": S, "dupes": D }
rm -rf the /tmp/vole-stress/ directories for all terminal
seeds in this roundRound N complete: X/K passed, Y bugs fixed, S skipped, D dupesround >= max_rounds: output <promise>STRESS_HUNT_DONE</promise> and stopconsecutive_clean to 0consecutive_cleanconsecutive_clean == 1: double K for next round (capped at 100),
do NOT evolve the generator — just run more seeds at current complexityconsecutive_clean >= 2: run Generator Evolution (see below),
then reset K back to base_k for the next roundseeds array, increment roundAll commands use the timeout utility:
timeout 60 for vole test (test suites can be slow)timeout 30 for vole run (single main function)timeout 90 in vole-reduce oracle command (extra headroom for reduction)--timeout 60 as vole-reduce oracle flag (for hang detection)Timeout = potential infinite loop. Check the reduced code to determine:
Tickets exist to track your investigation data as you work, not just to
file and forget. When a sema or codegen bug is found (after reduction), create
a tk ticket immediately, then use it as a running log of your investigation.
Generator bugs are simple vole-stress fixes and don't need tickets.
tk create "stress-hunt: <short description of bug>" -d "<detailed description>"
The description should include:
.claude/stress-hunt-repros/<name>/)Record the ticket ID in the seed's ticket_id field in state.
As you investigate, add notes with findings. The ticket is your running log — if you hit the 15-minute limit, these notes are what makes the ticket useful for future investigation:
tk add-note <id> "Root cause: <explanation>"
tk add-note <id> "Attempted fix: <what you tried>"
tk add-note <id> "Blocker: <why this is hard>"
tk add-note <id> "Relevant code: <file:line — what it does>"
tk add-note <id> "Fixed in commit <hash>: <what was changed>"
tk close <id>
tk add-note <id> "Skipping after 15min. Tried: <approaches>. Blocker: <issue>. Leads: <suggestions>"
Leave the ticket open with all investigation notes — it becomes a backlog item with enough context for someone to pick up where you left off.
When a round completes with zero non-dupe failures, decide whether to evolve or just run more seeds:
The goal is to generate increasingly exotic but valid Vole code — with a bias toward feature interactions and edge cases over breadth.
This runs before the next round starts, not in parallel.
.claude/stress-hunt-journal.md before picking a feature —
check for reserved words, syntax gotchas, and known pitfalls.vole file in /tmp/ that uses the feature you plan to
generate. Run it with cargo run -- test /tmp/test_feature.vole (or
cargo run -- run /tmp/test_feature.vole). This catches wrong assumptions
about syntax early (e.g. tuple indexing is tuple[0] not tuple.0).
If the syntax doesn't work as expected, adjust your plan or pick a
different feature. If you learn something new (reserved words, syntax
quirks), append it to the journal.test/**/*.vole files to understand valid syntax/structuresrc/tools/vole-stress/just pre-commit to verify the change compilesfor seed in 999999 888888 777777 666666 555555; do
cargo run -p vole-stress -- --seed $seed --profile full --output /tmp/vole-stress-validate
timeout 60 cargo run -- test /tmp/vole-stress-validate/<dir>/
rm -rf /tmp/vole-stress-validate/<dir>
done
Also validate any new profile specifically with 3+ seeds.profiles array in
the state file so it gets used in round-robin seed distribution going forward.When a generator evolution causes test failures, determine the failure type:
Generator bug (invalid code generated):
Action: fix the generator. If you can't fix it, revert entirely
(git checkout -- src/tools/vole-stress/).
Compiler bug (valid code that exposes a compiler/runtime issue):
Action — two parts, BOTH required:
Part 1: commit the generator change — it found a real bug, that's good.
Part 2 (mandatory — do NOT skip): attempt to fix each failing seed's compiler bug before proceeding to the next round. For each distinct failure:
vole-reduce.claude/stress-hunt-repros/tk ticket for the compiler bugAfter all failures are attempted (fixed or skipped), check whether the generator change causes widespread failures:
tk create "stress-hunt: re-add <feature> after <compiler-bug-id> is fixed" \
-d "Generator evolution for <feature> was reverted because it triggers <bug>. Re-add once the compiler bug is resolved."
tk dep <re-add-id> <compiler-bug-id>
Only proceed to the next round after Part 2 is complete.
Roll a dice (shuf -i 1-10 -n 1) to select the evolution category:
| Roll | Category | Why |
|---|---|---|
| 1-4 | Feature interactions | Cross-cutting combos are where real bugs hide |
| 5-7 | Edge cases & boundaries | Boundary conditions stress codegen/RC/sema |
| 8-9 | Language features | New syntax coverage, structural patterns |
| 10 | Breadth | Stdlib methods, API surface |
Record the roll and chosen category in the commit message (e.g.
vole-stress: [roll=3/interactions] generate closures capturing union fields).
Reference test/**/*.vole files for valid syntax patterns before implementing.
Feature interactions (rolls 1-4) — combine 2+ features that stress different compiler subsystems together. These find the most bugs because they exercise codegen paths that individual features don't reach alone.
Pick combinations, not individual features. Examples:
try block inside closure)Edge cases & boundaries (rolls 5-7) — boundary conditions, unusual but valid inputs, and patterns that stress reference counting, memory layout, or type resolution.
Examples:
x??.field??.method())Language features (rolls 8-9) — new syntax or structural patterns the generator doesn't use yet. Focus on features that are likely to interact with existing generation in complex ways.
Examples:
Array<Optional<MyClass<T>>>)unreachable keyword in exhaustive branchesBreadth (roll 10) — adding individual stdlib methods, API surface coverage. Still useful but unlikely to find new bug categories.
Examples:
test/ files for syntax, but don't copy tests wholesaleWhen fixing compiler bugs, always prefer fixing upstream in the pipeline. The compiler has a strict layered architecture:
Parser → Sema → VIR Lowering → Codegen
The golden rule: Codegen should read decisions, never make them. Desugar early — when codegen needs type-specific behavior, add annotations/lowering in sema or VIR rather than type-detection special cases in codegen.
Sema (src/crates/vole-sema/src/analyzer/) — type-driven decisions,
annotations, error detection. If the bug is "codegen doesn't know X about
this expression", the fix is to have sema annotate X in the NodeMap or
VIR metadata, NOT to have codegen re-derive X from types.
VIR Lowering (src/crates/vole-sema/src/vir_lower/) — desugaring,
normalization, making implicit operations explicit. If the bug involves
a missing operation (e.g. RC inc/dec, type coercion, iterator wrapping),
the fix is usually to emit the correct VIR nodes during lowering rather
than adding special-case detection in codegen.
VIR types/metadata (src/crates/vole-sema/src/vir/) — if codegen
needs additional type information to generate correct code, prefer adding
it to VIR node metadata (e.g. VirExpr fields, VirType variants) so
codegen can read it mechanically.
Codegen (src/crates/vole-codegen/src/) — instruction selection and
memory layout ONLY. Fix here only when the bug is genuinely about wrong
instruction emission, not wrong type classification or missing metadata.
if type_id.is_X() checks in codegen → should be sema/VIR annotationmatch arm in codegen for a type variant → likely needs VIR nodeAlways inspect VIR output for the failing test case to understand what sema and VIR lowering are producing:
cargo run -- inspect vir path/to/reduced/file.vole
cargo run -- inspect vir -f function_name path/to/file.vole
This shows the VIR that codegen receives. If the VIR looks wrong (missing coercions, wrong types, missing RC ops), the fix belongs in VIR lowering. If the VIR looks correct but codegen produces wrong code, the fix belongs in codegen.
Also useful:
cargo run -- inspect ir path/to/file.vole # Cranelift IR (what codegen emits)
cargo run -- inspect ast path/to/file.vole # Parser output
When fixing sema or codegen bugs, always add a regression test to
test/unit/. The test should:
test/unit/language/interfaces/)filter().collect()), find and fix the
root cause in src/tools/vole-stress/. A recurring generator bug that is
never fixed is wasted signal in every future round.just pre-commit before any committk if any are takentest/unit/ for sema/codegen fixes