| name | reduce-test-cases |
| description | Reduce, shrink, or minimize a failing Wasmtime/Cranelift test case (.wasm/.wat, .wast, or .clif) down to the smallest input that still reproduces a bug, crash, trap, panic, or miscompilation. Gives you a predicate/command skeleton per reducer (wasm-tools shrink, Binaryen wasm-reduce, creduce, clif-util bugpoint), a wasm fallback chain for when a reducer can't parse your module, and a manual reduction playbook.
|
Reduce Test Cases
A minimal reproducer is the most valuable artifact in a bug report. Reduce a
test case in two phases:
- Automated reducers — the approach from
docs/contributing-reducing-test-cases.md, extended to .wast and .clif
and to a fallback chain for wasm the primary reducer can't parse.
- Manual reasoning — when a reducer stalls, doesn't apply (e.g. a
.clif
miscompile, which bugpoint can't touch), or over-shrinks past your bug.
The predicate is the whole game. Every reducer does the same dumb loop:
mutate the input, ask a yes/no script "does this still reproduce the bug?", keep
the mutation if yes. The reducer is generic; the predicate is where all the
per-bug intelligence lives. This skill does not ship a do-everything driver
— your bug is not knowable in advance. Instead templates/ holds a skeleton
per reducer that you copy and edit to pin your exact symptom:
| Reducer | How it hands you the candidate | "Interesting" means | Skeleton |
|---|
wasm-tools shrink | as $1 | script exits 0 | templates/predicate-shrink.sh |
Binaryen wasm-reduce | writes a fixed --test file | script's stdout and exit code both match the original | templates/command-wasm-reduce.sh |
creduce | candidate by basename, fresh temp dir | script exits 0 | templates/predicate-creduce-{wat,wast,clif}.sh |
clif-util bugpoint | (no predicate) | compile panics | (built in) |
For a miscompile — no trap or panic to grep, just a wrong result — the
predicate diffs behavior across two configs instead. That shape is orthogonal
to the reducer, so it has its own skeleton, templates/predicate-differential.sh,
that you drop into the shrink or creduce contract above. In practice this was the
most common real-world case, so reach for it first when the bug is a wrong answer.
All paths below are relative to the repo root.
Prerequisites
Install the reducers (wasm-tools ships with the toolchain; the rest are
external):
cargo install wasm-tools
brew install creduce
brew install binaryen
Build the engines your predicates will run. A debug build is required to
reproduce debug_assert! panics; a --release build iterates much faster for
everything else, which matters when a reducer runs the predicate thousands of
times:
cargo build --bin wasmtime
cargo build --release --bin wasmtime
cargo build --release --manifest-path=./cranelift/Cargo.toml --bin clif-util
Every skeleton has a WASMTIME= / CLIF_UTIL= line you must set to an
absolute path — reducers run from other directories, so a relative path
breaks. Print the absolute path and paste it in:
echo "$PWD/target/release/wasmtime"
echo "$PWD/target/release/clif-util"
Workflow
Work in a scratch directory holding your failing input. The loop never changes:
- Reproduce the bug once, by hand, to pin the exact symptom — the specific
trap string, panic message, ISLE error, or diverging output. This is the most
important step: a vague predicate reduces to a different bug. For a
miscompile, also confirm it's deterministic — run each config a few times
and check every run of a config agrees and the two configs disagree; a flaky
config makes a differential predicate unsound.
- Copy the right skeleton from
templates/ next to your input.
- Edit it: set the absolute engine path, and replace the example symptom
with yours. Keep it tight — a loose predicate lets the reducer converge on
some other, spec-legal failure.
- Test the predicate on the ORIGINAL. If it doesn't fire here, it never
will on a smaller file. (For
shrink/creduce: ./pred.sh input should exit
0. For wasm-reduce: the command should print your marker.)
- Run the reducer (per format, below).
- Eyeball the result and re-confirm it's still your bug — reducers are
happy to find a smaller program that fails for an unrelated reason.
- Re-verify the final case independently against the predicate — especially
anything a subagent/fork produced, or a hand-reduction. Convert every
candidate to the same form before comparing sizes (wasm bytes and canonical
wasm-tools print line count); raw creduce/hand whitespace isn't comparable.
- Re-author the winner to be legible, not just small. Give locals/functions
readable names, add a one-paragraph comment explaining the mechanism, and for
a
.wast make it self-checking with assert_trap / assert_return so the
file proves the bug on its own. The best artifact teaches the fix.
wasm / wat — a three-tool fallback chain
Each wasm reducer only understands the proposals its own parser implements, and
those subsets differ. Work down the chain until one bites:
1. wasm-tools shrink — try this first
Smartest, most structural, produces the smallest output. Uses
templates/predicate-shrink.sh (candidate is $1, exit 0 == reproduces):
cp .agents/skills/reduce-test-cases/templates/predicate-shrink.sh pred.sh
chmod +x pred.sh
./pred.sh input.wasm && echo "reproduces on original"
wasm-tools shrink ./pred.sh input.wasm -o reduced.wasm
wasm-tools print reduced.wasm
Verified end-to-end: a 273-byte module (noise functions, globals, a table, data
segments) reduced to 42 bytes — exactly the trapping core:
(module
(type (;0;) (func))
(export "_start" (func 0))
(func (;0;) (type 0)
i32.const 42
i32.const 0
i32.div_s
drop))
2. Binaryen wasm-reduce — if shrink can't parse your module
Different proposal coverage from shrink — and the gap is real, not theoretical:
Binaryen can't parse wide-arithmetic (i64.mul_wide_s, i64.add128) or some
SIMD, while wasm-tools shrink handles them. So for a module using those, skip
straight from shrink to creduce; wasm-reduce is a non-starter. It also has a
different contract: it writes each candidate to a fixed --test file (not
$1), and keeps a candidate only if your command's stdout and exit code both
match the original run — so print one canonical marker when the bug
reproduces. Uses templates/command-wasm-reduce.sh:
cp .agents/skills/reduce-test-cases/templates/command-wasm-reduce.sh cmd.sh
cp input.wasm t.wasm && bash cmd.sh
wasm-reduce input.wasm --command 'bash ./cmd.sh' --test t.wasm --working reduced.wasm -f
The -f matters: before reducing, wasm-reduce round-trips your module through
Binaryen's own parser/writer, and if it can't (an unsupported proposal) it prints
failed to read and write the binary and stops. -f forces past that. Verified:
with -f, the same 273-byte module reduced to 91 bytes and still trapped —
larger than shrink's 42 bytes (it kept the table/memory/globals), which is
exactly why this is a fallback, not the first choice.
3. Disassemble + creduce — if neither reducer can parse it
creduce reduces text and never re-encodes the wasm, so only the engine
has to understand the proposal — the reducer stays out of the way. This always
runs, at the cost of being purely textual: because it doesn't understand .wat
structure it tends to plateau early on parenthesis-balance constraints (in one
real case it stalled at ~43% of the original and no further). Treat its output as
a starting point for the manual playbook, not the finish line. wasmtime runs a
.wat directly, so no reassembly is needed. Uses templates/predicate-creduce-wat.sh:
wasm-tools print input.wasm -o case.wat
cp .agents/skills/reduce-test-cases/templates/predicate-creduce-wat.sh pred.sh
chmod +x pred.sh
mkdir -p /tmp/reduce && cp pred.sh case.wat /tmp/reduce && cd /tmp/reduce
creduce ./pred.sh case.wat
Verified: a 52-line disassembly reduced to two lines that still trap:
(module(export "_start"(func $a))(
func $a i32.const 2 i32.const 0 i32.div_s drop))
wast — creduce
wasm-tools shrink reduces one module; a .wast is a whole script
(multiple modules + assertions), so reduce the text with creduce. Run the
script with wasmtime wast. Uses templates/predicate-creduce-wast.sh:
target/release/wasmtime wast input.wast
cp .agents/skills/reduce-test-cases/templates/predicate-creduce-wast.sh pred.sh
chmod +x pred.sh
mkdir -p /tmp/reduce && cp pred.sh input.wast /tmp/reduce && cd /tmp/reduce
creduce ./pred.sh input.wast
Verified: a 30-line, 3-module script (with passing-assertion noise) reduced to a
single line that still traps with integer divide by zero:
(module(func $a i32.const 2 i32.const 0 i32.div_s drop)(start $a))
Feature flags come from -W on the CLI (e.g. -W gc,function-references), not
from the ;;! feature = true header — that header is only read by
cargo test --test wast. creduce will delete the ;;! header (the CLI ignores
it), so re-add it if the reduced case is destined for tests/.
clif — bugpoint (panics only) or creduce (everything else)
clif-util bugpoint <FILE> <TARGET> reduces compile panics only. It has no
predicate hook — it catches panics internally and treats a clean compile or a
verifier error as "not interesting". On a .clif that fails gracefully (the
common test compile expect-fail case) it bails:
target/release/clif-util bugpoint be.clif riscv64
For anything that isn't a hard panic — miscompiles, wrong output, graceful
codegen errors, test/run failures — use creduce with a clif-util
predicate. Uses templates/predicate-creduce-clif.sh:
cp .agents/skills/reduce-test-cases/templates/predicate-creduce-clif.sh pred.sh
chmod +x pred.sh
mkdir -p /tmp/reduce && cp pred.sh be.clif /tmp/reduce && cd /tmp/reduce
creduce ./pred.sh be.clif
Verified: a 287-line, 46-function big-endian.clif reduced to one function
that still hits the target codegen error:
function % (i64)->i128 { block0(v0 : i64) : v1 = load.i128 big v0 return v1 }
target/release/clif-util compile --target riscv64 be.reduced.clif
The predicate's sub-command is your choice: clif-util compile (codegen; target
on the CLI), clif-util test (honors in-file ; run: / ; check: directives),
clif-util run (host JIT), or clif-util interpret (backend-independent — good
for miscompile differentials). creduce deletes the test/set/target header
lines, so re-add them if you want the reduced file to run under clif-util test.
Miscompiles — differential predicates
A miscompile has no trap or panic to grep; "interesting" means the answer is
wrong, which you detect by running two configs and checking they disagree.
This was the most common real-world case. The reducer doesn't care — it's just a
predicate shape — so templates/predicate-differential.sh drops into either the
shrink ($1) or creduce (basename) contract. The load-bearing detail is the
guard:
a=$("$WASMTIME" -O opt-level=0 --invoke f "$CAND" 2>/dev/null); ra=$?
b=$("$WASMTIME" --invoke f "$CAND" 2>/dev/null); rb=$?
[ $ra -eq 0 ] && [ $rb -eq 0 ] && [ -n "$a" ] && [ -n "$b" ] && [ "$a" != "$b" ]
Both configs must run cleanly and produce output and differ. Drop the
exit-0/non-empty guards and a candidate that merely became invalid or started
trapping satisfies a != b for free, so the reducer converges on a broken
program instead of a miscompile. Real fuzzer repros invoke a named export with
arguments and need the right -W feature flags (-Wwide-arithmetic --invoke f "$CAND" 1 -1) — the defaults in the trap skeletons won't cut it.
Confirm determinism first (Workflow step 1). For a .wast, prefer the
self-checking form: encode the right answer as assert_return / assert_trap
and make "interesting" == "opt-level=0 passes but default fails".
Manual reduction playbook
When the reducer stalls, over-shrinks, or the bug is a miscompile no predicate
cleanly captures, reduce by hand. Reason from the bug's mechanism back to the
minimal trigger, and re-verify after every cut:
- Bisect coarse first. Delete whole functions / modules / assertions, half at
a time; keep the half that still reproduces.
clif-util bugpoint's own strategy
is a good mental model: remove instruction → replace with const → replace with
trap → remove block → drop unused entities → merge blocks.
- Then bisect instructions inside the surviving function the same way.
- Neutralize data. Shrink memory/table sizes to the minimum; truncate or zero
data/element segments; drop unused globals, types, imports, and exports.
- Simplify operands. Replace computed values with constants; collapse control
flow (drop unused blocks/params); inline single-use locals.
- Re-verify each step with
wasm-tools print / clif-util cat and by
re-running the engine. If a cut kills the repro, revert it and try another.
- Keep a copy of the last-good reduction so a bad cut is always recoverable.
For a miscompile, the manual pass is where you win — automated reducers
have no idea which bytes carry the bug, so they grind on structure while the
real reduction is semantic. Hunt the diverging value directly:
- Neutralize inputs one at a time, don't delete them. Replace each
local.get / global.get (or .clif value) feeding the wrong result with a
constant, one at a time; the one whose neutralization makes the divergence
vanish is carrying the corruption. Prefer this over deleting/truncating:
cutting code changes what the optimizer does, so a cut that "fixes" the bug
may have perturbed the pass rather than removed the cause — neutralization
keeps the shape and isolates the value.
- Return the suspect directly. Once you've found the diverging value,
return (local.get $culprit) (or make it the sole result) collapses the
entire downstream computation in one edit — often dropping the bulk of the
program at a stroke.
- Constant-fold the setup. Hash/mix stages, loop trip counts, and data
segments that only produce the suspect value can be replaced by the constant
they compute. Collapse a loop to the smallest trip count that still diverges.
- Then rebuild from scratch. Once the mechanism is clear, hand-writing a
fresh minimal module from the understood cause beats continuing to chip at the
reduced-but-messy one — and yields the legible artifact you actually want. In
one real fuzzer case this took a 649-line module to ~22 lines (4%), where
creduce plateaued at 43% and shrink barely moved it.
Race the reducers; don't serialize them. shrink, creduce, and a manual pass
are cheap to run at once — start each in its own temp dir (creduce reduces
in place and will collide otherwise), let creduce run in the background (it far
exceeds a single tool-call timeout), and spin up a fork/subagent for the
semantic reduction with full context on the bug's mechanism. Then take the
smallest legible result. The manual fork usually wins the hard cases.
The goal is the smallest input where the bug's root cause is still obvious — not
just small, but legible.
Gotchas
Battle scars from actually running these:
wasm-reduce preserves behavior, not "exit 0". It keeps a candidate only
if stdout and exit code match the original run — unlike shrink/creduce, which
just want exit 0. Collapse "reproduces" to one canonical stdout marker and keep
the exit code stable. And it reads the fixed --test file, not $1.wasm-reduce needs -f for unsupported proposals. It round-trips the
module through Binaryen first and prints failed to read and write the binary
if it can't. -f forces past it; if even that gets nowhere, drop to
disassemble + creduce. Some proposals it simply can't represent (e.g.
wide-arithmetic's i64.mul_wide_s) — for those, -f won't save it; skip to
creduce.- Environment variables don't reliably reach
wasm-reduce's --command.
Hardcode the absolute engine path in the command script (the skeleton does).
- creduce strips header directives. It deletes anything that doesn't change
interestingness — the
.clif test/set/target lines and the .wast
;;! feature = true header included. Fine for a repro whose predicate bakes the
target/features into the command, but the reduced file won't run under
cargo test --test wast or clif-util test unmodified. Re-add the header (or
the matching -W / --target / --set flags) afterward.
- creduce runs in a fresh temp dir containing only the candidate, referenced
by basename. Predicates must reference the candidate by basename and every
tool by an absolute path. If you hand-roll a wrapper that
cds, resolve the
predicate to an absolute path before changing directory.
- Don't put
set -e in a shrink/creduce predicate. A trapping module makes
the engine exit non-zero; under set -e the out=$(engine …) line aborts the
script before your grep runs, so the predicate never reports "interesting". Let
the final grep be the exit status.
bugpoint is panic-only, and only on the plain compile flow. It ignores
verifier and graceful codegen errors ("compiled successfully or gave a verifier
error"), and it only drives — a panic that fires only under
a pass (e.g. ) is invisible to
it. It cannot reduce a miscompile either. For any of these, use .
Troubleshooting
- Predicate fails on the original — it doesn't reproduce yet. Wrong trap
string, wrong
--invoke export, missing -W feature flag, or wrong engine
build. Fix the predicate before reducing anything.
wasm-reduce says "failed to read and write the binary" / "very unlikely
reduction can succeed" — Binaryen can't parse a proposal in your module. Add
-f; if that stalls too, disassemble + creduce.bugpoint prints "compiled successfully or gave a verifier error" (exit
0) — the input fails gracefully, it doesn't panic. Expected for
expect-fail files. Switch to creduce.creduce finishes but the file is unchanged — the predicate never passed
on any mutation. Usual causes: a relative tool path that broke in creduce's temp
dir, a predicate that isn't chmod +x, or set -e swallowing a trapping exit.
Re-test the predicate on the original and confirm absolute tool paths.
