| name | absolute-deflake |
| version | 0.5.0 |
| description | Flaky test fixes: detect nondeterministic tests empirically (repeat/shuffle/parallel runs), diagnose the root cause, fix it — never retry/skip/sleep — and verify across many randomized runs. Triggers on "absolute deflake", "fix flaky tests", "CI is flaky", "this test fails randomly/intermittently".
|
| category | workflow |
| tags | ["workflow","testing","flaky-tests","maintenance"] |
| platforms | ["claude-code","gemini-cli","openai-codex","mcp"] |
| user-invocable | true |
| argument-hint | [target] |
| license | MIT |
| maintainers | [{"github":"maddhruv"}] |
Start your first response with the 🧪 emoji.
Absolute Deflake
Find tests that pass and fail nondeterministically, diagnose the root cause of each,
and fix it — not by retrying or skipping, but by removing the source of nondeterminism.
Output is evidence (failure rate per test) → cause → fix, verified by repeated runs.
Runs the shared engine in references/health-engine.md — read it for the
DETECT → SCAN → TRIAGE → FIX → VERIFY → REPORT loop and the safety contract. This file
covers only what's specific to flaky tests.
When to use
- "Our CI is flaky", "this test fails randomly", "fix the intermittent failures".
- A test passes locally but fails in CI (or vice versa), or fails ~1 in N runs.
- Burning down a backlog of
retry/skip-marked tests that mask real flakiness.
Not for tests that fail deterministically — that's a real bug or a real regression
(/absolute work for a fix, or just fix it). deflake targets nondeterministic failures.
What it scans
Establish flakiness empirically — a test isn't flaky because someone said so. Use
preferences.health.deflakeRuns from config as the default N for repeat-runs (else 20):
| Ecosystem | Repeat-run / detect |
|---|
| Jest/Vitest | run suite N× (--run loop), randomize order (--shuffle / testSequencer) |
| pytest | pytest-randomly + pytest --count=N (pytest-repeat); -p no:randomly to A/B |
| Go | go test -count=N -shuffle=on ./..., -race |
Also mine signals: existing retry/flaky/skip annotations, CI history if reachable, and
run the suite both in isolation and in full/parallel — order- and concurrency-
dependent failures only show one way. Record a failure rate per suspect test.
Common root causes (diagnose, don't guess)
