菜单
Flaky test fixes: detect nondeterministic tests empirically (repeat/shuffle/parallel runs), diagnose the root cause, fix it — never retry/skip/sleep — and verify across many randomized runs. Triggers on "absolute deflake", "fix flaky tests", "CI is flaky", "this test fails randomly/intermittently".
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/maddhruv/absolute --skill absolute-deflake命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
正在显示 SKILL.md
| name | absolute-deflake |
| version | 0.5.0 |
| description | Flaky test fixes: detect nondeterministic tests empirically (repeat/shuffle/parallel runs), diagnose the root cause, fix it — never retry/skip/sleep — and verify across many randomized runs. Triggers on "absolute deflake", "fix flaky tests", "CI is flaky", "this test fails randomly/intermittently". |
| category | workflow |
| tags | ["workflow","testing","flaky-tests","maintenance"] |
| platforms | ["claude-code","gemini-cli","openai-codex","mcp"] |
| user-invocable | true |
| argument-hint | [target] |
| license | MIT |
| maintainers | [{"github":"maddhruv"}] |
Start your first response with the 🧪 emoji.
Find tests that pass and fail nondeterministically, diagnose the root cause of each, and fix it — not by retrying or skipping, but by removing the source of nondeterminism. Output is evidence (failure rate per test) → cause → fix, verified by repeated runs.
Runs the shared engine in references/health-engine.md — read it for the
DETECT → SCAN → TRIAGE → FIX → VERIFY → REPORT loop and the safety contract. This file
covers only what's specific to flaky tests.
retry/skip-marked tests that mask real flakiness.Not for tests that fail deterministically — that's a real bug or a real regression
(/absolute work for a fix, or just fix it). deflake targets nondeterministic failures.
Establish flakiness empirically — a test isn't flaky because someone said so. Use
preferences.health.deflakeRuns from config as the default N for repeat-runs (else 20):
| Ecosystem | Repeat-run / detect |
|---|---|
| Jest/Vitest | run suite N× (--run loop), randomize order (--shuffle / testSequencer) |
| pytest | pytest-randomly + pytest --count=N (pytest-repeat); -p no:randomly to A/B |
| Go | go test -count=N -shuffle=on ./..., -race |
Also mine signals: existing retry/flaky/skip annotations, CI history if reachable, and
run the suite both in isolation and in full/parallel — order- and concurrency-
dependent failures only show one way. Record a failure rate per suspect test.
| Cause | Tell | Fix |
|---|
| Test-order / shared state | passes alone, fails in suite (or vice versa) | isolate state; reset/teardown between tests |
| Time / clock | fails near midnight, DST, or under load | fake timers / inject clock; no real sleep |
| Async race / missing await | fails under parallelism or slow CI | await the actual condition; no fixed timeouts |
| Randomness | fails ~X% with no pattern | seed the RNG; fix the seed in tests |
| Network / external I/O | fails offline or on slow links | mock/stub the boundary |
| Unordered collections | fails on map/set iteration order | sort before asserting |
| Resource leak / port reuse | fails on repeat or parallel runs | unique resources; clean up |
| Wave | Class | Default |
|---|---|---|
| 1 | clear, isolated cause (seed, await, fake clock, sort) | fix now |
| 2 | shared-state / ordering — needs fixture refactor | fix this pass, per test |
| 3 | flakiness pointing at a real product race, not just the test | gated — surface; may be a genuine bug to fix in code |
A flaky test sometimes means the code has a race, not the test. Don't "stabilize" the test into hiding a real concurrency bug — flag wave-3 cases for a real fix.
-race) — green once is not deflaked; green across N randomized runs is.retry/skip/flaky annotation that was masking it once the cause is fixed.sleep, or skipping the test —
that hides flakiness, doesn't remove it.sleep to dodge a race. Slows the suite and still flakes under load. Await the condition./absolute upgrade — a flaky suite makes upgrade verification unreliable; deflake first./absolute debt — flaky-test annotations are test debt; this clears them at the root./absolute work — when the flake is a genuine product-code race needing real design.基于 SOC 职业分类
