| name | flaky-test-detector |
| description | Identifies non-deterministic or unreliable tests through static code analysis and test result analysis. Use when Claude needs to find flaky tests, analyze test reliability, or investigate intermittent test failures. Supports Python (pytest, unittest) and Java (JUnit, TestNG) test frameworks. Trigger when users mention "flaky tests", "intermittent failures", "non-deterministic tests", "unreliable tests", or ask to "find flaky tests", "analyze test stability", or "why tests fail randomly". |
Flaky Test Detector
Identify and fix non-deterministic tests that intermittently fail without code changes.
Quick Start
When a user reports flaky tests or asks for test reliability analysis:
- Identify the approach: Determine if analyzing code patterns or test execution results
- Analyze for flakiness: Look for common flaky patterns in test code or execution history
- Report findings: List identified flaky tests with specific issues
- Suggest fixes: Provide concrete remediation strategies
What Makes Tests Flaky
Flaky tests fail intermittently without code changes due to:
- Timing issues: Race conditions, fixed sleeps, async/await problems
- State management: Shared state between tests, improper cleanup
- External dependencies: Network calls, database connections, file system
- Randomness: Unseeded random data, UUID generation
- Time dependencies: Current time/date, timezone assumptions
- Resource issues: Leaks, insufficient cleanup
- Test order: Dependencies between tests
- Environment: Hardcoded paths, missing env vars
Detection Methods
