بنقرة واحدة
بنقرة واحدة
| name | testo-flaky-tests |
| description | Stabilize flaky Testo tests with |
Two attributes, two different jobs. Don't mix them up.
| Attribute | Purpose |
|---|---|
#[Retry] | On failure, run again up to maxAttempts total. Pass = the run is green. By default the run is also marked flaky if a retry was needed. |
#[Repeat] | Always run times runs in total. Used to surface flakiness or stress-verify a fix. |
Both should be a last resort — first investigate the root cause (shared global state, time/timezone, ordering, randomness, network). Surface this to the user before reaching for #[Retry].
Fetch https://php-testo.github.io/llms.txt for the current attribute namespaces and parameters.
#[Retry] — make a known-flaky test green-ishuse Testo\Retry;
use Testo\Test;
#[Test]
#[Retry(maxAttempts: 3)]
public function pollsExternalService(): void
{
$response = $this->api->fetch();
Assert::same($response->status, 200);
}
Constructor (verified against plugin/retry/Retry.php):
public function __construct(
public int $maxAttempts = 3,
public bool $markFlaky = true, // ← default ON
) {}
maxAttempts is the total number of attempts (3 = first run + up to 2 retries).markFlaky is on by default — when a retry was needed, the run is reported flaky even though it eventually passed. Do not disable this unless the user explicitly asks: silent retries are how flakiness rots a suite.TARGET_CLASS is allowed). Apply at the class level only when every test in it is independently flaky for the same external reason — that's rare; usually it's a smell.#[Repeat] — run a test N times unconditionallyuse Testo\Repeat;
#[Test]
#[Repeat(times: 50)]
public function concurrentInsertNeverDeadlocks(): void
{
$this->runConcurrentInsert();
}
Constructor (verified against plugin/repeat/Repeat.php):
public function __construct(
public int $times = 2, // total runs, NOT additional repetitions
public int $maxFailures = 0, // failures tolerated before the whole loop fails
public bool $markFlaky = true, // ← default ON, reports flaky if any run failed but stayed within maxFailures
) {}
times is the total number of runs. #[Repeat(times: 3)] runs the test three times (mirrors Kotlin's repeat(n) and JUnit's @RepeatedTest(n)). It is not "additional repetitions on top of one run".maxFailures defaults to 0 — any single failure fails the whole loop.#[Retry]: Repeat runs inside Retry — each retry attempt re-runs the full repeat cycle. Possible, but the semantics are subtle; surface it to the user before suggesting both.Use cases:
#[Repeat(times: 100)], run locally, remove before merging).Don't ship #[Repeat(times: 50)] long-term on a fast suite — CI cost adds up. Remove or scale down once the fix has been validated.
#[Retry(maxAttempts: 3)] (markFlaky is already on by default).#[Repeat(times: N)], run locally, remove before merging.#[Repeat], never #[Retry].markFlaky on #[Retry] / #[Repeat] (it defaults to true). Setting markFlaky: false is silent rot — a flaky test that retries to green hides the underlying defect. Only flip it off when the user explicitly asks.Repeat(times: N) is total runs, not extra runs. Repeat(times: 1) runs the test once. People coming from older PHPUnit @Repeat semantics expect "additional" — they're wrong here.#[Retry] with #[Repeat] is allowed: Repeat runs inside Retry (each retry attempt re-runs the full repeat cycle). Only suggest both when the user genuinely wants that nesting.Expect::exception(...) and #[Retry] is almost always wrong — expected exceptions are deterministic by design.SkipTest / CancelTest from the body short-circuits both #[Retry] and #[Repeat] — the loop stops immediately and the result keeps the Skipped / Cancelled status. That's intentional (skipping isn't a failure to retry against), but worth knowing when a "flaky" test is actually skipping on some runs.Migrate an existing PHPUnit test suite to Testo. Use when the user says "migrate from PHPUnit", "port phpunit tests", "convert TestCase to Testo", or has files extending PHPUnit\Framework\TestCase that need to move to Testo's attribute-based style.
Author a Testo plugin — event listeners, interceptors, custom container bindings, or new test attributes. Use when the user wants to extend Testo's behaviour (custom reporters, lifecycle hooks across the suite, attribute-driven middleware, integrating an external system) rather than writing a single test.
Write or modify tests in a project that uses the Testo PHP testing framework. Use when adding a
Set up or edit `testo.php` — the Testo application config. Use when the user is bootstrapping a project (including running `vendor/bin/testo init`), adding/removing a suite, scoping a finder, wiring an application-wide plugin (coverage, JUnit), or asking "where do I configure Testo" / "how do I initialize Testo".
Parameterize Testo tests with
Write or tune Testo performance benchmarks with