Menu
How to make a good property-based test.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Non-obvious considerations when writing logic that shells out to Git from an app/lib.
How to decompose a design into reviewable implementation stages. Use when planning how to implement multi-stage features, breaking down design documents into PRs, or reviewing implementation plans before starting work.
Based on SOC occupation classification
| name | property-based-testing |
| description | How to make a good property-based test. |
Generate correct test cases, don't filter down to correct test cases. Example: don't generate all lists and filter to non-empty ones; instead, generate an element and a list and concatenate them.
When writing a test that has nontrivial requirements on the input distribution, add instrumentation to it and assert the actual observed distribution.
Example: if the last line of the test is someSet.Contains elt |> shouldEqual (referenceImplementation.Contains elt), we want to make sure we do hit the positive case sometimes; so additionally increment mutable variables positiveCount and negativeCount, and after the call to the testing framework to assert the property, check that the counts are in a reasonable ratio to each other. Assert this, rather than merely printing it; it's a bug in the test if the test isn't exploring the space.
Aim for a false-positive failure probability of 1e-11 or less, because flaky tests are terrible and the tests will probably be run very often.
Fuzz over the distribution itself where appropriate. You can cheaply explore very different regimes by biasing the generator according to some parameters and fuzzing over the bias parameters themselves.
The tests give more interpretable output if the property throws on failure, rather than merely failing an equality check.
In FsCheck, use b |> shouldEqual a (a unit-returning property) rather than a = b (a bool-returning one).
