Property-based and metamorphic testing expert for choosing a library, finding good properties (invariants, round-trips, idempotence, oracle/differential, metamorphic relations), and wiring generative tests into the existing test runner.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Property-based and metamorphic testing expert for choosing a library, finding good properties (invariants, round-trips, idempotence, oracle/differential, metamorphic relations), and wiring generative tests into the existing test runner.
auto_activates
["property-based testing","property based test","metamorphic testing","generative testing","fuzz property","proptest","quickcheck","hypothesis testing library","fast-check","fscheck","jqwik","shrinking"]
Expert guidance for property-based and metamorphic testing: instead of
asserting one hand-picked example, you state a rule that must hold for every
input, and the library generates hundreds of inputs to try to break it.
Where This Fits — the third leg of the formal-methods triad
amplihack ships two other formal-methods skills. Property-based testing is the
practical middle ground between them:
Approach
Skill
What it checks
Cost
Example tests
(unit tests)
Behavior on inputs you thought of
Low
Property-based / metamorphic
property-based-testing
Invariants over generated inputs, at runtime, against real code
Medium
BDD acceptance
gherkin-expert
Human-readable behavior scenarios
Low–medium
Model checking / proof
tla-plus-expert
Exhaustive state-space or temporal properties of a model
You can state a rule the code must obey but can't enumerate every input.
You have a reference implementation (an old version, a slow-but-correct
oracle, or a sibling implementation in another language) to compare against.
Prefer TLA+ instead when the property is about concurrency, ordering, or
temporal/liveness behavior of a design you can model. Prefer Gherkin when the
value is a shared, human-readable acceptance spec. These are complements, not
substitutes — a mature component often has all three.
Pick the library for your stack
Stack
Library
Runner it plugs into
Rust
proptest (preferred) or quickcheck
cargo test
Python
Hypothesis
pytest /
unittest
.NET
FsCheck (F#-friendly) or CsCheck (C#-friendly)
xunit / nunit
JS/TS
fast-check
jest / vitest / mocha
Java
jqwik
JUnit 5 platform
Do not build a separate harness. Every library above runs inside your
existing test runner as ordinary test cases. A property test is just a test
that loops over generated inputs.
Finding good properties
A "property" is a rule that holds for all valid inputs. The reliable families:
Invariant — something always true of the output (e.g. output length ≤
input length; a sorted list is still a permutation of the input).
Round-trip — decode(encode(x)) == x. Great for serializers, parsers,
compressors, config load/save.
Commutativity / associativity — order of operations doesn't change the
result: merge(a, b) == merge(b, a).
Oracle / differential — compare against a trusted reference: a naive
slow implementation, the previous release, or a sibling implementation in
another language (cross-language conformance).
Metamorphic relation — you don't know the exact output, but you know how
the output must change when the input changes. E.g. adding an item to a set
can never shrink its size; re-scaling all weights by k scales the total by k;
searching a superset returns at least as many hits.
When you can't state an exact expected value, a metamorphic relation is usually
still available. That is the whole point of metamorphic testing.
Shrinking, seeding, reproducibility
Shrinking: when a property fails, the library automatically reduces the
failing input to the smallest counterexample (e.g. from a 400-char string
to "a"). Never disable shrinking — the minimal case is the bug report.
Seeding: every run prints/records a seed. On failure, re-run with the
same seed to reproduce deterministically. Commit the seed (or the reduced
counterexample as a regression example test) so the bug stays fixed.
Budget: start with the default case count (usually 100–1000). Raise it in
CI for critical invariants; keep it low locally for fast feedback.
Worked examples (concrete property families)
The snippets below use amplihack's own invariants as worked examples.
using FsCheck;
using FsCheck.Xunit;
publicclassCoverageProperties
{
// bundle >= in_scope AND scanned + excluded == in_scope
[Property]
public Property CoverageTallyHolds(PositiveInt inScope, PositiveInt extra, NonNegativeInt excluded)
{
int in_scope = inScope.Get;
int bundle = in_scope + extra.Get; // bundle is a supersetint exc = Math.Min(excluded.Get, in_scope);
int scanned = in_scope - exc;
bool bundleSuperset = bundle >= in_scope;
bool tallyBalances = scanned + exc == in_scope;
return (bundleSuperset && tallyBalances).ToProperty();
}
}
JS/TS — fast-check (telemetry totality / safety)
import fc from"fast-check";
import { recordTelemetry } from"../src/telemetry";
// totality: recordTelemetry never throws and always returns a well-formed eventtest("telemetry is total and safe", () => {
fc.assert(
fc.property(fc.anything(), fc.string(), (payload, name) => {
const event = recordTelemetry(name, payload); // must not throwexpect(event.name).toBe(name);
expect(typeof event.timestamp).toBe("number");
expect(typeof event.dropped).toBe("boolean"); // always a boolean, never undefined
}),
);
});
For amplihack's multi-runtime components, run the same generated inputs
through each language implementation and assert identical outputs. Generate the
corpus once (with a fixed seed), feed it to every implementation, and diff the
results. Any divergence is a conformance bug — shrink it to the minimal input.
Quick quiz
Check your understanding (answers below).
A function normalize_path is claimed to be safe to call more than once.
Which property family proves that, and how do you write it?
You are testing a JSON serializer in TypeScript and want to catch encoding
bugs. Which library and which property family fit best?
You can't predict the exact ranking a search function returns, but you know
adding a matching document must never reduce the number of results. What
kind of property is this?
Which stack does jqwik target, and which runner does it plug into?
A redaction function must satisfy two rules. Name them and classify each.
Answers
Idempotence: assert normalize_path(normalize_path(p)) == normalize_path(p)
over generated paths.
fast-check, using a round-trip property: decode(encode(x)) == x.
A metamorphic relation — you constrain how the output must change, not
its exact value.
Java, plugged into the JUnit 5 platform.
Idempotence (redact(redact(x)) == redact(x)) and a no-secret-leak
invariant (the output never matches the secret). The first is an
idempotence property; the second is a safety invariant.
Related Skills
tla-plus-expert — exhaustive model checking of designs (the high-cost leg).
