| name | writing-tests |
| description | Use when writing, reviewing, removing, or restructuring ANY test in StarEnchants — deciding whether a test should exist, which layer it belongs in, and how to write it so it catches real bugs without breaking on non-bugs. Supersedes the unit-vs-live notes in live-server-testing / matrix-gate. |
Writing tests
The governing policy for tests in this repo: whether a test should exist, which
layer it belongs in, and how to write it so it catches real bugs without breaking
on non-bugs. The target design is docs/dev/internals/testing-architecture.md.
This supersedes the "when to write a live test" guidance and the unit-vs-live
notes in the live-server-testing / matrix-gate skills. When this conflicts with
older prose, this wins.
Two failure modes are equally bad: tests too dense to catch a real bug
(boilerplate, "verify the mock was called", exact-string assertions) and too
sparse on the edges that actually break (lifecycle teardown, Folia regions, error
paths). Push every test toward the middle: assert the contract a real bug would
violate, once, against the one place that owns it.
Rule 0 — A test must justify its existence
Name the specific bug it would catch that nothing else catches. If you can't,
don't write it. A test that can only fail when a human edits the test itself (a
getter returning the value the test set two lines up; a != null on a field the
canonical constructor already requires) is noise.
Coverage is a tool, not a target. Write a test per contract — a routing
decision, a numeric formula, a serialization layout, a precedence rule, an error
path — not per public method. One contract may be a single @ParameterizedTest row;
one method may carry several contracts.
Rule 1 — Never re-type a string production owns (single source of truth)
The #1 rule. A test must never re-type a user-facing or config string production
already defines — a display name, lore line, color code, message, doc/example,
tab-complete label, command output, default material/identifier, DSL token, spec
head, alias, or delimiter. Re-typing creates a two-place edit: change the
production string and the test breaks for a non-bug, pointing at the test, not a bug.
Assert against the same source production reads, or feed your own input:
- A value with an accessor/registry/spec/constant → reference it.
assertEquals(TraksConfig.defaults().block().material(), …), not SLIME_BALL.
LoreStyle.DEFAULT.unknownLabel(), not "§8Unknown Enchant". For a section of
defaults, compare the whole record: assertEquals(SoulGemConfig.defaults(), gem).
- A transform/algorithm → feed a test-owned fixture and assert the output;
the literal is then your input, not the catalogue.
LoreRendererTest passing its
own LoreStyle + name map is the correct model.
- A diagnostic → assert the code through
schema.diag.DiagCode, never the
message wording: assertTrue(d.is(DiagCode.E_COND_TYPE)). Producers emit
DiagCode; tests read it back — one symbol, no wire-string literal.
- A DSL token / spec head / param key → drive it from the kind's own
EffectSpec (SpecDrivenCtx fills a FakeEffectCtx straight from the spec), not a
bare "give"/"AOE"/"who".
- A synthetic test-owned code (a diagnostic the test itself constructs to check
propagation, e.g.
Diagnostic.error("X_BAD", …)) is fine — the test owns both
ends, so there is no drift.
Exact text may be pinned in exactly two sanctioned ways: (1) a golden file
regenerated from the single production source via a regen flag (never a hand-copied
literal); (2) a format/algorithm spec (render(), usage(), numerals,
word-wrap) evaluated over a test-owned fixture.
Litmus test. If changing one production string would force editing a test that
is not specifically testing that string's transform, the test is coupled. Fix
it — reference the constant, or use a synthetic fixture.
Reconstructing the string is re-typing in disguise — and worse in a live suite.
Reading the config template and re-applying the translator yourself
(ChatColor.translateAlternateColorCodes, a renderer) is not single-sourcing: in a
booted server, production also round-trips the value through Bukkit's ItemMeta
(set→get normalises colour codes — a redundant §r§d collapses to §d) and a
configurable suffix, so the rebuilt expectation can never equal the real display
name. (This is the bug that bit EconomyItemsSuite's transmog check — its colored()
helper was byte-identical to production's translator, yet the assertion still
diverged.) In a live suite assert the state (PDC, counts, booleans) or the
structure of the change (base name kept, the count present, a re-apply idempotent),
or compare against production's own output — never a string the test rebuilt.
Colour translation lives once, in platform.text.Colors.translate (ADR-0033);
RenderingDisciplineArchTest bans ChatColor from the harness so this can't creep back.
Rule 2 — A family of one behavior is one data-driven test, not N files
When several tests differ only by input and expected output — the effect-kind wiring
tables, per-error diagnostics, per-store TTL contracts — they are one parameterized
test with rows, not one file each. A new instance is one row; a missing case is a
structurally absent row, loudly visible, not a silently-missing file.
But — collapsing must preserve every assertion the per-instance tests carried.
The audit overturned 27 of 34 proposed effect deletions because each was the sole
guard of its kind's contract. A collapse is a coverage-preserving rewrite (Rule
5): each old @Test becomes a row that keeps its distinct-value arguments (to
catch a transposition), its multi-target case (to catch a broken loop), and its
verifyNoMoreInteractions (to catch a spurious intent). Dropping any of these is a
regression, not a cleanup.
The effect-kind tables are the worked example: FanOutEffectTest (per-target / per-
player / player-only fan-out), ModeDispatchEffectTest (mode/channel/side/type
branchers), LocationEffectTest (world/block/spawn + no-op guards),
FlagAndSoulEffectTest (no-target flags + soul). Each row is a DynamicTest so a
typed verify(sink)… survives intact; the ctx is testfx.FakeEffectCtx.
Rule 3 — Pick the layer by what only it can prove
Default to the lowest layer that can prove the contract (cheapest first):
- Pure-unit — deterministic logic over real production types, no Bukkit, no
mocks. DSL/compiler/engine-math/codec/render-algorithm/resolution. The default.
- Data-driven corpus — a family of the above (Rule 2).
- Contract & property — single-source enforcement and architectural invariants
(
SpecConformanceTest, purity guards, structural shape).
- Golden / snapshot — the only home for shipped user-facing strings and
generated artifacts (Rule 1, sanctioned way #1).
- Live integration — the last resort (Rule 4).
A mocked Sink is not "testing a mock": the Sink is the mutation boundary,
so verifying the emitted intent verifies the contract. Use the strict
FakeEffectCtx for the ctx (it throws on an unset param, so a mistyped param key
fails loudly instead of passing on a mock's silent 0/null).
Rule 4 — Live tests are reserved for what a booted server alone can prove
Write a live suite only when the truth lives in the real game and nothing below
can reach it: a real Bukkit event producing observable server-side state (assert
the state change, never client motion — clientless fake players have none); real
PDC serialize/deserialize across the mapping flip; Folia region/thread
correctness (stage a second region for a cross-region contract — a single-region
test proves nothing); real inventory/GUI routing; cross-version registry presence on
the running version; reload atomicity under a concurrent reader.
Live mechanics are non-negotiable (each fixes a real flake): wait in game ticks,
never wall time; fresh actors per scenario with listener teardown
(HandlerList.unregisterAll); reset captors immediately before the staged
action; pin event counts (exactly-once), not just values; sanitize the arena.
Run the same suites on real Folia, not just Paper. See live-server-testing.
Rule 5 — Never delete coverage; re-home it
When you remove or merge a test, point at the specific row, golden line, or case
that now carries every contract the old test pinned — before the old file is
deleted, with both green. A change may not shrink covered branches. "This looks
redundant" is a hypothesis to verify against the production code, not a license to
delete.
The only deletions that need no re-homing: tests of test doubles, non-tests
in the test set (a code-gen tool with zero assertions), and dormant gates the
maintainer signs off as dead (e.g. a one-off migration-equivalence gate after the
migration ships).
When you add a feature — the rubric
- Is there a contract a real bug would violate? No → no test (Rule 0).
- Does production already own a string/value I'd assert? Yes → reference it; if
it's shipped display copy, it belongs in a golden, not a literal (Rule 1).
- Is this one instance of a family already tested as a table? Yes → add a
row, not a file (Rule 2).
- What is the lowest layer that can prove it? Start there (Rule 3); reach for
live only if a booted server is the only oracle (Rule 4).
- Did I touch an edge that ships dark? Error path, absence, overflow, a Folia
region boundary, a teardown/lifecycle path — pin it; sparseness there is a defect.
- If I removed or merged anything, is its contract re-homed and green? (Rule 5.)
In practice
- A new effect kind → one row in the matching effect table (a
FanOut row,
or a ModeDispatch row if it branches), built from a FakeEffectCtx.
SpecConformanceTest covers it automatically (it drives every kind from its own
spec, so an undeclared/mistyped param trips the strict ctx). A new file only for
genuinely novel branch logic. No new live suite — the combat spine already
proves an effect reaches the world.
- A new compiler shape → build the def through
testfx.Defs (the one place the
AbilityDef/LoweredAbility record arity lives), so a new field is a one-place
change, not a parallel edit across every stage test.
- A runtime record (
Ability/WornState/Snapshot) → build it through
testfx.Abilities/WornStates/Snapshots (the one place their arity — incl. the
ADR-0039 factMask/kindId defaults — lives), never a raw constructor or a mock.
- A sink or dispatcher → wire it with
testfx.Envs.sink()…build() (a SinkEnv
over an EngineStores aggregate with override-what-you-test store slots), never a
telescoping ctor that default-constructs a shadow store.
- A scheduler double → install
testfx.SyncSchedulerBackend (inline) or
testfx.RecordingSchedulerBackend (captures *Later for runDelayed(), arms
repeating* as cancellable handles), never a hand-rolled SchedulerBackend.
- Structural (server-free) content validation → compile permissively with
testfx.PermissiveResolvers.INSTANCE and, for migrator writers,
testfx.Specs.lookup(registry) — not a re-typed anonymous resolver/lookup.
- A new config field → assert it round-trips through the real loader from your
own YAML; assert defaults via
.defaults(); assert any clamp/reorder with a
reversed/out-of-range input. No exact-string default copy.
- A new user-facing message/lore/menu label → it lands in a golden via one regen;
review the diff. Don't assert the literal.
- A new Folia-relevant world mutation → a live suite with cross-region staging
and an exactly-once event count.
The compiler fuzz gate (registry-derived)
engine/boot/CompilerFuzzTest + LoaderFuzzTest fuzz the content compiler and
loader FROM the live registries: every registered effect/selector kind gets seeded
valid + near-valid generated content on every ./gradlew build, asserting totality
(never throws; every fault a closed-set DiagCode) and erase-stage soundness.
Coverage grows automatically with each registered kind — zero per-kind authoring.
- A new kind needs no fuzz edit — the
@TestFactory registry walk picks it up.
A new ParamType.Kind/ValueKind breaks ContentFuzz's exhaustive switches at
compile time: extend the generator, never add a default.
- A kind that gains a
CrossRule will fail its valid-content case — teach
ContentFuzz.validNamed the rule; do not loosen the no-blocking-diags assertion.
- Determinism is load-bearing: fixed
BASE_SEED, per-(head,case) seeds, draws
only from deterministically ordered lists (registry.kinds(),
TriggerRegistry.names(), sorted var keys). Never draw by iterating a
Map.copyOf/Set.of collection — immutable-collection iteration order is salted
per JVM run.
- Failures print
head / case / seed / generated content / diagnostics; the seed
plus case index reproduces the sample exactly.
- Pin a mutation's
DiagCode only when it is deterministic for every kind
(unknown head/param/trigger, missing arg, range); fuzzier mutations assert
totality + closed codes + structural soundness only — pinning them breaks on
non-bugs.
Anti-patterns — never do these
- Assert an exact lore/menu/command string a production constant defines (Rule 1).
- Reconstruct a production-rendered display name/lore in a live suite (read the
template, re-apply the translator) and assert equality — it drifts on Bukkit's
ItemMeta colour-code normalisation even when your translator is correct; assert
state/structure instead (Rule 1, RenderingDisciplineArchTest).
- Assert a diagnostic message; assert the code via
DiagCode (Rule 1).
- One file per effect kind / store / provider when they share a contract (Rule 2).
- A live suite for pure logic a mocked
Sink or synthetic event proves (Rule 3/4).
verify(mock).x() where the asserted value is the mock's own default (ticks=0,
multiplier=0.0) — use a distinct non-default value so the wiring is pinned.- A single-target effect test (can't catch a broken fan-out loop) — use two.
- A single-region live test for a cross-region contract (proves nothing).
- Delete a test because it "looks redundant" without re-homing its contract (Rule 5).
