| name | roblox-proof-and-analysis-toolkit |
| description | Use when a claim needs proof, not a retry — a placed model faces the wrong way (CFrame/basis/Euler math), client and server disagree on an orientation, an economy or difficulty number needs a derived prediction before tuning, a ProcessReceipt path might double-grant or drop on crash, seeded generation should be identical across restarts but isn't, or a player-keyed table goes nil after a yield. Also for pre-tuning predictions (change-control rule 4 evidence) and "prove it, don't just try it" requests. |
Roblox Proof And Analysis Toolkit
Overview
Every recipe here turns a hunch into a checkable statement: a formula you computed by hand, an assert that fails loudly on the wrong branch, or a table derived from the actual data modules. "I ran it once and it looked right" is not proof on this stack — the placement saga (Recipe 1's case study) is the canonical precedent: five stacked causes each survived a "just try it" fix.
These recipes assume the firebit house stack: Luau strict, a single currency-service write choke point, tuning values in src/Shared/Data modules, a FeatureFlags class gating dormant features, and a ledgered ProcessReceipt flow. Verify each anchor against the current project before relying on it (one-liners in the final section).
When to use / When NOT to use
Use when:
- You are about to claim an orientation, an economy number, a difficulty value, an idempotency property, a determinism property, or a concurrency-safety property is correct.
- You need the predicted-metric half of a tuning rationale (hard rule 4 — see roblox-change-control for the rule itself).
- A bug survived one "obvious fix" already.
Do NOT use when:
- You need the lifecycle around proving (when to form a hypothesis, when to stop, how results get accepted) — that is roblox-research-methodology. This skill is the HOW; that one is the WHEN.
- You want live measurements (telemetry, console, flag audits) — roblox-diagnostics-and-tooling.
- You are triaging an unknown symptom — roblox-debugging-playbook first; come here once it names a recipe.
- You want the history of a settled battle — roblox-failure-archaeology.
- You are deciding what evidence counts as "tested/done" — roblox-validation-and-qa.
- You need platform theory (replication rules, receipt semantics, CFrame internals) rather than a proof procedure — roblox-platform-reference.
All paths below are repo-root-relative in the current project. For arithmetic, use python3 (confirm with which python3); the house aftman.toml carries no Luau script runner (no lune), so do not plan on running derivations in Luau outside Studio.
Recipe 1 — Orientation / CFrame proof
When to run: a placed model, ghost preview, spawned prop, or replicated part faces the wrong way; client and server disagree about a rotation; you are writing any code that constructs a CFrame from vectors or round-trips Euler angles.
Definitions: a CFrame is Roblox's position+rotation matrix. A basis is its three axis vectors (right, up, back). Right-handed means right × up = back; CFrame.fromMatrix(pos, right, up, back) expects that form and does not auto-correct a mirrored (left-handed) one.
Steps:
-
Prove the basis is right-handed. Two cases:
-
Building the basis yourself: derive back with the right-handed cross order — right:Cross(up), never up:Cross(right) — and guard only against degeneracy:
local right = WORLD_FORWARD:Cross(up)
assert(right.Magnitude > 1e-3, "degenerate: up parallel to reference axis")
right = right.Unit
local back = right:Cross(up).Unit
Right-handed by construction. (A fallback to a second reference axis, e.g. WORLD_RIGHT:Cross(up), is an acceptable alternative to asserting on degeneracy.)
-
Validating vectors you did NOT construct: given the exact right/up/back you hand to CFrame.fromMatrix, assert(right:Cross(up):Dot(back) > 0, "left-handed basis"). back must be the incoming vector — if you re-derive it as right:Cross(up), the assert compares the cross product with itself and passes for every non-degenerate basis, mirrored or not, and the check is vacuous.
The failure mode is silent: Roblox's network serialization normalizes a left-handed CFrame to right-handed (flips an axis), so server-local and client-post-replication versions of the "same" CFrame disagree by a 180° mirror.
-
Prove the Euler round-trip is lossless for your order. cframe:ToEulerAnglesYXZ() pairs with CFrame.fromEulerAnglesYXZ — NOT with CFrame.Angles, which is the XYZ-order constructor. Proof harness (run in Studio command bar or a spec):
local original = CFrame.fromMatrix(Vector3.zero, right, up, back)
local rY, rX, rZ = original:ToEulerAnglesYXZ()
local rebuilt = CFrame.fromEulerAnglesYXZ(rY, rX, rZ)
assert(original.RightVector:FuzzyEq(rebuilt.RightVector, 1e-4))
Prefer an Euler wire format. When a client must tell the server an orientation, send position + a YXZ Euler table ({ rY, rX, rZ } from ToEulerAnglesYXZ(), rebuilt server-side with CFrame.fromEulerAnglesYXZ) rather than raw CFrames or basis vectors. Euler angles cannot encode a mirrored basis, so the whole left-handed class of bug is unrepresentable on the wire.
Case study (learned in a shipped firebit game, 2026 — the trap placement saga): five stacked causes, each of which survived "just try rotating it": bounding-box wrapping drifting orientation, a left-handed basis that mirrored only after network normalization, CFrame.Angles vs ToEulerAnglesYXZ order mismatch, an unparented PivotTo, and an offset WorldPivot. The settled fix was the Euler wire format above. A follow-on lesson: the shared reference derivation module ended up with zero require sites after the wire format replaced its server usage — it was deliberately kept as the documented reference derivation. Flag-gated or superseded-but-documented code like that is deliberate; do not "clean it up" (hard rule 3, roblox-change-control). Full incident chronicle: roblox-failure-archaeology.
"Proven" means: the handedness assert and the Euler round-trip assert both pass, AND you logged the CFrame on both sides of replication once and they matched. One Studio run without the asserts is not proof.
Recipe 2 — Economy flow audit
When to run: before any tuning change to a price, reward, or drop rate (hard rule 4 requires a predicted metric); when asked "can a solo player afford X per cycle"; when adding any new source or sink.
Step 1 — enumerate sources/sinks from the choke point, not from memory. On the house stack every currency mutation goes through the currency service's add/take:
grep -rn "CurrencyService\.\(add\|take\)" src/Server --include="*.luau"
(Adjust the service name if the project diverges — find it by grepping the currency key, e.g. "coins".)
Step 2 — classify each hit live vs dormant. For every mutation site, trace its gating: is it behind a FeatureFlags value that is currently off, or reachable only from UI that no longer mounts? Read the flag class and the call site — never classify from memory. Dormant flows are excluded from the audit but never deleted or refactored (hard rule 3; see roblox-config-and-flags for the flag audit itself).
Step 3 — build the source/sink table, one per currency. Note each currency's persistence class (session-only vs persistent — see roblox-architecture-contract; session currencies reset on leave, which changes what "afford" means). Table shape, every row anchored to a data module:
| Flow | Amount | Source of truth |
|---|
| + NPC kill reward | value | src/Shared/Data/.../XxxData.luau:<line> |
| + Loot/pickup scatter | count range × value range (state the expected value) | file:line |
| + Level-up / daily / progression grants | formula or ladder | file:line |
| − Purchases (items, unlocks, upgrades) | price ladder | file:line |
| − Repair / maintenance sinks | rate | file:line |
Include modifiers (friend bonuses, multipliers, subscriptions) with their flag state, and the XP-to-level curve if level-ups grant currency — read the actual curve formula from its module, don't recall it.
Step 4 — compute per-cycle net for a solo player, showing arithmetic. The model: kills-per-cycle × reward-share × reward-value, plus scatter expectation, minus the realistic sinks — compared against the cheapest meaningful sink ("about one basic purchase or one upgrade tier per cycle" is the shape of a useful answer). Write the arithmetic out; use python3.
Case study (shipped firebit game, 2026): a session-currency kill reward was tuned 2 → 3 only after deriving "~9–15 per night solo" from the spawn table × composition share × reward value, i.e. roughly one cheap trap or one upgrade tier per night. The derivation, not the playtest feel, was the rationale that satisfied rule 4.
"Proven" means: every number in your prediction traces to a file:line in a data module, the arithmetic is written out, and the prediction names a measurable ("solo net session-currency/cycle ≈ +5") that a playtest or balance telemetry (see roblox-diagnostics-and-tooling) can confirm or kill.
Recipe 3 — Difficulty curve derivation
When to run: before touching any value in the wave/NPC/day-cycle data modules; when asked "how hard is cycle N"; when a playtest "felt" too hard and you need to know whether the numbers agree.
Step 1 — read the actual formulas from the data modules. Never derive from memory. The house pattern keeps them in src/Shared/Data/ (e.g. a wave-data module for spawn counts and composition, an NPC-data module for stats and per-cycle scaling, a day-cycle config for phase lengths). While reading, inventory the traps that break head-arithmetic:
- Special-case multipliers (e.g. a doubled teaching first cycle) and exactly where
ceil/floor sit in the expression.
- Caps (active-entity caps, damage-multiplier caps) and which values are deliberately uncapped.
- Multipliers that apply twice — a zone/tier difficulty multiplier may scale BOTH the spawn count AND each entity's stats; missing one halves your prediction.
- Composition/share lookup semantics (inclusive vs exclusive night ranges when picking a row).
- Boss/elite cadence (every Nth cycle) and any determinism exceptions attached to it.
Step 2 — derive the before/after table with python3. Encode the formula verbatim and print the cycles your change touches, e.g.:
python3 -c "
import math
for n in range(1, 11):
base = BASE + math.floor(n**EXP * K) # from the data module
total = math.ceil(base * (A + B * players) * (2 if n == 1 else 1))
print(n, total)"
Columns worth deriving: spawn total (solo AND max party), active cap, boss cycles, phase length, composition share. Then a worked stat example for one entity at one cycle ("a cycle-10 basic NPC has ceil(base × scale) HP, deals X to players, Y to the defended object").
Step 3 — state which rows move and by how much, BEFORE editing the file. A tuning diff without the derived before/after table violates hard rule 4.
Warning — debug leftovers distort observation. Before trusting any "per-cycle" intuition from Studio runs, check the cycle-length config for leftover test values (learned in a shipped firebit game, 2026: a day length left at 2 seconds made every day-scale observation garbage for weeks of sessions). See roblox-config-and-flags for the debug-toggle audit.
"Proven" means: you printed the before/after table for the cycles your change touches and stated which row(s) move and by how much, before editing.
Recipe 4 — Idempotency proof for receipt paths
When to run: touching the marketplace receipt service (house path src/Server/Features/Shop/MarketplaceService.luau), any receipt-callback registration, or adding a DevProduct. A DevProduct is a Robux consumable; Roblox retries ProcessReceipt until your handler returns Enum.ProductPurchaseDecision.PurchaseGranted, so every path must be safe to re-run. (Platform semantics: roblox-platform-reference.)
The house receipt flow shape (verify against the current project's service): in-progress dedupe → player-present check → processed-purchases-ledger check → unsaved-ledger retry path → product lookup → grant callbacks → write purchases[PurchaseId] → profile save → on save failure, park the receipt in unsaved → return PurchaseGranted.
Method: enumerate the crash windows and show which mechanism covers each. Fill this table for YOUR product's callback:
| # | Crash/failure window | What happens on retry | Covered by |
|---|
| 1 | Before any grant | Nothing persisted or granted; full clean re-run | Roblox's retry loop itself |
| 2 | After grant callbacks, before the purchases write | Grant already happened, no record → callbacks run again | Usually NOT fully covered — the residual window. See below |
| 3 | purchases written but the profile save fails | Receipt parked in unsaved; the retry path replays the ledger move and save — grant callbacks are NOT re-run | unsaved ledger |
| 4 | Everything succeeded; Roblox retries anyway | purchases[PurchaseId] hit → immediate PurchaseGranted, no re-grant | purchases ledger |
| 5 | Same receipt re-enters concurrently in one server session | In-progress map hit → NotProcessedYet | in-memory dedupe |
Window 2 analysis discipline: for a state-checked grant (the callback re-derives eligibility server-side — "is the target still in the grantable state?"), a window-2 retry is refused, but may then retry forever until the state recurs; decide and document whether that loop is acceptable. For a stateless grant (add N currency), window 2 is a genuine double-grant residual — either accept it explicitly in the plan file or add a pre-grant marker. Do not silently "fix" a documented accepted gap.
Case study (shipped firebit game, 2026 — a revive-type receipt): the callback re-derived everything server-side (target still dead? defended object still alive?) and returned a boolean mapped to Granted/NotProcessedYet. The player-left path deliberately returned true with the comment "report success so the receipt isn't retried in a loop" — that comment IS the idempotency reasoning pattern to copy: every return value justified in terms of what the retry loop will do with it.
"Proven" means: you wrote the five-row table for your product's callback: for each window, either name the covering mechanism or explicitly accept the residual risk in the plan file. A callback is safe only if running it twice is either harmless or detectably refused.
Recipe 5 — Determinism audit
When to run: anything that must look identical across server restarts or across clients (world generation, per-structure loot/pickup scatter), or when a "same seed, same world" claim appears.
Step 1 — build the seeded-vs-unseeded inventory for the current project:
grep -rn "Random\.new(\|Rng\.new(" src --include="*.luau"
For each hit, record: seeded from what, and is the seed input restart-stable? The house stack ships a seeded xorshift RNG with a Random-compatible surface (NextNumber/NextInteger) at src/Shared/Modules/Rng.luau (verify it exists in your project) — prefer it over raw Random.new() for anything that claims determinism. Random.new() with no argument is time-seeded: fine for intentional per-spawn randomness, disqualifying for "same every restart" claims. Some unseeded sites are deliberate design (per-spawn variety) — classify, don't "fix".
Step 2 — verify the seed derivation uses only restart-stable inputs: world position, chunk index, structure pivot (e.g. bit32.bxor of scaled X/Z coordinates with large odd multipliers) — never os.time, entity creation order, or player join order.
Step 3 — prove it with a two-run log diff: log the first N outputs of the stream keyed by seed (e.g. pickup count + first 3 positions per structure) in two separate Studio play sessions, then diff. Any divergence means an unseeded consumer shares the stream or the seed input isn't stable.
Step 4 — watch for consumption-order coupling: two consumers pulling from one RNG instance are only deterministic if their call ORDER is fixed. If consumer A sometimes rolls before consumer B, split the streams (derive a sub-seed per consumer).
Case study (shipped firebit game, 2026): per-structure loot scatter was made deterministic by seeding from the structure's pivot (bxor of scaled X and Z), while chunk/biome placement stayed on unseeded Random.new() — a known, documented gap: same structure ⇒ same loot, but chunk layout differed every restart. The audit's value was naming which half was which instead of a blanket "world gen is seeded" claim.
"Proven" means: two independent runs produced byte-identical logs for the same seeds. "The seed code looks right" is not proof — the mixing/consumption order can betray you.
Recipe 6 — Yield-safety audit
When to run: reviewing or writing any server loop that touches player-keyed state across a yield (a call that suspends the coroutine — during it, players can leave and PlayerRemoving handlers can nil out your tables). Symptom shape: intermittent "attempt to index nil" in a loop that "can't be nil".
Steps:
-
Find the yields inside loops:
grep -rn "task\.wait\|:Wait()\|WaitForChild\|Async(\|GetValue" src/Server --include="*.luau"
GetValue, all *Async, WaitForChild, task.wait, and Signal:Wait() yield. Flag every hit that sits inside a for/while iterating players or a player-keyed table.
-
For each flagged site, verify post-yield revalidation. After EVERY yield the code must either (a) re-check player.Parent (against Players, not just non-nil) and re-index the shared table, or (b) have captured the table in a local BEFORE the loop body's yields so a concurrent nil-out can't crash the tail of the iteration.
-
Check the control flow, not just the checks. The subtle variant: a guard placed after the yield but skipped by a continue/early branch on the yield's falsy return.
Case study (learned in a shipped firebit game, 2026-06 — a leaderboard friends cache): a web-API GetValue yielded inside a player loop; when it returned falsy, continue skipped the player-left check, so a player leaving mid-yield let PlayerRemoving nil the cache and a later FireClient indexed nil. The fix shape to copy: capture local cache = Service.cache[player] at the top of each outer iteration, abort if nil, re-check player.Parent ~= Players after the yield on the path that proceeds, and use only the captured local afterward.
"Proven" means: for every yield in the loop you can point at the revalidation (or pre-yield capture) that dominates every code path after it — including continue branches. If you can't trace one, write the failing scenario as a comment and fix it; to reproduce live, use roblox-debugging breakpoints to park a player mid-yield and kick them.
Common mistakes
- Fixing an orientation bug by rotating 180° at the symptom site instead of proving handedness/Euler-order at the construction site — how the trap saga started.
- Comparing CFrames by printed components instead of
FuzzyEq on axis vectors (float noise creates false negatives; mirrored bases can create false positives on position-only prints).
- Tuning an economy or difficulty number without the derived before/after table — violates hard rule 4 and produces unreviewable diffs.
- Counting dormant flag-gated sinks in the economy audit — they are dormant, and touching them violates hard rule 3.
- Declaring a receipt path safe because "the ledger has a dedupe" without enumerating window 2 (grant-then-crash-before-write).
- "Proving" determinism by reading the seed code — only a two-run log diff counts.
- Adding
player.Parent checks after a yield but leaving a continue path that skips them.
- Doing Recipe-3 arithmetic in your head: special-case multipliers,
ceil/floor placement, and tier multipliers that apply to both count AND stats are all easy to drop. Use the python3 snippet.
- Deleting a zero-require-site reference module because "nothing requires it" — check whether it is the documented reference derivation for a settled battle first (roblox-failure-archaeology).
Provenance and maintenance
Derived on 2026-07-05 from the firebit house-stack reference project. The recipes are the durable content; every path, service name, and flow shape above is a house convention that a given project may have drifted from. Re-verify the anchors against the CURRENT project before relying on them:
grep -rn "CurrencyService\.\(add\|take\)" src/Server --include="*.luau"
ls src/Shared/Data
ls src/Shared/Class/FeatureFlags.luau
grep -n "PurchaseGranted\|NotProcessedYet\|purchases\[\|unsaved" src/Server/Features/Shop/MarketplaceService.luau | head
ls src/Shared/Modules/Rng.luau
grep -rn "Random\.new(\|Rng\.new(" src --include="*.luau"
grep -rn "task\.wait\|:Wait()\|WaitForChild\|Async(" src/Server --include="*.luau"
cat aftman.toml
which python3
If a re-verify grep comes back empty, the project has diverged from the house convention — find the equivalent (grep a currency key, a ProcessReceipt reference, or registerCallback) and note the divergence rather than assuming the recipe's anchor.
