benchmark

name	benchmark
description	This skill should be used whenever the user asks to run the benchmark, or perf tests for this state-machine engine (e.g. "run the benchmark", "run the benchmarks", "check perf", "pnpm benchmark", "how fast is it now"). It runs the suite, shows the results, and then ASKS whether to update the documented result tables before touching any docs.

Run the benchmark suite

Runs the @dunky.dev/state-machine benchmark and, only after asking, refreshes all documented result tables across the repo and website.

The rule

Running the benchmark and updating the docs are two separate steps. Always run first, show the user the numbers, and then ask whether to update the documented tables. Never edit docs as part of "run the benchmark" without an explicit yes — benchmark numbers are noisy and the user may just want a look.

Step 1 — run it

The suite is its own workspace package under benchmark/, but the root has a delegating script, so this works from the repo root or from benchmark/:

pnpm benchmark

It runs in one Node process with --expose-gc (~1–2 min) and prints a series of console.tables grouped by section (fan-out, compose, computed, engine, construction, memory, rendering). If the run errors:

• memory numbers look wildly noisy / a no --expose-gc warning → the script passes --expose-gc itself via the benchmark npm script; make sure you're running pnpm benchmark, not tsx index.ts directly.

Show the user the result tables (or a tidy summary of the headline rows: single-event throughput, fine-grain 1/5000, construct 10k, memory 64-field, rendering 1000-row mount + re-render).

Step 2 — ASK before updating docs

After showing the numbers, ask the user — use the AskUserQuestion tool — whether to update the documented results. Offer at least:

• Update all docs — refresh numbers in the benchmark README, the core package README, and the website benchmark page.
• Don't update — just keep the run output.

Do not proceed to Step 3 unless the user says yes.

Step 3 — update the results (only on a yes)

Three files carry benchmark numbers. Update all three in one pass:

A. benchmark/README.md — full tables (source of truth)

Each ## N. section has its table inline, followed by a one-line → takeaway. Update the numbers in those tables and recheck each takeaway's "X faster" claim (e.g. "~3.5× the events/sec") against the fresh figures.

Update only the numbers — never reword prose, change column layouts, or alter footnotes. Keep each table's existing alignment. Two formatting rules to preserve:

• Every comparison table keeps all three engine columns (Dunky, XState, Zag). Where an engine can't run a test, keep the marker — n/a ᵃ (async) or n/a ᶠ (no equivalent feature). Never drop a column to "tidy up".
• ops/sec values use K/M notation: ≥1,000,000 → M (e.g. 3,038,317 → 3.0 M), ≥1,000 → K (e.g. 9,532 → 9.5 K), below that stay raw (709). One decimal. The construction (µs), memory (KB), and rendering (ms) tables are small decimals — leave those as-is, no K/M.

Map the run output:

• Throughput → single machine, one event: core + xstate-raw from section C.
• Throughput → fine-grain 1/5000: core + xstate from section B ("Fine-grain … 5000 cells") — diffed variant.
• Construct 10k: the µs / machine column of "Construct 10,000 machines".
• Memory: the KB / machine column, written rows (thin = 2-field, fat = 64-field).
• Rendering: the "list of 1000" table — mount (ms) + re-render wall (ms) for core/instance, xstate/selector, zag; rows-woken = avg rows / move.

B. packages/core/README.md — headline ratios only

The core README carries short prose claims ("up to ~8× the event throughput", "~33× more") and a link to the benchmark README — no tables. Update only if a headline ratio clearly crossed a round number (e.g. throughput drops from ~8× to ~6×, or memory from ~33× to ~25×). Don't churn it for a rounding wobble.

Relevant lines to check:

• The ### Performance section prose claim ("up to ~8× the event throughput").
• The memory comparison claim ("~33×" in the ## How it compares diff table footnotes).

C. website/src/pages/benchmark.mdx — headline table + section numbers

The website benchmark page at /Users/ivanbanov/dev/dunky-dev/.worktrees/website/website/src/pages/benchmark.mdx carries:

1. The headline table — event throughput ops/s, memory 2-field and 64-field KB/machine. Update all three Dunky / XState / Zag cells.
2. Each section's table — "~8× the event throughput", "~10× faster selection at scale", "~33× less memory as context grows wide", "Construction cost". Update the numbers and recheck the section heading ratios against the fresh figures (e.g. if throughput is now 6.1 M vs 897 K, the heading becomes "~7×", not "~8×").

Apply the same K/M notation rules as benchmark/README.md.

After updating all three

• Re-read each edited file's tables to confirm markdown pipes still line up.
• Run pnpm format at the repo root so formatting matches the repo style.
• Remind the user these are first-look numbers from one machine — a single run is a snapshot, not a verdict.