ledger

name

ledger

description

The quantitative research ledger — the twin of `trail`. trail records *that* a hypothesis was confirmed; the ledger records *by how much*. A long-running agent logs each experimental run as commit-message trailers (Metric / Config / Artifact / Runtime), so the runs reconstruct from `git log` with no database, and the ledger mechanically checks every machine-readable `Predicts:` against the recorded `Metric:` — flagging the dangerous case where the written `Outcome: confirmed` disagrees with the number. Renders a fixed-template black-and-white run table / sweep / leaderboard in ~/.anu/atlas/ledger/<repo>, no LLM in the loop. Use when running experiments whose numbers matter, when recording a measured result, when comparing runs or a parameter sweep, or when the user runs /ledger or asks to see the run ledger / leaderboard / sweep.

ledger — record how much, not just whether

trail makes a long run's reasoning legible: the forks, the bets, the roads not taken. But research turns on numbers, and a decision graph that says Outcome: confirmed never says by how much, against what config, backed by which artifact — and never checks whether the number actually clears the bar the hypothesis set. The ledger is that missing quantitative half. It asks one more thing of a run that already commits its result: name the measurement, the knobs, and the file — and state the bar as a number you can be wrong about.

Same substrate as trail, same promise: the record lives in the commit message, so it travels with every clone and the whole ledger rebuilds from git log with no database and nothing hidden. The one thing the ledger does that you must not do by hand: it computes the verdict itself. When your Predicts: reads as metric op number, the ledger tests it against the matching Metric: and tells you holds or violated — and if your Outcome: says confirmed while the number says violated, that contradiction is the loudest thing on the page. Cheap verification of an expensive run; the false-confirmation a verification-first harness exists to catch.

The vocabulary

Run trailers go in the last paragraph of a commit message (standard git-trailer shape — they coexist with trail's Hypothesis:/Resolves: and with Co-Authored-By:). They sit on the commit that produced the number — usually the same forward commit that resolves a hypothesis.

Trailer	Meaning
Metric: <name> = <value> [unit]	A measured number. Repeatable — one line per metric. The value must parse as a number; the unit is kept for display.
Config: <k>=<v> <k>=<v> …	The knobs that produced this run — its identity in a sweep. Repeatable; space-separated k=v. Put seed= here.
Artifact: <path>[@<sha>]	The file that backs the number (a CSV, a plot, a checkpoint). Repeatable. @<sha> pins the blob.
Runtime: <k>=<v> …	Optional provenance: gpu=, wall=, node=, commit=. Repeatable.
Run: <label>	Groups runs into one experiment (a sweep). Rows sharing a label become one comparison table. Defaults to the resolved hypothesis slug.
Predicts: <metric> <op> <number>	The checkable bar. op ∈ >= > <= < == ~= (~= holds within 5%). This is trail's Predicts: made machine-readable — write it as an assertion and the ledger grades it.

Reused from trail (read that skill for the grammar): Hypothesis:/Resolves:/ Outcome:/Evidence:/Why:/Goal:. A run is just an ordinary resolving commit that also carries Metric:/Config:/Artifact:.

Apodized taper beats the baseline insertion loss

Resolves: apodized-beats-baseline
Outcome: confirmed
Run: taper-sweep
Config: taper=apodized seed=42
Metric: insertion_loss = 0.34 dB
Metric: bandwidth = 40 nm
Predicts: insertion_loss < 0.50
Artifact: results/apod.csv@a1b2c3d
Runtime: gpu=A100 wall=412s
Evidence: results/apod.csv
Why: 0.34 dB beats the 0.61 dB uniform baseline across the band

The check — the one thing no model does by hand

If Predicts: parses as metric op number and a Metric: of that name exists, the ledger grades it mechanically:

• holds — the number satisfies the assertion.
• violated — it does not. This is a real result, not a failure — record it plainly.
• unchecked — no checkable prediction, or no matching metric. Most runs start here; that's fine.

And the contradiction the page shouts about: Outcome: confirmed with a violated prediction (or Outcome: refuted with one that holds). The number disagrees with the verdict you wrote — almost always you wrote confirmed out of optimism, or the threshold was wrong. Either way a human should look. This is the ledger's reason to exist; do not paper over it by loosening the Predicts: after the fact.

Direction for "best in column" is read from the prediction's operator: </<= means lower is better, >/>= higher. With a known direction the ledger marks the best run per metric (●); with none, it marks nothing — it will not guess which way is good.

Recording discipline

• One run = one commit's trailers. The commit that produced the number carries it. Forward-only and append-only, exactly like trail: a re-run is a new commit, never an --amend.
• State the bar before you know the answer, in the hypothesis-opening commit's Predicts:. A bar you set after seeing the number isn't a prediction.
• Don't over-record. Trailers go on genuine runs, not every tidy-up. A run with no measured number is just a commit; the floor is still a good subject + Why:. Over-recording is its own slop.
• The metric must be real and re-checkable. The Artifact: is how a reviewer re-derives it without rerunning. A fabricated number is the worst thing you can put here — the whole point is a record someone can trust without repeating the work.

Render the ledger

Reconstruct the table from git and render the fixed B&W page (no LLM):

LEDGER="$HOME/.local/share/anu/plugins/ledger/skills/ledger"
ROOT=$(git rev-parse --show-toplevel)
OUT="$HOME/.anu/atlas/ledger/$(basename "$ROOT")"; mkdir -p "$OUT/history"
TS=$(date +%Y%m%d-%H%M%S)
[ -f "$OUT/ledger.json" ] && cp "$OUT/ledger.json" "$OUT/history/$TS.json"
python3 "$LEDGER/build.py" "$ROOT" "$OUT/ledger.json"      # HEAD's runs
python3 "$LEDGER/render.py" "$OUT/ledger.json" "$OUT/index.html"
open "$OUT/index.html"

As an investigation companion (the quantitative sibling of the trail.html companion /investigate already renders): build with --swarm to merge every worktree-agent's runs into one cross-agent comparison, and write it into the investigation folder so the page is self-contained:

python3 "$LEDGER/build.py" --swarm "$ROOT" "$OUT_INV/ledger.json"   # OUT_INV = the investigation dir
python3 "$LEDGER/render.py" "$OUT_INV/ledger.json" "$OUT_INV/ledger.html"

Render the --swarm companion before tearing the swarm down — it reads the live worktrees. history/ accumulates prior runs; never delete it — it's how a sweep is diffed over time.

Where it sits

• Composes trail. trail is the decision graph (qualitative — why); the ledger is the run table (quantitative — how much). An Outcome:'s Evidence: should point at a run the ledger holds; the two render side by side.
• The quantitative half of /investigate. Each contained worker already records Resolves:/Outcome:; have it also record Metric:/Config:/Predicts: for every experiment, and the --swarm ledger becomes the investigation's comparison matrix — who got which number under which config, and whose verdict the number actually supports.
• Never edit template.html / render.py per run — fixed by design; all per-run content lives in ledger.json. Same contract as /map, /study, /investigate, trail.