zicato

The board-builder copilot's deep board-craft guide — designing board entries, judges, and the weighted loss through a GUI as a DRAFT applied on confirmation. Use when a copilot is helping an operator design a board, declare judges, shape the loss to express an objective, hold out a slice, or fix a toothless eval. Covers entries (single/multi-turn, expectations, tags incl. holdout, weight), judges (declared/in-run, judge_name, per_judge_weights, board_meta, the collusion-guarded emulator), and how the loss knobs combine to express an objective. Defers the exact scalar formula to SCORING.md, the schema to BOARD-FORMAT.md, anti-overfitting to OVERFITTING.md, discrimination diagnosis to LOOP-HEALTH.md.

The tournament-builder copilot's guide to helping an operator assemble a whole zicato evaluation contract through a GUI — structure, field_size/replicates, per-structure params, the board & train/holdout split, the overfitting block, the proposer (quality/screen/memory), the weights, and the promote gate incl. the evidence gate — as a DRAFT edited through tools and applied only on confirmation. Use when a copilot is walking an operator through building a tournament. Teaches the consequence-forward discipline (every choice carries a COST in board-runs and a CONTRACT impact — applying rolls the epoch) AND the statistical-soundness discipline (the margin must clear the measured noise floor; preflight the draft before burning rounds). Defers structure choice to zicato-design-tournament-structure, board craft to zicato-build-board, holdout detail to OVERFITTING.md, proposer to zicato-design-proposer, gate weights to SCORING.md.

The entry point for CHANGING zicato's own source (not operating a workspace). Routes to the full 14-chapter development guide under docs/dev-guide/ (~22k lines, code-grounded), and inlines the non-negotiables an agent must never skip — the 10 Golden Rules, the chapter map (which chapter owns which surface), the pre-commit verification ladder, and the ten shipped bugs with the one-line tell for each. Use whenever you edit orchestrator/proposer/tournament/selection/scoring/runtime/storage/supervisor/dashboard/builder/CLI code, add a contract knob, touch the evaluation statistics or the overfitting envelope, or write tests. The operator skills (zicato-evolve, zicato-design-boards, …) teach how to RUN the loop; this teaches how to safely CHANGE it.

Tier 2 run — the flagship operating skill for driving `zicato evolve` (the self-improving loop). Choose --rounds / --mode / --max-wall-clock-seconds / stop conditions, launch with the live dashboard, and report the URL. Use this to run an evolve loop. ENFORCES the live-run gate — a real-LLM evolve requires the user's explicit go-ahead.

Trace a zicato run through its telemetry — the canonical per-run events.jsonl and loss.json, and the harmonograf session that renders it — and relate zicato's own meta-loop session (the tool itself) to the per-board-run sessions (the system under test). Use when you need to follow what a specific run did, read its drift/loss profile, or deep-link into harmonograf.

Edit a zicato scoring.json — drift-loss weights, per_judge_weights/default_judge_weight, severity and per-kind weights, the declarative transform registry (pass_transform / drift_kind_aggregation), the dotted-spec scalar_fn / drift_reducer plugins, and the promotion gate (promote_margin + pass_rate_monotonicity). Use when calibrating how generations are scored or when tournament decisions disagree with operator intuition. Lower scalar = better.

Tier 0 setup — scaffold a fresh .zicato/ workspace, register an ADK adapter + mutable trees, and prove the loop end-to-end against the deterministic mock target before spending any real LLM budget. Use this when starting zicato on a new project, wiring a new inner harness, or sanity-checking the plumbing.

Configure a zicato tournament and multi-round evolution correctly (alias: configure-tournament). Use before launching any tournament or `--rounds N` evolution run. Explains the two nested levels that are dangerously easy to conflate — the WITHIN-round bracket structure (gauntlet/single_elim/double_elim/swiss/racing) vs the ACROSS-round evolution loop — and the non-obvious fact that the proposer learns from prior_experiments + insights every round EVEN WITHOUT PROMOTION. Provides a decision guide, a recommended starting config, a cost/runtime estimator, and an 'is it actually evolving?' checklist. Use when picking a bracket structure, sizing field/replicates/rounds, debugging 'nothing is evolving', or controlling cost/throttling. Do NOT use to audit whether the board itself is correct — use zicato-audit-board for that first.

Configure zicato's proposer — the agent that proposes each round's mutation. The DEFAULT (no proposer dir) is a tool-using ADK agent that owns the read-only tool registry and runs on ADK's own Runner. Two opt-in customizations: a skill-composed text-shim proposer (drop skills/*.md, no code, runs single-shot on --auxiliary-call-llm) and a custom ADK agent (proposers/<name>/agent.py with its OWN model= and the read-only tool registry). Use when you want to steer HOW the proposer reasons (add a skill), or own the model / curate the tools (custom agent). Explains when each suffices, the Design-A model rule (a custom proposer's model should differ from the harness model), the read-only tools, and that editing the proposer or a skill ROLLS THE EPOCH — same as editing the brief.

Choose and configure a zicato per-epoch tournament structure — gauntlet (default), swiss, single_elim, double_elim, racing — and its params (field_size, replicates, swiss rounds_n, racing eta/board_fraction/rung0_board_size). Use when an epoch has more than one challenger to select among, when a single-challenger gauntlet is too noisy, or when picking the best of a large candidate field cheaply; explains the decision guide, the noise/incumbent design principles, the scoring.json `tournament` block, and that changing it rolls the epoch.

Run zicato health and interpret it — detect a toothless / degenerate evolve loop (one that runs cleanly but optimises nothing), read the detectors and severities, branch on the exit code (critical ⇒ exit 1), and prescribe the contract fix. Use whenever an epoch promotes nothing, scores look suspiciously identical, or you want to confirm the loop has real optimization signal before trusting a tournament.

Drive the zicato evolve loop one stage at a time (propose → tournament) for inspecting or debugging a single round. Use only when you need to look inside what `zicato evolve` does internally; for normal operation run `zicato evolve`.

Diagnose a zicato evolve loop that is not improving — many consecutive rejects or a flat scalar. Use when the loop runs but stops promoting; this is a decision tree that distinguishes a degenerate evaluation from a too-hard board, an over-constrained Forbidden set, or a stuck proposer, and prescribes the contract edit.

Open and read zicato's live "Console" dashboard during (or after) an evolve run — navigate the view hierarchy Environment → Epoch → Generations/round Match-ups → Boards → Mutation surface → Publication, read the configured tournament structure's match-up figure (racing survival funnel / swiss ladder / elim flow / gauntlet Δ-lanes), tell whether the loop improved, and narrate what is running. Screenshot it with browser-use. Use whenever you need to observe an in-flight or post-mortem epoch.

Audit a zicato eval board for HARNESS correctness — not candidate quality (alias: board-doctor). Use after building or changing a board, before trusting any verdict, tournament, or evolution result. Run a KNOWN baseline (a trivially-correct stub + a trivially-wrong stub) then audit the RUN for mechanics: ground-truth winnability, that the graded artifact is the agent's real output (not a summary proxy), judge-fire counts, loss-scalar ranking sanity, and determinism. Use when pass rates look flat/zero, pass seems uncorrelated with quality, judges contribute nothing, or a 'good' candidate scores worst. Do NOT use to decide if a candidate is good — use it to decide whether the board can be trusted to tell you.

Write or extend a zicato board.jsonl (the per-epoch evaluation contract) — entry kinds, the single expectation, judges, the board_meta/disable_drift header, the emulator two-callable rule. Use when adding tasks, expectations, or process judges to a zicato target, or hand-editing a frozen board.

Design a discriminating zicato board — not the JSON syntax (that is zicato-author-board) but the principles: coverage of the behaviors you care about, entries that actually separate champion from challenger (avoid all-pass/all-fail), weighting, when to use single-turn vs scripted-multi-turn vs emulated-multi-turn, the board_meta/disable_drift/judge_only header, and the emulator collusion rule. Use when deciding WHAT belongs on a board, or when a loop runs clean but promotes nothing.

Help the operator author the MANDATORY pre-run hypothesis for a zicato experiment — pick a mutation target justified by observed loss patterns and predict the outcome BEFORE the run. Use when designing the next change to test; this is a thinking/authoring skill, not a run command.

Design zicato judges and expectations — the principles behind the syntax. The two expectation families (predicate/rubric and the matcher kinds), OUTCOME expectations vs in-run PROCESS judges, the drift kinds and how drift telemetry becomes loss, severity, the weighting knobs (per_judge_weights/default_judge_weight, per_kind_weights, severity_weights, namespace_weights), the collusion guard (judge/aux callable distinct from the harness callable), and judge_only mode. Use when deciding WHAT to measure and how it should weigh — not the JSON (that is zicato-author-board) or the weight values (zicato-tune-scoring).

Rebuild the derived SQLite analytical index (`zicato reindex`) and run read-only SQL against `.zicato/index.db` for cross-run analytics. Use when the index looks stale after a hand-edit, or when you want to query loss/tournament/judge history across many runs.

Tier 2 run — audit the mutable surface of a registered inner harness with `zicato mutations` (per-id span/file/code points, --id glob, --kind, --show full, --format json, [forbidden] annotations) to decide and verify what the proposer may change before an evolve run. Use this when reviewing the mutation surface, confirming markers resolve, or checking which ids are off-limits.

Explain a single promote/reject decision — re-score one champion/challenger generation pair in isolation and read exactly why the gate fired (verdict transparency, the per-entry A/B grid, the scalar breakdown, the mutation heatmap, the score trajectory, and cost). Use this when you need to understand why a specific round promoted or rejected.

The mental model + operations for zicato's epoch/round/generation hierarchy — what an epoch is (a sealed evaluation CONTRACT), how contract-hash auto-epoching rolls epochs, the TWO senses of "round" (outer evolve round vs a tournament's inner rounds), champion/challenger vs parent/child, champion_eval_mode (full/fast/fast-degraded), the mandatory pre-run hypothesis, and the advanced `zicato epoch` escape hatches (list/new/close/switch/set-goal). Use this when an operator is confused about epoch boundaries, "rounds", lineage attribution, or wants to force an epoch by hand.

Read how the harness got here across epochs and generations — list epochs, read lineage.json, distinguish parent/child (lineage) from champion/challenger (tournament role), and drive the dashboard's side-by-side conversation diff for a board entry between two generations. Use this when you need the cross-epoch tree, not a single round's verdict.

Close an epoch and read its retrospective — run the LLM analysis pass, read analysis.md / journal.md, and re-render reports against current on-disk data. Use this when an epoch's rounds are done and you want the hypothesis-vs-outcome ledger and the closing narrative, or when an older epoch's analysis.md needs repairing.

Author or refine a zicato proposer brief (brief.md) — the epoch goal, mutation budget/constraints, and the Forbidden list of mutation ids. Use when steering what the proposer is allowed to change for an epoch, or before opening a new epoch.