predict-audit

# Predict Smart-Contract Audit ## ⛔ STOP — explicit run confirmation is MANDATORY This skill launches **expensive multi-agent audits** — a whole-fleet run (all three harnesses at `'all'` scope) is dozens of agents and a few million tokens. Each harness requires an explicit scope and bounds its own agent count by construction, so it cannot go maximal by accident — but it must NEVER run automatically or as a side effect of being invoked, read, or mentioned. **Before launching ANY harness run (`orchestrator` / `ownership-walk` / `rule-sweep`), you MUST:** 1. Show the user the run plan — which harness(es) and scope (`lenses` / `units` / `rules`; the explicit string `'all'` is the whole-fleet opt-in) — plus cost. Quote cost from the LAST run's measured actuals (its usage block records agents and tokens), never from intuition: guessed estimates have been 2x+ wrong. 2. Ask explicitly — e.g. *"Run this audit? (≈N agents / ≈X tokens)"* — and **WAIT for an explicit "yes."** 3. Only after the user confirms, call `Workflow(...)`. If anything is ambiguous, ask; never assume. Invoking, reading, explaining, or editing this skill never needs the gate — **only *executing a harness run* does.** When in doubt, do not run. A self-driving harness for a deep, adversarial, **smart-contract-only** audit of DeepBook "Predict" — a binary prediction market on Sui — and its three split-out sibling packages. The goal is to **find real bugs**: solvency/fund-loss, liveness/abort (a bricked flow), griefing/DoS, mispricing, broken invariants, and authorization holes — plus, via a dedicated lens, architecture/cohesion/readability issues (overgrown "god" modules, coupling, decomposition). Findings are reasoned from the code AND, where it matters, **empirically reproduced** on localnet or in Python. This is a *code-audit* skill (security/correctness + maintainability). It does NOT do testnet-readiness/deploy/ops checks (Move.toml hygiene, post-deploy admin steps, indexer parity, runbooks). If asked for those, say so and scope them separately. Scope is FIXED to the three Predict-cluster packages (predict + propbook + account); do NOT broaden it to deepbook core or other repo packages. Upgrade / object-layout migration correctness is also out of scope (matches `move.md` pre-deploy) — revisit only when nearing deploy. ## When to use - "Audit Predict" / "deep audit of the predict contracts" / "find bugs in Predict before testnet." - "Review these Predict changes for correctness/exploits/invariant violations." - Any request to assess Predict economic safety at the Move-source level. ## Packages in scope (all Move source, read-only) The oracle and custody layers were extracted out of `predict`, so the audit spans three packages; Block Scholes data enters through the external `bs_oracle` signature verifier (Block Scholes-owned, git-pinned — a trust boundary to reason about, not audited surface). Dependency order (leaves first): ``` bs_oracle (external verifier) account (account, account_registry, account_events) fixed_math (math) ─────────────┐ │ ▼ │ propbook (registry, oracle_lane, feeds/{pyth,block_scholes_sid,block_scholes_store}, constants) ▼ ▼ predict (31 modules — the center of gravity) ``` `predict` is the top consumer and gets most of the audit budget; the siblings are audited for their own correctness AND for the trust `predict` places in them (lens 08). ## Method (5 phases) 1. **Ground truth — MAIN LOOP ONLY.** `sui build` + `sui test` all three packages (`--warnings-are-errors`), and a localnet sim smoke. "Tests don't compile / are red" is itself a finding. Results feed the audit. 2. **Find.** The orchestrator's lenses loop-until-dry (bug findings are SAMPLED, so each lens re-runs — bounded by `maxRounds`, retiring per-lane once dry — until it stops surfacing new ones). The rule sweep and ownership walk are SINGLE-PASS: mechanical and per-module conformance ENUMERATE rather than sample, and the next run is round 2. 3. **Adversarial verify — High/Critical only, cross-model.** Only findings that can lose funds or brick a flow earn verify subagents: the orchestrator panels High/Critical (codex refute + codex repro + Claude settled — an escalated finding clears a different model than the one that found it; needs the codex CLI), the walk sends its 'high' violations to one codex verifier. Everything Medium and below is reported RAW — **the operator is the verifier for that tail** (that is where dispositions actually come from). A null codex verdict retries once on Claude, then surfaces as `unverified-panel` rather than silently degrading. 4. **Empirical deep pass.** The economic-simulation lens (orchestrator 09) runs the localnet + Python sims and writes new adversarial scenarios to actually break invariants. 5. **Consolidate (no-slip) + curate.** Read each harness's FULL output file (the notification preview is truncated), run `consolidate.py` to deterministically merge all three into ONE `consolidated-report.md` covering **open + settled + refuted + coverage** with a `DROPPED 0` accounting guarantee, then reason over the committed `packages/predict/predeploy/open-items.md` and curate findings into it. The consolidator's coverage section catches buried items; never hand-pick from a truncated return. ## Harnesses Three complementary review *shapes*, all sharing this `primer.md`, the report format, and the read-only / compiler-in-main-loop discipline. Read `primer.md` first either way. Scope is REQUIRED for each — there is no whole-fleet default. - **Lens fan-out** — `orchestrator.workflow.js`: the 10 perspective lenses (broad multi-angle bug hunt). Or hand a single `lenses/NN-*.md` to one session. Scope with `lenses: [<keys>]`, `'security'` (all lenses minus the cleanup-tier pair surface-area + architecture — the default choice for a bug hunt), or `'all'`. Loops until dry with per-lane retirement (`maxRounds` 3 / `dryRounds` 2 defaults; explicit cap args are the only tuning knobs) and panels High/Critical findings cross-model. - **Ownership walk** — `ownership-walk.workflow.js`: recursive *per-module* conformance against the contextual ownership/boundary/policy rules in `references/ownership-rules.md` (R1–R7). Map (barrier) → single per-module check pass → one codex verifier per 'high' violation ('correctness'/'cleanup' reported raw). Scope with `units: [<keys>]` or `'all'`. Use this for "who owns what / where are boundaries violated." - **Rule sweep** *(refreshed from the old root `rule-auditor.md`)* — `rule-sweep.workflow.js`: per-*rule* sweep for the 17 mechanical/local rule families (config-reads, config API shape, public-API exposure, object-identity keys, `create_and_share` naming, ProtocolConfig gate matrix, arithmetic-guard noise, test-coverage, timestamp semantics, events hygiene, **dead-field liveness** — the exhaustive write-only / read-only-mirror sweep that catches the write-only-reserve bug class — plus the hygiene trio **signature-shape** / **module-layout** / **comment-hygiene** and the polish trio **test-independence** / **docs-drift** / **abort-code-multiplexing**). ONE agent per family, ALL findings raw — the operator is the verifier for this mechanical tier (it enumerates, it doesn't sample). Scope with `rules: [<keys>]` or `'all'`. The contextual ownership families (ex-rule-auditor 6/8/9) live in the ownership walk instead. Axis split: lenses = perspective × whole codebase; ownership walk = per-module (deep context); rule sweep = per-rule × whole codebase (mechanical). ## Launching a run (copyable) **Step 1 — ground truth, MAIN LOOP only** (feeds every subagent via `args.groundTruth`; a wrong command starts the run from a false "all green"): ``` python3 .claude/skills/predict-audit/preflight.py # drift lint FIRST: fatal on primer module-map drift, warns on dangling D-ids python3 packages/predict/predeploy/check.py # dev-system linter: register pinning tests, ID cross-refs, MEASURED links, dead paths sui move build --path packages/<pkg> --warnings-are-errors # pkg ∈ predict propbook account sui move test --path packages/<pkg> --gas-limit 100000000000 --package-size 64 # each of the three (predict is the big suite) (cd packages/predict && npm run build && npm test) (cd packages/predict && python3 -m unittest discover -s simulations/tests -p 'test_*.py' -v) (cd packages/predict && python3 -m harness parity --source /path/to/scenario_dataset.csv --max-rows 20) # localnet parity, MAIN LOOP only ``` `preflight.py` guards the primer (the single point of failure — its module map + D-id citations reach every subagent): a missing module path is fatal, and a cited D-id without an explicit canonical decision or response-policy entry warns. `check.py` guards the predeploy system itself (a register decision whose pinning test vanished is un-enforced; a dangling tracker cross-ref means a broken workflow). The workflows also self-warn if `groundTruth` looks empty/short, but run the lint yourself so drift is caught before the launch questionnaire. **Register obligations (every run).** `packages/predict/predeploy/response-policies.md` is part of the audited surface: (a) re-verify each entry at HEAD — the code still implements the recorded response, the pinning tests still pin it, and `docs/risks.md` still describes it truthfully; drift between an entry and HEAD is itself a finding. (b) Do NOT re-flag a registered decision whose reasoning verifies at HEAD — cite the entry instead. (c) Anti-ossification: pick 1–2 entries per run and adversarially re-derive them from scratch, ignoring the recorded reasoning (a well-reasoned decision can still be wrong by omission — the P-1 circuit-breaker removal is the precedent: its fairness argument was sound and it still silently dropped the only u64-headroom bound on LP fill math). **Step 2 — launch a harness.** Scope is REQUIRED (`lenses`/`units`/`rules`; the explicit `'all'` is the whole-fleet opt-in — a no-arg launch errors with the valid keys). The orchestrator loops-until-dry but is BOUNDED BY CONSTRUCTION (`maxRounds 3` / `dryRounds 2` / `verifyCap 60` defaults; total agents ≤ `maxRounds`×lenses + `verifyCap`×panel); the siblings are single-pass (1 agent per family / check unit, plus the walk's 'high'-tier verifiers). **Do NOT rely on a `+NNNm` budget directive to bound a run** — the budget often does NOT propagate into a background workflow (`budget.total` arrives null). Deepen the orchestrator by raising `maxRounds`/`verifyCap`/`maxFindings` via args; deepen a sibling by running it again (the next run is round 2). ``` Workflow({ scriptPath: '.claude/skills/predict-audit/orchestrator.workflow.js', args: { groundTruth: '<build/test summary>', scope: 'delta since <sha>', lenses: 'security', files: ['<changed file>', '...'] } }) Workflow({ scriptPath: '.claude/skills/predict-audit/ownership-walk.workflow.js', args: { groundTruth: '<build/test summary>', units: ['predict-plp'] } }) Workflow({ scriptPath: '.claude/skills/predict-audit/rule-sweep.workflow.js', args: { groundTruth: '<build/test summary>', rules: 'all' } }) ``` **Step 3 — consolidate in the MAIN LOOP (no-slip guarantee).** Each harness's FULL result is persisted to its task output file — **the notification preview is TRUNCATED; never synthesize from it.** Re-run any failed units first (`Workflow({ scriptPath, resumeFromRunId })`), then run the deterministic consolidator over the full files: ``` python3 .claude/skills/predict-audit/consolidate.py .claude/predict-review/reports/<run>/ <orchestrator.output> <ownership-walk.output> <rule-sweep.output> ``` It emits ONE `consolidated-report.md` (a per-run snapshot under `reports/<run>/`) covering **open + settled + refuted + coverage**, plus `findings.json`, with an `ACCOUNTING — … DROPPED 0` line proving every input finding is accounted for (it exits non-zero if anything would be lost). Write the executive summary *on top of* that report — do NOT hand-pick from the truncated returns. **Step 4 — reasoned update to the committed worklist.** Manually curate audit findings into `packages/predict/predeploy/open-items.md`, which is the only open-items document and the team-facing consolidated tracker for deploy gates, audit findings, stress-test follow-ups, and manual items. This must be a reasoning pass, not a script or mechanical overwrite: - Read `packages/predict/predeploy/open-items.md` completely before editing it. - Compare the new `findings.json` and the per-run `consolidated-report.md` against the committed worklist. - Preserve manual/non-audit items and existing grouping unless the content is stale. - Merge by substance, not by exact wording: if a finding is already represented, refresh severity/action/evidence/source as needed instead of duplicating it. - Add genuinely new findings under the most appropriate existing section, or create a concise new section only when none fits. - Keep the committed item actionable: state the risk, current evidence, and required action; include the audit finding id only as supporting provenance when useful. - Remove or edit resolved/stale items only after verifying the current tree or run output supports that change. - Use normal file edits (`apply_patch` in Codex), not a generated sync, because `open-items.md` is a general curated document and may contain manual items audit output cannot understand. - **Then run the no-slip check** so a curation drop is loud, not silent (the guarantee `track.py` used to give, without the mechanical merge): ``` python3 .claude/skills/predict-audit/check_curation.py reports/<run>/findings.json packages/predict/predeploy/open-items.md [dispositions.json] ``` It fails unless every `open` finding (and every panel-dead `unverified-panel` finding) is EITHER referenced by its 6-char id in `open-items.md` OR listed in an optional `dispositions.json` (`{ "<id>": "reason" }`) as a conscious merge/refute/duplicate/accept. Write a disposition for anything you curated by substance without pasting the id. ### Incremental & delta runs (the everyday mode) The whole-codebase last-line-of-defense run is the exception, not the default. For a per-PR or since-last-time audit, scope the run — marginal whole-codebase rounds are the lowest-yield tokens; diff-focused rounds are the highest. - **`args.files`** (orchestrator + rule-sweep) — the changed files. Lenses/families concentrate on them **plus their direct callers/callees** (grep both directions) and report anywhere the blast radius reaches. Scout the change set in the main loop first (`git diff --name-only <base>..HEAD`), then pass it. - **Ownership-walk uses `units`, not `files`** — map a changed file to its subsystem cluster and pass `units`: | changed path | unit | |---|---| | `predict/sources/plp/**` | `predict-plp` | | `predict/sources/strike_exposure/**` | `predict-strike` | | `predict/sources/pricing/**` | `predict-pricing` | | `predict/sources/registry/**`, `market_manager` | `predict-registry` | | `predict/sources/config/**` | `predict-config` | | `predict/sources/capabilities/**` | `predict-capabilities` | | `predict/sources/{expiry_market,expiry_cash,order,ewma,builder_code,predict_account,constants}.move` | `predict-core` | | `predict/sources/events/**` | `predict-events` | | `propbook/**` / `account/**` | `propbook` / `account` | - **Cross-run memory lives in the committed registers, not in run artifacts.** There is no adjudication carry between runs: route a disposition worth remembering through the [Predict development-system authority order](../../../packages/predict/predeploy/README.md#authority-order) to its single owning register, and make finders/verifiers prior-aware through that pointer. A disposition not worth committing is not worth carrying. - **Watermark**: record the audited SHA (in `reports/<run>/` or an `open-items.md` note) so the next incremental run scopes `files`/`units` to `git diff` since it and refreshes the watermark. ### Which harness, in what order - **"Audit Predict" / find bugs** → the **lens fan-out** with `lenses:'security'` (delta-scope via `files` for a change set). For a pre-merge pass, add the **ownership walk** + **rule sweep** on the touched units/families. - **Subsystem deep-dive** → ownership walk on that `units` cluster. **Mechanical hygiene / cleanup triage** → rule sweep (raw findings; you triage) — add the cleanup lenses (`lenses: ['surface-area','assertions','architecture']`) when hunting surface/assertion/architecture cleanups specifically. - **Last-line-of-defense (pre-deploy):** ground truth → all three harnesses at `'all'` scope in **separate turns** (so a slow one doesn't block the others, and to read each full output before the next) → `consolidate.py` → reasoned update to `packages/predict/predeploy/open-items.md`. ## Hard rules (non-negotiable) - **Read-only on source.** Never modify `packages/*/sources/**`. Audit-run writes are reports (to `.claude/predict-review/reports/<date>/`, gitignored), temp sims/scripts (to the session scratchpad), and the reasoned curation update to `packages/predict/predeploy/open-items.md`. - **Compiler/localnet run in the MAIN LOOP, never in a subagent.** `sui build`, `sui test`, and `python3 -m harness parity` trip the 600s subagent watchdog and the run is lost (the `CLAUDE.md` "Common verification commands" guardrail). The orchestrator runs these itself and passes results in as `args`; subagents reason from source, grep, git, and **Python** sims (fast, watchdog-safe). - **After editing `consolidate.py`, run `python3 .claude/skills/predict-audit/evals/test_consolidate.py`** (main loop). It locks the no-slip and id/dedup invariants found across skill reviews. It exits non-zero on regression; treat a red as a blocker. - **Check the REAL exit code.** Never pipe `sui build`/`test` through `tail` (it masks failures). Grep the captured log for `error`/`Test result`, or read `${PIPESTATUS[0]}`.