mantis-calibrate

# Risk Calibrator (/mantis-calibrate) ## System Goal Risk Analysis Expert. Evaluates confirmed findings against a rigorous risk matrix, taking into account successful reproduction and production viability to produce a final risk score (1-10). ## Command Definition - **Command:** `/mantis-calibrate` - **Description:** Calibrates the risk level of findings based on evidence and impact. ## Input/Output Contract - **Reads**: - `workspace/findings/*.json` (all finding files to load full pipeline state). - `workspace/kb/THREAT_MODEL.md` (if exists, to check threat boundary overrides and asset criticality). - `workspace/.mantis_state.json` (to track the current loop pass, and to read `active_snapshot` — `{snapshot_id, snapshot_pinned, root}` — for finding provenance and history stamping; absent ⇒ degraded/today's behavior). - **Writes**: - Updates finding files in-place with scoring/calibration fields (`impact_score`, `likelihood_score`, `availability_tier`, `inferred_exposure`, `attacker_position`, `mantis_risk_score`, `priority`, `sanity_triage_applied` (may begin with `STALE_EVIDENCE` when the STALE-EVIDENCE guard suppresses heuristics), `calibration_checklist` (entries may carry `STALE_EVIDENCE:` reasons), `outrage_commentary`, `executive_summary`). Appends a `history` entry with `snapshot` provenance (see history JSON template in the body). - Reusable helper script `workspace/helpers/append_calibrate.py`. - **Preconditions**: - Confirmed or raw findings must exist in `workspace/findings/`. - **Idempotency Guarantee**: - Updates findings in place by overwriting existing keys with the calculated score. Running multiple times on the same inputs yields identical outputs, with no duplicated entries. ## Instructions Convert the raw security findings and their empirical results (repro/patch) into a prioritized, actionable risk report. Execute the calibration as follows: **Locator resolution (findings-only).** Calibrate never needs to read target source to compute a score, but some heuristics below may re-inspect code; resolve the code root the same way every stage does. **Block A** is inlined below: ``` LOCATOR RESOLUTION (before reading ANY target code or artifact): 0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect), you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from state for provenance/annotation; NEVER stop merely because a code root is unset. 1. Determine CODE_ROOT, in this priority order: a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root. It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback (used when a caller hands you a prepared tree, e.g. a patched shadow). b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it. c. Else read state_root/workspace/.mantis_state.json (state_root from --state_root if passed, else ./workspace/... relative to the current dir) -> active_snapshot.root / .snapshot_id / .snapshot_pinned. d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory, treat snapshot_pinned = false (MODE-OFF). Do NOT stop. 2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a): verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is deliberately mutated and is sentinel-EXEMPT.) 3. PATH FIELDS: - SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is NOT of the form <existing-path>:<integer> is a non-source LOCATOR (symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL line-range and line-existence logic. - STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT): kb_references, repro_file_path, reattack_file_path, helper scripts, report files, and all state/findings JSON. 4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy (mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may cd into CODE_ROOT. 5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT (the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT lacks .git/.hg/.repo. 6. Every shell command uses ABSOLUTE paths and sets its own working directory on that call. Do NOT assume the working directory persists between calls. ``` > [!NOTE] **CURRENT-PASS CHECK (defensive; the binding guarantee is on the > harness per `mantis-pipeline-adapter` Scenario 2):** if `active_snapshot` is > present AND `active_snapshot.pass != state.pass_number`, treat the snapshot as > STALE for this pass — STOP "stale active_snapshot: pass mismatch" or degrade > as HALT (`snapshot_pinned` effectively false: no authoritative verdicts, Block > B NOT_MATCHED, reproduce `not_attempted`). This catches a custom harness that > preserved `active_snapshot` across the Stage 15 pass increment without > re-pinning. The reference meta-agent re-pins every pass, so this check never > fires there. Block B itself cannot detect this (it is `snapshot_id`-only, not > `pass`-aware). **Snapshot provenance & STALE-EVIDENCE guard (mechanical; do this before scoring):** P0. Read `active_snapshot` (`{snapshot_id, snapshot_pinned, root}`) from `state_root/workspace/.mantis_state.json` (Block A step 0/1c). NEVER stop if it is absent — calibrate is a FINDINGS-ONLY stage (Block A step 0). P1. MODE (single decision for the whole run — branches on `active_snapshot` presence, 3-state model): - `active_snapshot` is ABSENT in state (no `--sync` was requested — MODE-OFF = today's default) -> MODE = MODE-OFF. Score **exactly as today**: run every heuristic as written, do NOT compute PROVENANCE, do NOT emit `STALE_EVIDENCE`, and do NOT emit any HALT/PINNED banners. (Backward-compatible default path — byte-for-byte today's behavior.) - `active_snapshot` IS present AND `snapshot_pinned` is not exactly `true` (HALT mode — the tree raced or could not be pinned) -> MODE = HALT. `SNAPSHOT_ID` = `active_snapshot.snapshot_id` (a `live:` id). Compute PROVENANCE per P2 below and fire the STALE-EVIDENCE guard conservatively (same as PINNED for guard purposes), because findings' locators may be stale. Authoritative verdicts (VERIFIED_SECURE, failed_to_reproduce, DUPLICATE, FALSE_POSITIVE, NON_VIABLE) are forbidden this pass. - `snapshot_pinned` == `true` -> MODE = PINNED. `SNAPSHOT_ID` = `active_snapshot.snapshot_id`. Full provenance + STALE-EVIDENCE guard as below. P2. In MODE = PINNED or HALT, compute PROVENANCE per finding F using Block B's compare rule: - `F.discovery_commit` missing OR empty OR the literal `"MIXED"` -> NOT_MATCHED. - `F.discovery_commit` != `SNAPSHOT_ID` (exact string compare, no fuzzy) -> NOT_MATCHED. - `F.discovery_commit` == `SNAPSHOT_ID` -> MATCHED. P3. A per-finding heuristic is **STALE** for F when MODE is PINNED OR HALT AND either: - PROVENANCE(F) == NOT_MATCHED, OR - the finding's `code_paths` target file does NOT exist under CODE_ROOT. (Resolve CODE_ROOT via Block A steps 1 & 3; strip a trailing `:<digits>`; read ONLY the pinned root, NEVER the live tree. If you cannot resolve CODE_ROOT at all, treat the file as absent -> STALE.) P4. When a heuristic is STALE for F, do NOT apply it; keep the **conservative** (pre-adjustment) score; and ensure the literal token `STALE_EVIDENCE` is the FIRST token of `sanity_triage_applied` (add it once, before any UNKNOWN warnings and any fired-rule list). The four guarded heuristics and their exact STALE handling are: - **Dead-code 0.2 multiplier** (Section 2, "Asset Criticality & Reachability"): do NOT apply the 0.2 reduction; use the multiplier you would otherwise have. - **`repro_failure`** (Section 3 rule): do NOT force-LOW; set `calibration_checklist.repro_failure.outcome = "UNKNOWN"`, `reason` beginning `"STALE_EVIDENCE: "`. - **`vague_code_paths`** (Section 3 rule): do NOT force-LOW; set `calibration_checklist.vague_code_paths.outcome = "UNKNOWN"`, `reason` beginning `"STALE_EVIDENCE: "`. - **`static_confirmation` trace-lift** (Section 3 rule, its "valid external stack trace/sanitizer…" exception): do NOT apply the trace-lift; KEEP the static HIGH cap in force (`likelihood_score` \<= 3, 0.8 Hazard multiplier, NOT CRITICAL). Set `calibration_checklist.static_confirmation.outcome = "APPLIES"`, `reason` beginning `"STALE_EVIDENCE: trace-lift suppressed; trace/crash-log may predate the active snapshot; "`. P5. NON-SOURCE finding: if the finding's `code_paths` entry is a non-source LOCATOR (contains `"://"`, or is not of the form `<path>:<integer>` — per Block A step 3: a symbol / offset / endpoint), SKIP all source-only heuristics for it — dead-code, file-path/imports/caller-hierarchy exposure inference, `vague_code_paths`, and static trace re-inspection — and score from the finding's DECLARED metadata (`attacker_position`, `privileges_required`, `production_viability`, `repro_status`, threat model). A non-source skip is NOT flagged `STALE_EVIDENCE` (it is normal, not drift). Non-source findings default `inferred_exposure` to `"INTERNAL"` (0.8) unless the finding/threat-model declares otherwise. 1. **Load Full Pipeline State:** - Read all JSON files from the `workspace/findings/` directory. Because the pipeline appends data to each finding file at each stage, these files provide the complete picture of each finding's journey (including its `id`, reproduction status, and production viability). - **Missing Fields Fallbacks:** If any finding is missing viability, or reproduction fields (such as chained findings), apply the following fallback defaults before scoring: - If `production_viability` is missing, treat it as `"CONDITIONAL_VIABLE"`. - If `repro_status` is missing, treat it as `"not_attempted"`. - **Snapshot provenance (only when MODE is PINNED or HALT):** for each finding, compute PROVENANCE (MATCHED / NOT_MATCHED) per preamble step P2 and record it in scratch. This value gates the STALE-EVIDENCE handling in Sections 2 and 3 (preamble steps P3–P4). In MODE == MODE-OFF, skip this — score as today. - Read `workspace/kb/THREAT_MODEL.md` from the Knowledge Base (if it exists) to evaluate component exposure, trust boundaries, asset criticality, and any custom **Calibration Overrides** (e.g., specific threat positions or caps that should be lifted or customized for the project). - **Batch Processing:** If there are more than a few findings to calibrate, split the task into batches (a few findings at a time). If you have the ability to invoke subagents, delegate each batch to a subagent to process in parallel, then aggregate the results. **Each batch/subagent inspects ONLY the pinned snapshot:** pass `--snapshot_root=<active_snapshot.root>`, `--snapshot_id=<active_snapshot.snapshot_id>`, and `--state_root=<workspace parent>` so every subagent resolves the same CODE_ROOT via Block A. Any code inspection a batch performs (exposure inference, dead-code, static-trace verification) MUST read through that CODE_ROOT (the pinned copy), NEVER the live tree, and MUST honor the STALE-EVIDENCE guard (preamble P3–P4). Findings-only fields (score, priority, history stamp) are computed from the finding JSON regardless. 2. **Calculate Risk Score (1-10):** For each unique finding file, calculate the actual technical risk score in a matrix form based on the following formula components, where **Hazard = Impact + Likelihood**: - **Impact (1-5):** Evaluate impact using the CIA triad (Confidentiality, Integrity, Availability) while strictly considering **Blast Radius**. - 5: Complete, systemic loss of Confidentiality (full data breach, leak of root cryptographic/HSM master keys) or Integrity (system compromise, e.g., clear Remote Code Execution (RCE) by an unprivileged attacker who isn't already in an effective position to execute code). MUST NOT be used for attackers who already have execution privileges. - 4: Substantial loss in one or more areas. This includes systemic Availability loss (total outage of a major service) or major data exposure. - 3: Moderate loss (e.g., partial data exposure, temporary or partial system disruption). - 2: Minor loss (e.g., minor information leak, localized disruption). A vulnerability whose blast radius is limited to affecting *only a single user's own data* MUST NOT be scored higher than 2. *Exception:* If the action lacks non-repudiation (allowing the user to plausibly deny the action to commit fraud or blame others), or triggers side-effects affecting other users/system stability, it should not be downgraded. - 1: Negligible impact on CIA, mostly a cosmetic issue. Findings of the type "the code is fragile", "lack of defense-in-depth", or purely theoretical hygiene issues MUST have an Impact score of 1, ensuring they are rated LOW at most. - **Security Control Bypass (Upgrading):** If the vulnerability directly bypasses a core security control (e.g., authentication, authorization, cryptographic signature verification) or defeats the primary security purpose of a library (e.g., a library meant to secure keysets allows attacker control), elevate the Impact score to at least **4** (or **5** if it leads to systemic compromise), even if the immediate technical impact seems localized. - *Note on Privileges Required & Lateral Movement:* - If the finding requires **HIGH** privileges (e.g., administrative privileges, admin-to-super-admin escalation) or only allows lateral movement/pivoting between internal components from an already compromised state, cap its individual Impact score at **2**, unless the exploit results in escaping the container boundary (to the host node) or cross-tenant escalation. - If the finding requires **LOW** privileges (e.g., standard authenticated user), cap its individual Impact score at **3** (unless it leads to systemic compromise of other tenants/users, OR it directly bypasses a core security control/library purpose, in which case it can be higher). - These caps apply to *individual* findings. If successfully chained into an Exploit Chain (Super Finding) by the chainer, the chain itself should be evaluated based on the privilege level required for the *entry point* (initial step) of the chain. - **Likelihood (1-5):** Evaluate the probability of occurrence based on proven exploitability rather than theoretical difficulty. - 5: Actively exploited in the wild, OR the agent successfully generated a functional, weaponized exploit (not just a unit test). - 4: Public Proof of Concept (PoC) exists, OR the agent generated a highly plausible but partially weaponized exploit. - 3: No functional exploit, but the attack vector is trivial to automate. - 2: Theoretical and highly complex (requires local access, strict timing). - 1: Strictly theoretical risk with no known exploit path. - **Reachability-in-Practice Modifier:** After determining the base likelihood, reduce the `likelihood_score` by **1 or 2** (but not below 1.0) if the exploit path relies on uncommon or non-default usage patterns. This applies if: - The specific tainted parameter is populated from attacker input only during rare API calls, uncommon configuration fields, or in data formats rarely processed in the wild. - The vulnerability requires non-standard or administrative-only configurations that are rarely enabled in practice. - **Context Multiplier (0.1 - 1.0):** - If `status` is **FALSE_POSITIVE** or **NEEDS_RESEARCH**, or if `production_viability` is **NON_VIABLE**: Drop this finding completely. Do not score it or update its file with calibration data. - If `production_viability` is **VIABLE**, **CONDITIONAL_VIABLE**, or **SAMPLE_OR_TEST**: - **Network/Trust Exposure:** - If the finding resides inside an **Exposed Interface / Trust Boundary** (directly accessible to untrusted inputs): 1.0. - If it resides in an **Internal Component** accepting semi-trusted parsed data: 0.8. - If deeply nested inside a **Privileged/Trusted Zone**: 0.5. - **Inference when Threat Model is Missing/Incomplete:** If `workspace/kb/THREAT_MODEL.md` does not exist or does not mention the component: - Analyze the file path, imports, and caller hierarchy to infer exposure (e.g., public APIs vs internal helpers). **Read only the pinned CODE_ROOT (Block A), never the live tree. NON-SOURCE (preamble P5): skip this file-path/imports/caller analysis for non-source LOCATOR findings — default `inferred_exposure` to `"INTERNAL"` (0.8) unless the finding or threat model declares otherwise.** When MODE is PINNED or HALT and the finding is NOT_MATCHED or its `code_paths` file is absent under CODE_ROOT, do NOT lower the exposure multiplier below the declared/default value on the basis of live re-inspection (keep the conservative, higher multiplier). - Default the Exposure Multiplier to **0.8** (Internal) and `inferred_exposure` to `"INTERNAL"` unless there is clear evidence of direct external exposure (EXPOSED) or deep nested isolation (PRIVILEGED). Local SUID/LPE binaries should default to `"INTERNAL"` exposure. - If the finding description, history, or critic reasoning suggests the component is "rarely exposed", "internal only", or "unlikely to be attacker-reachable", reduce the Exposure Multiplier to **0.5** or lower. - **Map Exposure and Attacker Position Metadata:** - Resolve **`inferred_exposure`** based on the Network/Trust Exposure multiplier: - Multiplier 1.0 (Exposed Interface) -> `"EXPOSED"`