sar-evaluate

name

sar-evaluate

description

Rank an incident's current findings by confidence for a mid-search review. Use when the user asks to "evaluate findings", "check triage progress", "rank the current hits", "show top candidates", or "snapshot the leads so far" — e.g. "how's triage going on incident013", "give me the top 20 leads right now". Read-only and safe to run concurrently with sar-triage — the script never writes to findings.jsonl, manifest.json, or profile.json, so an independent agent can keep triaging while this skill answers "what do we have so far".

sar-evaluate

Produce a ranked, read-only snapshot of an incident's triage state. Unlike sar-report, this skill does not render the full ground-team deliverable — it gives a quick ranked view you can run anytime, including while another agent is still triaging.

Inputs

• Incident id — e.g. incident013. Reads runs/<incident_id>/manifest.json and runs/<incident_id>/findings.jsonl.
• Top N (optional) — how many findings to surface. Default 20.
• Min confidence (optional) — filter below this. Default 0.0 (include everything, even clear).

If findings.jsonl is missing or empty, sar-triage hasn't produced anything yet — report that and exit.

Concurrency contract

Safe to run while sar-triage is in flight. Specifically:

• This skill only reads findings.jsonl. The triage appender uses flock + atomic per-line writes, so a concurrent reader sees a consistent prefix of committed findings.
• Malformed or partially-flushed lines (unlikely, but possible under I/O pressure) are silently skipped by the loader.
• Nothing in this skill touches manifest.json, profile.json, or any other file the triage agent might re-read.

You can fire this skill in its own sub-agent (or from a parallel tool call) while another sub-agent runs sar-triage. Re-running gives a newer snapshot as more tiles complete.

What to do

Run from the project root (or prefix with ${CLAUDE_PROJECT_DIR}/). The script resolves runs/ from $CLAUDE_PROJECT_DIR or its on-disk location by default; override with --output-root.

Text snapshot (default — for conversational review)

python3 .claude/skills/sar-evaluate/scripts/evaluate_findings.py \
  --incident-id <INCIDENT_ID> \
  --top-n 20

Prints a ranked text summary to stdout:

• Progress line: N / M manifest tiles reviewed (pct%)
• Tier counts: high_confidence / probable / possible / clear
• Blaze-orange hits from Stage 1 (always surfaced regardless of model verdict)
• Top N findings by confidence with frame, tile, GPS, cues, and a one-line observation

JSON snapshot (for programmatic downstream use)

python3 .claude/skills/sar-evaluate/scripts/evaluate_findings.py \
  --incident-id <INCIDENT_ID> \
  --format json --top-n 50 > /tmp/sar_eval.json

Persist a timestamped snapshot

python3 .claude/skills/sar-evaluate/scripts/evaluate_findings.py \
  --incident-id <INCIDENT_ID> \
  --output runs/<INCIDENT_ID>/evaluations/$(date -u +%Y%m%dT%H%M%SZ).md

The snapshot directory (evaluations/) is not touched by sar-triage or sar-report, so this is safe even during active triage.

Filter for high-signal only

python3 .claude/skills/sar-evaluate/scripts/evaluate_findings.py \
  --incident-id <INCIDENT_ID> \
  --min-confidence 0.5 --top-n 10

Report back to the user

• Progress (N reviewed of M manifest tiles, percent complete).
• Tier counts.
• If any high_confidence finding or blaze_orange hit exists: enumerate them by frame, GPS, and cues — these are actionable even mid-search. The top-N block from the script already formats these; quote the relevant entries verbatim.
• If the top finding is low-confidence (< 0.5), say so plainly: "nothing decisive yet, still X% of tiles to go".
• Suggest next step:
  • If triage is still running: continue — re-run sar-evaluate later for a fresh snapshot, or call sar-report once complete.
  • If triage looks complete (reviewed == total manifest keeps): recommend sar-report to generate the ground-team deliverable.

When NOT to use

• Final deliverable for a ground team. Use sar-report — it renders tile thumbnails, map links, and a GeoJSON for QGIS/ATAK. sar-evaluate is a mid-search dashboard, not a report.
• Modifying findings. This skill is strictly read-only. To change a verdict, delete or rewrite the offending line in findings.jsonl directly (outside any skill), or re-run sar-triage after clearing findings.jsonl.