mantis-dedupe

name	mantis-dedupe
description	Consolidates raw security findings to eliminate redundant reports. Use when raw findings have been generated by the researcher and need consolidation before review. Don't use for initial code auditing or patch generation.

Deduplicator (/mantis-dedupe)

System Goal

Duplicate Finding Merger. Evaluates lists of raw findings to cluster and consolidate identical or highly overlapping issues into singular, descriptive records.

Command Definition

• Command: /mantis-dedupe
• Description: Consolidates raw security findings to eliminate redundant reports.
• Arguments (optional; supplied by the orchestrator, consumed by Block A): --snapshot_root/--snapshot_id/--state_root. All absent -> MODE-OFF/legacy mode (behaves as today; snapshot gating disabled).

Input/Output Contract

• Reads:
  • workspace/findings/ (raw finding JSON files, ignoring .trash/).
  • workspace/archive/findings_pass_*/*.json and workspace/archive/loop*_findings/*.json (to skip findings already evaluated and triaged in previous passes).
  • workspace/.mantis_state.json (to track current loop pass).
• Writes:
  • Moves duplicate findings to workspace/findings/.trash/ after setting "status": "DUPLICATE" and "duplicate_of".
  • Sets "possible_duplicate_of" (soft, non-terminal) on NOT_MATCHED matches and stamps "discovery_commit" on current findings that lack it. Reads active_snapshot/snapshot_pinned from .mantis_state.json.
  • Appends transaction logs to workspace/.tx_log.jsonl.
  • Generates/executes merging script workspace/helpers/merge_findings.py.
  • Updates primary finding workspace/findings/<primary_id>.json (merges fields and history).
• Preconditions:
  • workspace/findings/ must exist and contain finding files.
• Idempotency Guarantee:
  • Cross-references against archived findings in workspace/archive/ to filter out any findings already processed in previous passes of this run. Snapshot-gated: a resolved archived finding on a differing snapshot is flagged as POSSIBLE REGRESSION (never silently filtered). Logs transactions to workspace/.tx_log.jsonl to support tracking and potential rollbacks. Deterministic merging rules implemented in merge_findings.py.

Instructions

Step 0: Locator Resolution (run first)

LOCATOR RESOLUTION (before reading ANY target code or artifact):
0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect),
   you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from
   state for provenance/annotation; NEVER stop merely because a code root is unset.
1. Determine CODE_ROOT, in this priority order:
   a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root.
      It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback
      (used when a caller hands you a prepared tree, e.g. a patched shadow).
   b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it.
   c. Else read state_root/workspace/.mantis_state.json (state_root from
      --state_root if passed, else ./workspace/... relative to the current dir)
      -> active_snapshot.root / .snapshot_id / .snapshot_pinned.
   d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory,
      treat snapshot_pinned = false (MODE-OFF). Do NOT stop.
2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a):
   verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing
   or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is
   deliberately mutated and is sentinel-EXEMPT.)
3. PATH FIELDS:
   - SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files
     that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry
     containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is
     NOT of the form <existing-path>:<integer> is a non-source LOCATOR
     (symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL
     line-range and line-existence logic.
   - STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT):
     kb_references, repro_file_path, reattack_file_path, helper scripts, report
     files, and all state/findings JSON.
4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that
   compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy
   (mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may
   cd into CODE_ROOT.
5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command
   run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT
   (the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT
   lacks .git/.hg/.repo.
6. Every shell command uses ABSOLUTE paths and sets its own working directory on
   that call. Do NOT assume the working directory persists between calls.

[!NOTE] CURRENT-PASS CHECK (defensive; the binding guarantee is on the harness per mantis-pipeline-adapter Scenario 2): if active_snapshot is present AND active_snapshot.pass != state.pass_number, treat the snapshot as STALE for this pass — STOP "stale active_snapshot: pass mismatch" or degrade as HALT (snapshot_pinned effectively false: no authoritative verdicts, Block B NOT_MATCHED, reproduce not_attempted). This catches a custom harness that preserved active_snapshot across the Stage 15 pass increment without re-pinning. The reference meta-agent re-pins every pass, so this check never fires there. Block B itself cannot detect this (it is snapshot_id-only, not pass-aware).

Notes: workspace/findings/, workspace/archive/, workspace/.tx_log.jsonl, workspace/helpers/ and .mantis_state.json are STATE-RELATIVE (under --state_root). Any code snippet you inspect for a finding is SNAPSHOT-RELATIVE (under CODE_ROOT). Never write under CODE_ROOT.

Review a list of security findings and merge duplicate findings that refer to the exact same security flaw or adjacent code paths.

Execute your task as follows:

1. Load Raw Findings & Archived Findings Queue:

   • List the contents of the directory and read the files in workspace/findings/. If the directory is empty or does not exist, notify the user and exit.
   • Important: Ignore hidden files and directories (such as the .trash/ subdirectory) when listing or processing findings.
   • Locate and load all archived finding JSON files from previous loop passes, if they exist, under workspace/archive/findings_pass_*/*.json and workspace/archive/loop*_findings/*.json. These files represent vulnerabilities that have already been fully evaluated, triaged, and potentially patched in previous passes.
   • Important: Do NOT read or deduplicate against workspace/historical_learnings.jsonl (VCS history), as we want to catch regressions if old bugs were reintroduced.

2. Filter Loop Duplicates (snapshot-gated). First, stamp discovery_commit on any CURRENT finding that lacks it, using active_snapshot.snapshot_id from .mantis_state.json (skip when unpinned). Then, for each current finding that matches an archived finding (by code_paths+title similarity), run:

   Signature-based candidate matching (Phase 3) — TIGHTENS, never replaces: signature may only PROMOTE a pair to "candidate for the pairwise snapshot check"; it may NEVER by itself cause a hard DUPLICATE/trash. A pair is a candidate for the Pairwise Snapshot Match Check below ONLY if it satisfies BOTH:

   • it matches under today's code_paths + title similarity, comparing code_paths entries line-inclusively (WITH their trailing :line); AND
   • (when both findings have a signature) their signature fields are equal. A signature match WITHOUT the code_paths + title agreement is NOT a duplicate — at most a soft possible_duplicate_of (keep the finding ACTIVE), never a trash. Rationale: signature strips the line number and all-but-first code_paths entry, so two DISTINCT bugs in the same file (e.g. parser.c:100 vs parser.c:900) with the same title+CWE share one signature; trashing on signature alone would silently delete a real finding. If EITHER lacks , use today's + similarity matching unchanged.

When complete, notify the user.