mantis-dedupe

name	mantis_dedupe
description	Consolidates raw security findings to eliminate redundant reports. Use when raw findings have been generated by the researcher and need consolidation before review. Don't use for initial code auditing or patch generation.

Deduplicator (/mantis_dedupe)

System Goal

Duplicate Finding Merger. Evaluates lists of raw findings to cluster and consolidate identical or highly overlapping issues into singular, descriptive records.

Command Definition

• Command: /mantis_dedupe
• Description: Consolidates raw security findings to eliminate redundant reports.

Instructions

Review a list of security findings and merge duplicate findings that refer to the exact same security flaw or adjacent code paths.

Execute your task as follows:

1. Load Raw Findings & Current Loop Queue:

   • List the contents of the directory and read the files in workspace/findings/. If the directory is empty or does not exist, notify the user and exit.
   • Check if learnings.jsonl exists in the workspace. If it does, read its contents. This file represents findings and insights that have already been evaluated by downstream agents (like Reviewer or Patch) within this current loop iteration against this specific version of the codebase.
   • Important: Do NOT read or deduplicate against historical_learnings.jsonl (VCS history), as we want to catch regressions if old bugs were reintroduced.

2. Filter Loop Duplicates: Cross-reference the current findings against the learnings.jsonl entries (using code_paths and title similarities). If a current finding exactly matches a flaw that was already processed in this loop (regardless of whether its status was FALSE_POSITIVE, NON_VIABLE, SAMPLE_OR_TEST, or VERIFIED_SECURE), you must delete the new finding file entirely and drop it from your active list. This prevents the pipeline from getting stuck re-evaluating the same issues in the current pass.

3. Filter Duplicate Findings in Current Batch: Check the current findings against each other to find duplicates (using code_paths and title similarities). If multiple findings refer to the exact same flaw or highly overlapping code paths, they must be merged.

4. Map/Reduce Chunking Strategy (For Scale): If there are many finding files (e.g., > 20 items), use a Map/Reduce approach to group them by target file or component before checking for overlaps to avoid context window limits.

5. Token-Optimized Consolidation and Merging: To minimize LLM output tokens and prevent data loss, do not manually rewrite or output the merged JSON files in your response. Instead, follow this pattern:

   1. Identify Duplicates: Internally map which findings are duplicates of a primary finding.
   2. Reusable Deterministic Scripting: Write a reusable helper script (e.g., workspace/helpers/merge_findings.py) during your first merge action. For all subsequent merges, do not regenerate the script; simply execute the existing helper script with the new finding IDs. The script must follow these deterministic rules:
      • Title: Pick the most comprehensive and descriptive title.
      • ID: Preserve the unique "id" of the primary finding being kept.
      • Severity: Pick the highest severity level specified among the merged items.
      • Privileges Required: Inherit the most severe privilege requirement (priority: NONE > LOW > HIGH).
      • User Interaction: Inherit the most severe user interaction requirement (priority: NONE > REQUIRED).
      • Code Paths: Collect and deduplicate all file paths and line numbers into a single unique array.
      • Description, Mitigation, & Impact: Concatenate cleanly.
      • History: Concatenate and preserve all "history" entries from the merged findings. Append a new entry to the "history" array for this merge action, referencing the IDs of the deleted duplicate findings that were merged into this one.
   3. Execute the Script: Run your script to update the primary finding's file (workspace/findings/<primary_id>.json) on disk.

6. Clean Up: Delete the redundant .json files of the duplicate findings that were merged to clean up the directory and prevent redundant analysis in downstream stages.

When complete, notify the user.