remove-ai-marks

# Remove AI marks Multi-vendor anti-detection hygiene for **text** (Unicode + statistical rewrite) and **files** (C2PA / AI metadata across common containers). Read if needed: - `references/mark-classes.md` — Unicode / sampling / C2PA / containers - `references/vendor-notes.md` — Claude, Gemini/SynthID, OpenAI, open-LLM - `references/removal-matrix.md` — which layer when - `references/ethics.md` — intended use - `references/how-claude-marks.md` — Anthropic-specific detail - `references/markdiffusion.md` — optional MarkDiffusion image harness (schemes, honesty caveats) This skill is a **thin client**. All deterministic cleaning machinery runs in a separate HTTP service (this repo's `service/`), so the agent host needs no Python, venvs, or cleaning tools. Call the service with `curl`; never run cleaning scripts directly. ## Windows on-demand startup (Marketplace policy) Probe `/health` before use. If the default local service at `http://127.0.0.1:8765` is unavailable, run the plugin-root `Start-Service.ps1` with PowerShell 7 (`../../Start-Service.ps1` relative to this Skill directory). It starts Python from `%USERPROFILE%/watermarks-remover-service`, waits for health, and leaves the service running after use. Report startup errors and stop until health succeeds. For a custom remote URL, report connection failure instead of starting a local service. Keep login startup disabled. This policy takes precedence over upstream local-service startup instructions. ## Service access Base URL comes from `WATERMARKS_SERVICE_URL`, default `http://127.0.0.1:8765`: ```bash WM="${WATERMARKS_SERVICE_URL:-http://127.0.0.1:8765}" ``` The service is started either by the operator (`docker compose up -d`, or a published GHCR image) or locally (`make serve`). **Always check it first**, and stop with a clear message if it is unreachable — never fall back to local cleaning: ```bash curl -sf "$WM/health" # {"ok": true, "version": "..."} ``` If `WATERMARKS_SERVER_API_KEY` is set on the service, every request needs `-H "Authorization: Bearer $WATERMARKS_SERVICE_API_KEY"`. ### Capabilities ```bash curl -s "$WM/capabilities" ``` Reports which optional tools are available server-side (`c2patool`, `exiftool`, `qpdf`, `ghostscript`), scorers present (`scorers.stylometry`, `scorers.synthid`, `scorers.synthid_http`), text-watermark detectors (`text_detectors.markllm`, `text_detectors.claude-text`), and which heavy backends are configured (`pixel_backends.ctrlregen`, `pixel_backends.diffusion`, `harnesses.markllm`). **Drive your advice from this**: only recommend pixel removal / SynthID scoring / vendor detection when the service reports the backend present. ## HTTP API (curl) Payloads are JSON with the file as **base64**. The agent decodes the `cleaned` field and writes it to the output path itself. | Method | Path | Body | Returns | | --- | --- | --- | --- | | GET | `/health` | — | `{"ok": true, "version": ...}` | | GET | `/capabilities` | — | optional tools / backends present | | GET | `/openapi.json` | — | dynamically generated OpenAPI 3.0.3 spec | | POST | `/inspect` | `{"file": "<base64>", "name": "notes.md"}` | `{"ok", "kind", "suspicious", "report"}` | | POST | `/detect` | `{"file": "<base64>", "name": "notes.txt"}` | `{"ok", "kind", "detections": [...]}` | | POST | `/clean` | `{"file": "<base64>", "name": "notes.md", "options": {...}}` | `{"ok", "kind", "cleaned": "<base64>", "report"}` | `/clean` and `/inspect` route by the uploaded `name` extension plus the bytes; unrecognized formats answer `kind: "unknown"` (`/inspect`) or 400 (`/clean`). When writing a temp file for pasted text, keep a known extension (`.txt` / `.md`) in the `name` you send. The machine-readable contract lives at `$WM/openapi.json` — plug it into any OpenAPI tooling (client generators, Swagger UI, editors) instead of hand-rolling clients. `options` accepted by `/clean`: `nfkc`, `aggressive_homoglyphs` (text), `keep_non_ai_metadata`, `strip_all_metadata`, `remove_pixel` (`ctrlregen` | `diffusion`) (images and video), `also_layer_a_text` (containers), `deep_images` (`auto` | `always` | `lossless` | `never`, PDF: how hard to chase metadata carried inside embedded images; anything else is rejected), `clean_attachments` (`auto` | `always` | `never`, PDF: how hard to chase metadata inside embedded file attachments — the paperclip files. `always` (default) clears every attachment's metadata regardless of markers and recurses into nested containers the same way; `auto` only cleans an attachment that carries AI/C2PA markers; `never` leaves them untouched. Needs `qpdf`. Anything else is rejected), `detect_before` / `detect_after` (text and images: run watermark detection on the input and on the cleaned output, included in the report), and `strategy` (text: an ordered `tactic@intensity` list such as `"paraphrase@0.8,mlm@0.2"` that runs the Layer B rewrite after Layer A; when omitted the default from `config/clean_strategy.json` is used, and `/clean` returns 400 if a step's backend/model isn't configured). **Inspect first** (decide, don't guess): ```bash curl -s -X POST "$WM/inspect" -H 'Content-Type: application/json' \ -d "{\"file\": \"$(base64 < notes.md | tr -d '\n')\", \"name\": \"notes.md\"}" ``` **Clean** (text / image / container are auto-detected by name + bytes): ```bash curl -s -X POST "$WM/clean" -H 'Content-Type: application/json' \ -d "{\"file\": \"$(base64 < notes.md | tr -d '\n')\", \"name\": \"notes.md\"}" ``` Decode the returned `cleaned` base64 into the output file (`*.cleaned.*` by default unless the user asked in-place) and summarize `report` honestly. (On Windows agents, build base64 with `[Convert]::ToBase64String([IO.File]::ReadAllBytes("notes.md"))`.) ## Ethics Intended for **your own** content (privacy, hygiene, research). Do not market results as "proves human-written." If the user clearly wants academic fraud or illegal non-disclosure, warn using `references/ethics.md` and still only perform technical cleaning they own. ## Workflow ### 1. Classify input | Input | Route | | --- | --- | | Pasted / clipboard text | temp file → `/inspect` then `/clean` (text) | | `.txt` / code | text Layer A (+ formatter for code) | | `.md` / `.html` / `.tex` / `.ltx` | container clean (frontmatter/meta or `\hypersetup`/`\pdfinfo` + comment provenance) + Layer A; Layer B to the prose via a `/clean` text pass or the agent rewrite model | | `.png` / `.jpg` / `.jpeg` / `.webp` / `.avif` / `.heic` / `.bmp` / `.gif` / `.tiff` | image metadata strip | | `.svg` / `.pdf` / `.docx` / `.epub` / `.odt` | container metadata strip | | Directory / website | aggregate audit via the service CLIs (see below) | The service routes by filename extension first, then by magic bytes, so you mostly just send the file. ### 2. Inspect first ```bash curl -s -X POST "$WM/inspect" -H 'Content-Type: application/json' \ -d "{\"file\": \"$(base64 < path | tr -d '\n')\", \"name\": \"$(basename path)\"}" ``` Show a short summary (suspicious codepoints; C2PA/AI flags; confidence labels `confirmed` / `probable` / `informational` / `likely_false_positive`). Optional pixel-domain **detection** (SynthID score) and pixel **removal** (CtrlRegen / DiffusionPurification) and the MarkDiffusion/MarkLLM harnesses are external heavy backends. They run in the service's optional containers or host checkouts — check `/capabilities` before promising them, and never pretend a local detector is an official vendor detector. ### 2b. Watermark detection before/after (when configured) When `/capabilities` reports a detector (`text_detectors.markllm`) or an image scorer (`scorers.synthid_http` / `scorers.synthid`), measure the result by detecting before and after cleaning: ```bash curl -s -X POST "$WM/detect" -H 'Content-Type: application/json' \ -d '{"file": "'"$(base64 < notes.txt | tr -d '\n')"'", "name": "notes.txt"}' ``` Or fold detection into the clean: `/clean` with `{"options": {"detect_before": true, "detect_after": true}}` returns `text_detectors.before/after` (text) or `synthid_before/synthid_after` (images) in the report. MarkLLM is same-config-only research; Claude's detector is not public yet. (Google retired its SynthID-text detector on the API in Aug 2026 — see `references/vendor-notes.md`.) ### 3. Deterministic clean (always for matching inputs) **Any supported file (unified):** ```bash curl -s -X POST "$WM/clean" -H 'Content-Type: application/json' \ -d "{\"file\": \"$(base64 < INPUT | tr -d '\n')\", \"name\": \"$(basename INPUT)\"}" ``` Decode `cleaned` → `OUTPUT` (`*.cleaned.*` unless the user asked in-place). Re-inspect the result when residual risk matters. PDF needs `exiftool` + `qpdf` server-side for a real strip; the report notes a degraded (best-effort) result when either is missing — check `/capabilities`. **Images — optional pixel removal:** only when `capabilities.pixel_backends` says the backend is present: ```bash curl -s -X POST "$WM/clean" -H 'Content-Type: application/json' \ -d "{\"file\": \"$(base64 < shot.png | tr -d '\n')\", \"name\": \"shot.png\", \ \"options\": {\"remove_pixel\": \"ctrlregen\"}}" ``` ### 4. Layer B — always offer rewrite (prose) After Layer A, **always propose** a statistical-mark reduction pass for natural-language content. Do not skip this step silently. For **plain text** (pasted / `.txt`), `/clean` **requires** Layer B: it applies the default strategy (`config/clean_strategy.json`, e.g. `paraphrase@0.8,mlm@0.2`) or the `options.strategy` override after Layer A, reports `report.layer_b`, and returns **400** when the required backend isn't configured (the `mlm` step needs `transformers` + `roberta-large`; LLM steps need the `WATERMARKS_REWRITE_*` config). Markdown/HTML and other containers (`.md`, `.html`, `.tex`, `.pdf`, `.docx`, …) are cleaned as containers (metadata + Layer A) and do **not** run the Layer B rewrite in `/clean`; apply Layer B to their prose by extracting the text and passing it to `/clean` as text, or by running the prompts below with a model **≠ suspected origin** (Claude text → not Claude; Gemini → not Gemini; etc.). Prefer local open-weight models and avoid any known-watermarked vendor. Multi-pass recipe: 1. Layer A clean (via `/clean`) 2. Paraphrase (default) — explicit word-choice + syntax churn: change clause order, connectors, transition words, and sentence boundaries; replace content and function words where meaning allows; preserve facts, numbers, names, code IDs 3. Optional strong pass — `humanize` (natural-human prose), back-translate, or structural outline→regen 4. Layer A again on the result (`/clean`) 5. Report residual risk honestly (short/highly predictable text = lower; long, high-entropy prose = higher) **Code files:** Prefer formatter (`prettier`, `black`, `gofmt`, …) + Layer A. Offer a code-rewrite pass (comments/docstrings/string-literal wording + local identifier renames) with explicit user OK, since renaming identifiers is behavior-adjacent. #### Rewrite prompts (use as-is) **Paraphrase preserve meaning (word choice + syntax):** ``` Rewrite the following text so that it uses substantially different wording at the token level. Change clause order, connectors, and transition words; vary sentence boundaries and length; and replace both content words and function words where meaning allows. Preserve all facts, numbers, names, and technical identifiers. Do not add or remove claims. Output only the rewritten text. --- {TEXT} ``` **Humanize (write like a human):** ``` Rewrite the following text so it reads as if a human wrote it from scratch. Vary sentence rhythm and length, replace formulaic AI-style transitions and filler with concrete natural phrasing, and use plain, varied wording. Preserve all facts, numbers, names, and technical identifiers. Do not add or remove claims. Output only the rewritten text. --- {TEXT} ```