prompt-injection-denylist

# Prompt Injection Denylist Analysis Detect fraud groups that forward traffic from third-party AI tools through Warp's agent mode by pasting full system prompts as their first message. ## Schema config Table/project/database names are **logical** placeholders (`{{ fq.* }}`) resolved from `config/schema.json` (see `SCHEMA.md`). Run the SQL via `api_client.py query --database ugc --file <path>` (or `--sql`) to resolve them, or substitute physical names yourself. The default mapping uses project `YOUR_GCP_PROJECT` and tables `agent_mode_ugc`, `server_events`, `core_user_facts`. ## Background ### How It Works Fraudsters use Warp as a free LLM backend by: 1. Creating throwaway accounts on disposable email domains (aol, yahoo, icloud, mail.ru, protonmail, hotmail, zohomail) 2. Sending requests to Warp's agent mode where the "user query" is actually a full system prompt from another AI tool 3. The LLM enters infinite loops calling tools that don't exist in Warp (e.g., `Bash`, `Read`, `exec`), burning credits ### Known Fraud Groups (as of Feb 2026) **OpenClaw** — An open-source AI assistant framework. Fraud pattern: - First message is the full OpenClaw system prompt, including `AGENTS.md`, `SOUL.md`, `USER.md`, `HEARTBEAT.md`, skill definitions, and tool instructions - Contains distinctive opener: `"You are a personal assistant running inside OpenClaw"` - Includes `<available_skills>` XML blocks and `openclaw gateway` CLI references - Users build crypto/DeFi bots, stock trading assistants, digital twin systems **Claude Code Spoof** — Users spoofing Anthropic's Claude Code CLI. Fraud pattern: - Prefixes message with fake billing header: `x-anthropic-billing-header: cc_version=...; cc_entrypoint=cli; cch=00000;` - Pastes the full Claude Code system prompt after the header - Distinctive opener: `"You are Claude Code, Anthropic's official CLI for Claude"` **Shared injection wrapper** (`<language_requirement>` ring) — Both groups (and others) wrap their prompts with a common Chinese-language injection framework: - `<language_requirement>` blocks specifying response language - `<tool_instructions>` blocks with Chinese batch-operation rules - Contains a distinctive all-caps "batch operations" banner line - Contains Chinese-language batch-read / tool-usage instruction phrases ### Enforcement Mechanism Denylist strings live in your server configuration (e.g. a `user_query_denylist_strings` list) and are evaluated by the reputation/enforcement layer as a substring match against the user's query. When a query matches and enforcement is enabled, the request is handled by the server-side reputation system. Keep the exact matching semantics, enforcement behavior, exemptions, and code locations in your private server repository — do not document them in this open-source skill. ## Detection Workflow ### Step 1: Identify Suspicious Prompt Patterns Look for high-volume identical first messages in the UGC table. Fraud prompts are long system prompts, not natural user queries. ```sql -- Find suspiciously long first messages with repeated content SELECT LEFT(query, 200) AS query_prefix, COUNT(*) AS occurrences, COUNT(DISTINCT user_id) AS distinct_users, ROUND(SUM(total_cost_dollars), 2) AS total_cost FROM ( SELECT user_id, total_cost_dollars, JSON_VALUE(request, '$.input.userInputs.inputs[0].userQuery.query') AS query FROM {{ fq.agent_mode_ugc }} WHERE event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) AND is_new_conversation = true ) WHERE LENGTH(query) > 2000 -- system prompts are very long GROUP BY query_prefix HAVING occurrences > 10 ORDER BY total_cost DESC ``` ### Step 2: Extract Full Examples Once you spot a pattern, pull full message examples to identify the exact system prompt being injected: ```sql SELECT user_id, conversation_id, email_domain, days_since_signup, LEFT(query, 5000) AS full_query FROM ( SELECT user_id, conversation_id, email_domain, days_since_signup, JSON_VALUE(request, '$.input.userInputs.inputs[0].userQuery.query') AS query FROM {{ fq.agent_mode_ugc }} WHERE event_date = CURRENT_DATE() AND is_new_conversation = true ) WHERE query LIKE '%<suspected_pattern>%' LIMIT 4 ``` Look for: - Verbatim system prompt openers (e.g., "You are a personal assistant running inside...") - XML/structured blocks (`<tool_instructions>`, `<available_skills>`, `<language_requirement>`) - Fake billing/auth headers - References to tools that don't exist in Warp (Bash, Read, Write, exec) - Chinese-language instruction blocks wrapping the system prompt ### Step 3: Check JA4 TLS Fingerprints Fraud rings often have distinct TLS fingerprints. Check via server events: ```sql SELECT json_extract_scalar( to_json_string(event_properties), '$.payload.fraud_signals.request_headers.X-JA4-Fingerprint' ) AS ja4, COUNT(DISTINCT json_extract_scalar(to_json_string(event_properties), '$.payload.user_id')) AS distinct_users, COUNT(*) AS requests FROM {{ fq.server_events }} WHERE event_name = 'agentmode.request' AND event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) AND -- join to suspected fraud user list GROUP BY ja4 ORDER BY distinct_users DESC ``` Note: Distributed fraud rings (like Claude Code Spoof) may have 90+ distinct JA4s, making JA4 blocking impractical. In that case, content-based denylist strings are the better approach. ### Step 4: Propose Denylist Strings Choose substrings that are: 1. **Unique to the injected system prompt** — not something a legitimate user would type 2. **Exact system prompt openers** — e.g., "You are a personal assistant running inside OpenClaw" is safe, but "openclaw" alone would false-positive on users asking "how do I set up openclaw" 3. **Case-sensitive** — the denylist uses case-sensitive substring matching **Good candidates:** - Exact system prompt opening sentences - Fake billing/auth header prefixes - XML block markers unique to the injection framework - CLI reference strings embedded in the system prompt **Bad candidates (too broad):** - Single product names (e.g., "openclaw", "claude code") — legit users may mention these - Common programming terms that appear in the system prompt - Short strings that could appear in normal code ### Step 5: Backtest for False Positives **Critical step.** Before adding any denylist string, verify zero paid users would be caught. Join on `core_user_facts` for reliable paid status (don't trust `plan_type` from UGC table): ```sql SELECT match_type, COALESCE(cuf.active_subscription_plan_type, 'none') AS paid_status, cuf.is_on_team_with_active_subscription, COUNT(DISTINCT ugc.user_id) AS distinct_users, COUNT(*) AS total_requests, ROUND(SUM(ugc.total_cost_dollars), 2) AS total_cost FROM ( SELECT user_id, total_cost_dollars, CASE WHEN query LIKE '%<candidate_string_1>%' THEN 'pattern_1' WHEN query LIKE '%<candidate_string_2>%' THEN 'pattern_2' END AS match_type FROM ( SELECT user_id, total_cost_dollars, JSON_VALUE(request, '$.input.userInputs.inputs[0].userQuery.query') AS query FROM {{ fq.agent_mode_ugc }} WHERE event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY) ) WHERE query LIKE '%<candidate_string_1>%' OR query LIKE '%<candidate_string_2>%' ) ugc LEFT JOIN {{ fq.core_user_facts }} cuf ON ugc.user_id = cuf.user_id GROUP BY match_type, paid_status, is_on_team_with_active_subscription ORDER BY match_type, paid_status ``` **Requirements to proceed:** - Zero paid users (`active_subscription_plan_type` is not null/none) - Zero users on a team with active subscription - All matches come from disposable email domains and recent signups ### Step 6: Monitor with Metabase Create a day-by-day cost chart to track each fraud group's impact over time: ```sql SELECT event_date, CASE WHEN query LIKE '%<denylist_string_1>%' THEN 'Group 1' WHEN query LIKE '%<denylist_string_2>%' THEN 'Group 2' END AS fraud_group, ROUND(SUM(total_cost_dollars), 2) AS daily_cost, COUNT(*) AS requests, COUNT(DISTINCT user_id) AS distinct_users FROM ( SELECT user_id, event_date, total_cost_dollars, JSON_VALUE(request, '$.input.userInputs.inputs[0].userQuery.query') AS query FROM {{ fq.agent_mode_ugc }} WHERE event_date BETWEEN {{start_date}} AND {{end_date}} ) WHERE <denylist conditions> GROUP BY event_date, fraud_group ORDER BY event_date, fraud_group ``` Use Metabase template tags (`{{start_date}}`, `{{end_date}}`) for date parameters. Display as line chart, X-axis = event_date, Y-axis = daily_cost, series = fraud_group. Create a Metabase question in your fraud collection for ongoing monitoring. ## Managing Denylist Strings The live denylist strings are maintained in your private server configuration and are intentionally **not** listed here: publishing exact strings would let abusers trivially edit their prompts to evade them. Record new strings in your internal config/change log rather than in this open-source skill. When you add a string, backtest it over a recent window and confirm zero paid users are affected before enabling it (see Step 5). ## Important Notes - **Denylist is a content-based approach** — it catches the prompt regardless of IP, device, or JA4 fingerprint. This is important because distributed fraud rings rotate all of those signals. - **Enforcement is handled server-side** — how matched requests are treated, and any exemptions (e.g. paying customers), live in your private server code. Keep those specifics out of this public doc. - **New fraud tools will appear** — When you see a new cost spike, check if the queries contain a new third-party system prompt being injected. Follow this same workflow to identify, backtest, and add new denylist strings. - **Confirm match semantics** — Check how your denylist matches (e.g. case sensitivity) so proposed strings match the fraud messages exactly.