llm-redteam

# LLM Red-Team > Reflection is not exploitation. A model that *echoes* your payload is noise; a model that *acts* on it — leaks its system prompt, emits your canary, calls a tool, renders raw HTML downstream — is a bug. Prove the action, then chain it to concrete impact. This is the first-class LLM-attack skill. The scanner behind it is `bughunter/tools/llm_redteam.py` (canary-based corpus runner, `/llm-redteam`). For auditing an MCP *server*, use `skills/mcp-server-audit`; for attacking a *deployed agent* black-box, use `skills/agentic-app-audit`. ## 0. QUICK KILL CHECKLIST Kill the lead (classify Informational, move on) when: - The model refuses and no canary / leak / tool-call ever lands. "It said something edgy" is not a finding. - System-prompt "leak" is just generic assistant boilerplate ("I am a helpful assistant"), not the target's actual instructions. - Prompt injection fires but reads/changes nothing the current user couldn't already access (no cross-tenant data, no tool with impact). - Output looks like XSS but the host app escapes it before rendering (check the actual downstream sink, not the chat response). - Exfil payload is a markdown image beacon but the client never fetches remote images. ## 1. ROUTING TABLE — signal on target -> what to run | Signal | Run | |---|---| | Any chat/assistant endpoint | `bughunter/tools/llm_redteam.py --url <endpoint> --field <json-field>` (full corpus) | | OpenAI-style API | `--template '{"messages":[{"role":"user","content":"{{PAYLOAD}}"}]}' --response-path choices.0.message.content` | | Only want one class | `--category jailbreak` (see `--list-categories`) | | Uploads/RAG/"summarize this doc" | indirect-injection + token-smuggling (invisible unicode in the doc) | | Agent has tools | excessive-agency category + `skills/agentic-app-audit` | | Confirm blind exfil | plant a canary host, correlate with `bughunter/tools/oob_listener.py` | ## 2. ATTACK CLASSES (corpus categories) The runner fires a canary per run (`RT_PWNED_xxxx`); a category "lands" when the canary / a real leak / a tool-call shows up in the response. - **prompt-injection** — direct instruction override, delimiter breaks. - **jailbreak** — DAN / developer-mode persona escape. - **system-prompt-leak** — repeat-above, keyword-anchor, scenario-escape. - **data-exfil** — markdown-image beacon / canary emission to an attacker host. - **indirect-injection** — payload framed as a retrieved document / tool result (the high-value one for RAG apps). - **guardrail-bypass** — base64 / split-string instruction smuggling. - **multi-turn-crescendo** — single-shot simulation of a gradual escalation jailbreak. - **cross-lingual** — override instruction in another language to dodge English-only filters. - **cipher-obfuscation** — leetspeak / ROT13-wrapped instruction. - **token-smuggling** — invisible "Sneaky Bits" unicode (U+2062/U+2064) carrying a hidden instruction (shared encoder with `bughunter/tools/hai_payload_builder.py`). - **excessive-agency** — coerce an unsafe tool call / exfil (SSRF via "fetch URL", email send, payment). - **insecure-output** — get the model to emit raw HTML/script that a downstream sink renders (XSS via AI). ## 3. CANONICAL OWASP ASI01-ASI10 (single source of truth) This table is the ONE authoritative ASI mapping for the whole toolkit — `skills/bug-bounty` and `skills/web2-vuln-classes` both defer to it. If you edit the taxonomy, edit it here. | ID | Class | What to test | Chain to | |----|-------|--------------|----------| | ASI01 | Prompt Injection / Goal Hijack | Override objectives via direct or indirect injection | IDOR / exfil / tool abuse | | ASI02 | Tool Misuse | Attacker-controlled tool params ("fetch this URL", code tool) | SSRF / RCE | | ASI03 | Privilege Compromise | Agent uses broader perms / admin tokens than the user | Priv-esc / cross-tenant | | ASI04 | Supply Chain | Compromised plugin / MCP server / tool-output poisoning next agent | RCE / data theft | | ASI05 | Code Execution | Unsafe code-interpreter / sandbox escape | RCE | | ASI06 | Memory & Context Poisoning | Persistent RAG/memory corruption across sessions/users | Stored injection affecting all users | | ASI07 | Agent Communication | Inter-agent spoofing / IDOR (agent A reads agent B's context) | Cross-tenant disclosure | | ASI08 | Excessive Agency | Destructive action without confirmation; cascading failures | Funds/email/delete | | ASI09 | Insecure Output Handling | AI output rendered as XSS / SQLi / command injection downstream | XSS / injection | | ASI10 | Sensitive Information Disclosure | Leaks system prompt / keys / configs / user data | Secrets -> deeper access | **Triage rule:** ASI alone = Informational. It is a bounty only when chained to IDOR / exfil / RCE / ATO with a working PoC. ## 4. CHAIN TO IMPACT (what makes it payable) | Standalone | Chain | Result | |---|---|---| | Prompt injection | + reads another user's conversation/data (IDOR) | High | | Indirect injection (RAG doc) | + persists in memory -> hits every user | High/Critical | | Tool misuse "fetch URL" | + hits internal service / IMDS and returns data | SSRF (Medium/High) | | Insecure output | + the host renders it -> stored XSS -> session theft | High | | System-prompt leak | + the prompt contains an API key / internal URL | High | | Excessive agency | + unconfirmed destructive action (send funds, delete) | High/Critical | ## 5. CONFIRMATION DISCIPLINE (no false positives) - A category only counts when the **canary / real leak / tool-call** is observed in the response — never on "it sounded compliant." - For blind exfil, require a correlated out-of-band callback (`bughunter/tools/oob_listener.py`), not just a rendered beacon tag. - For insecure-output XSS, confirm execution at the **downstream sink** with the XSS verifier (`tools/verifiers/xss.py` / `bughunter/tools/dom_xss_harness.py`), not in the chat reply. - Record the exact payload + response excerpt; the report must be reproducible verbatim.