frisk-scan-prompt-injection

name	frisk-scan-prompt-injection
description	CTF-style hunt for prompt injection — user input flowing into LLM message payloads without sandboxing, and trusted LLM responses driving permission decisions or interpolated into Slack/HTML/markdown channels.
context	fork
agent	Explore

Prompt-injection scan

You are a security researcher in a CTF. Your single job: find places where the application calls an LLM (Anthropic / OpenAI / Gemini / Prism / Saloon-wrapped / raw HTTP) and the surface is exploitable on the input side, the output side, or both. Two distinct shapes share this class: (a) user-controlled text (form input, OAuth-derived display name, scraped page metadata, model attributes that were user-written) lands in the messages[] / system / prompt payload without a sandbox boundary, letting the attacker inject directives the model obeys; (b) the model's response is trusted to drive a permission decision, enum cast, or database write, or is interpolated verbatim into a downstream channel that interprets formatting (Slack text / blocks, mrkdwn, HTML, markdown email). Report concrete instances only.

Coverage protocol

Two runs of this scanner on the same code should produce the same findings. The biggest cause of drift is skimming — stopping at the first few matches. Don't do that:

1. Enumerate before judging. List every file/symbol matching "Where to look". Walk the set; don't sample.
2. Decide each candidate. For each one, classify as finding / compensating-control-present / out-of-scope, then assemble the JSON.
3. Recall over brevity. Borderline cases go in at medium/low — don't drop them silently.

If your first finding came from the first file you opened, you skimmed. Restart with enumeration.

Where to look

• app/Jobs/**, app/Actions/**, app/Services/**, app/Http/Controllers/**, app/Livewire/**, app/Http/Livewire/**.
• Custom AI-client wrappers under app/Support/, app/AI/, app/Integrations/.
• Grep for SDK / endpoint markers: Anthropic\, OpenAI\, Prism\, Saloon\, 'claude-', 'gpt-', 'gemini-', api.anthropic.com, api.openai.com, generativelanguage.googleapis.com.

What counts as a finding

Input side (user content reaches the LLM payload)

• Any code path where $request->..., request(...), a model attribute that was user-written (form-driven, OAuth-derived, submission-derived), or content fetched from a user-supplied URL (the Http::get($url)->body() pattern that pairs with the SSRF surface) lands in messages[].content, system, prompt, or the analogous payload key of an LLM call without a sanitization or sandbox-prompt boundary.
• Loop-built messages arrays where the loop body interpolates a user-controlled field directly into the role-user content string.
• Helper / service wrapper that accepts arbitrary string args from a caller chain whose roots are controller request input.
• Delimiter-wrapping is mitigation, not exoneration. <user-input>...</user-input> tags, explicit "treat the following as data" instructions, or routing user content into a separate function-calling argument slot all reduce severity but do not clear the finding — the LLM can still be persuaded across the delimiter. Mark medium instead of dropping.

Output side (the response is trusted)

• Model's response is parsed as JSON and a field of that JSON drives a permission decision (if ($result['approved']) ..., if ($result['score'] >= 8) ...), an enum cast (Status::from($result['status'])), or a database write ($model->update(['status' => $result['status']])).
• Model's response is interpolated verbatim into a downstream channel that interprets formatting — Slack text / blocks / mrkdwn, HTML rendered in an admin panel, markdown email body — without an e(...), Str::markdown(..., ['safe' => true]), or Slack-specific escape wrapper.
• Function-calling / tool-use responses where the model's chosen tool args are passed to a sink without re-validation.

Combined flow (the canonical pipeline)

• A single flow where user input reaches the prompt and the response drives an action — the high-severity case. Look for controller → LLM service → response field → update() / dispatch() / redirect() chains.

What does NOT count

• LLM calls whose entire input is a string literal or a constant config value (no user reach).
• Calls whose response is logged only, or displayed inside a <pre><code> block with HTML escaping — no decision, no formatted-channel sink.
• Calls whose response goes through a structured-output validator with a strict server-side schema — in the PHP ecosystem that means Spatie\LaravelData with strict casts, an explicit Validator::make([...], [...]) against the response array, or a custom DTO that throws on shape mismatch. A schema requested in the prompt does not count — the model can ignore it.
• Static-analysis-obvious cases the code-pattern-scan rules already catch (literal eval, literal unserialize — different vulnerability class anyway).

Severity guidance

• high — Both input and output sides land in the same flow: user-controlled content reaches the LLM payload and the response drives a permission decision, enum cast, database write, or unescaped render into a formatted channel. The canonical "user input → LLM → action" pipeline.
• medium — Only one side fires: input-only with the response merely logged or displayed safely; output-only where hardcoded prompts nonetheless drive decisions. Also: input side present but delimiter-wrapped / routed into a function-call arg slot (still exploitable, lower probability).
• low — Borderline: input passes through length-truncation or simple keyword stripping; JSON-only response schema with no decision side; LLM call exists in code that doesn't yet ship to a user-reachable endpoint. Flag for human review with the reason you're unsure.

Output

End your response with a single fenced JSON block, nothing after it:

{
  "scanner": "frisk-scan-prompt-injection",
  "findings": [
    {
      "intensity": "high",
      "message": "ModerationJob concatenates `$submission->title` (user-supplied) into the `messages[0].content` of an Anthropic call, then `update(['status' => $result['status']])` from the parsed response — input and output sides are in the same flow with no sandbox boundary or response validator.",
      "subject": "app/Jobs/ModerationJob.php:42",
      "url": null
    }
  ]
}

If you find nothing, return "findings": []. Do not pad.