apply-llm-output-sanitization

name	apply-llm-output-sanitization
description	Use when rendering LLM-generated text in a browser, executing LLM-generated code, or inserting LLM output into a database or downstream system — any place LLM output leaves the AI layer and enters another execution context.
source	OWASP Top 10 for LLM Applications 2025 LLM02 (owasp.org/www-project-top-10-for-large-language-model-applications/); OWASP XSS Prevention Cheat Sheet; CWE-79; CWE-94
tags	["security","owasp","llm","output-handling","xss","code-injection","ai-security","emerging"]
emerging	true

Apply LLM Output Sanitization

Treat LLM-generated content as untrusted input to downstream systems — encoding for context before rendering HTML, sandboxing generated code before execution, and validating structure before inserting into databases.

Why This Is Best Practice

Adopted by: OWASP Top 10 for LLM Applications 2025 LLM02 (Insecure Output Handling) is a dedicated category. Microsoft's AI Red Team and Google DeepMind's safety teams publish guidance on output handling. NIST AI RMF Govern 1.3 includes output validation controls. Production AI systems at Anthropic, OpenAI, and major cloud providers all implement output sandboxing for code execution. Status: Emerging — the attack class has been well-documented since 2023, but standardized defense tooling is still maturing. Impact: If an LLM generates HTML containing <script>alert('xss')</script> and it's rendered without escaping, XSS occurs. If an LLM generates shell commands and they're executed without sandboxing, RCE occurs. Bing Chat (2023) was demonstrated generating JavaScript that exfiltrated conversation history. ChatGPT plugins were shown capable of triggering SQL injection through LLM-generated database queries. The LLM is the attack vector; the browser/shell/database is the target. Why best: Trusting LLM outputs because they came from your own model is the common approach — it ignores that LLMs can be manipulated via prompt injection (LLM01) to generate malicious content. Context-aware output encoding (the same defense as for user input XSS) prevents the malicious output from executing, even if generated.

Sources: OWASP LLM Top 10 2025 LLM02; OWASP XSS Prevention Cheat Sheet; CWE-79; Microsoft AI Red Team research

Steps

1. HTML-encode LLM output before rendering in a browser:

   import html
   from markupsafe import Markup, escape
   
   def render_ai_response(llm_output: str) -> str:
       # Escape HTML entities — treat LLM output as user input
       return str(escape(llm_output))
   
   # In templates (Jinja2, Django templates) — always use auto-escaping
   # {{ ai_response }}  ← auto-escaped (safe)
   # {{ ai_response|safe }}  ← NEVER use |safe on LLM output
   

2. For Markdown rendering: sanitize after converting to HTML:

   import DOMPurify from 'dompurify';
   import { marked } from 'marked';
   
   function renderAIMarkdown(markdownOutput) {
     const rawHtml = marked.parse(markdownOutput);
     // Sanitize after markdown conversion — LLM may inject HTML in markdown
     return DOMPurify.sanitize(rawHtml, {
       ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'ul', 'ol', 'li', 'code', 'pre', 'h1', 'h2', 'h3'],
       : [],  
     });
   }
   

Rules

• Every context where LLM output is consumed has its own encoding requirement — the same output needs HTML encoding for browsers, shell escaping for CLI, and parameterization for SQL.
• |safe or innerHTML = applied to LLM output requires DOMPurify sanitization first — no exceptions.
• Generated code execution requires sandboxing even when the prompt explicitly requested "safe" code — the LLM may have been injected.
• Structured output (JSON mode) reduces but does not eliminate the risk — a JSON string value can still contain XSS payloads.

Common Mistakes

• Trusting LLM output because it's your own model — even your model can be manipulated via prompt injection in user-supplied context.
• Rendering LLM markdown as raw HTML without sanitization — markdown [click here](javascript:evil()) becomes an XSS.
• Using eval() for LLM-generated expressions — even "math expressions" can escape the intended scope.
• Sanitizing output once but using it in multiple contexts — output sanitized for HTML may still be unsafe for SQL or shell.