llm-security

# LLM Security Guidelines (OWASP Top 10 for LLM 2025) Security rules for building secure LLM applications, based on the OWASP Top 10 for LLM Applications 2025. ## How to Use This Skill **Proactive mode** — When building or reviewing LLM applications, automatically check for relevant security risks based on the application pattern. You don't need to wait for the user to ask about LLM security. **Reactive mode** — When the user asks about LLM security, use the mapping below to find relevant rule files with detailed vulnerable/secure code examples. ### Workflow 1. Identify what the user is building (see "What Are You Building?" below) 2. Check the priority rules for that pattern 3. Read the specific rule files from `rules/` for code examples 4. Apply the secure patterns or flag vulnerable ones ## What Are You Building? Use this to quickly identify which rules matter most for the user's task: | Building... | Priority Rules | |-------------|---------------| | **Chatbot / conversational AI** | Prompt Injection (LLM01), System Prompt Leakage (LLM07), Output Handling (LLM05), Unbounded Consumption (LLM10) | | **RAG system** | Vector/Embedding Weaknesses (LLM08), Prompt Injection (LLM01), Sensitive Disclosure (LLM02), Misinformation (LLM09) | | **AI agent with tools** | Excessive Agency (LLM06), Prompt Injection (LLM01), Output Handling (LLM05), Sensitive Disclosure (LLM02) | | **Fine-tuning / training** | Data Poisoning (LLM04), Supply Chain (LLM03), Sensitive Disclosure (LLM02) | | **LLM-powered API** | Unbounded Consumption (LLM10), Prompt Injection (LLM01), Output Handling (LLM05), Sensitive Disclosure (LLM02) | | **Content generation** | Misinformation (LLM09), Output Handling (LLM05), Prompt Injection (LLM01) | ## Categories ### Critical Impact - **LLM01: Prompt Injection** (`rules/prompt-injection.md`) - Prevent direct and indirect prompt manipulation - **LLM02: Sensitive Information Disclosure** (`rules/sensitive-disclosure.md`) - Protect PII, credentials, and proprietary data - **LLM03: Supply Chain** (`rules/supply-chain.md`) - Secure model sources, training data, and dependencies - **LLM04: Data and Model Poisoning** (`rules/data-poisoning.md`) - Prevent training data manipulation and backdoors - **LLM05: Improper Output Handling** (`rules/output-handling.md`) - Sanitize LLM outputs before downstream use ### High Impact - **LLM06: Excessive Agency** (`rules/excessive-agency.md`) - Limit LLM permissions, functionality, and autonomy - **LLM07: System Prompt Leakage** (`rules/system-prompt-leakage.md`) - Protect system prompts from disclosure - **LLM08: Vector and Embedding Weaknesses** (`rules/vector-embedding.md`) - Secure RAG systems and embeddings - **LLM09: Misinformation** (`rules/misinformation.md`) - Mitigate hallucinations and false outputs - **LLM10: Unbounded Consumption** (`rules/unbounded-consumption.md`) - Prevent DoS, cost attacks, and model theft See `rules/_sections.md` for the full index with OWASP/MITRE references. ## Quick Reference | Vulnerability | Key Prevention | |--------------|----------------| | Prompt Injection | Input validation, output filtering, privilege separation | | Sensitive Disclosure | Data sanitization, access controls, encryption | | Supply Chain | Verify models, SBOM, trusted sources only | | Data Poisoning | Data validation, anomaly detection, sandboxing | | Output Handling | Treat LLM as untrusted, encode outputs, parameterize queries | | Excessive Agency | Least privilege, human-in-the-loop, minimize extensions | | System Prompt Leakage | No secrets in prompts, external guardrails | | Vector/Embedding | Access controls, data validation, monitoring | | Misinformation | RAG, fine-tuning, human oversight, cross-verification | | Unbounded Consumption | Rate limiting, input validation, resource monitoring | ## Key Principles 1. **Never trust LLM output** - Validate and sanitize all outputs before use 2. **Least privilege** - Grant minimum necessary permissions to LLM systems 3. **Defense in depth** - Layer multiple security controls 4. **Human oversight** - Require approval for high-impact actions 5. **Monitor and log** - Track all LLM interactions for anomaly detection ## References - [OWASP Top 10 for LLM Applications 2025](https://genai.owasp.org/llm-top-10/) - [MITRE ATLAS - Adversarial Threat Landscape for AI Systems](https://atlas.mitre.org/) - [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework)