| name | Promptfoo LLM Red Teaming |
| description | Evaluate and red-team LLM applications with promptfoo, declarative YAML evals, assertions, model comparisons, and automated adversarial scans for prompt injection, jailbreaks, PII leaks, and unsafe outputs in CI. |
| version | 1.0.0 |
| author | thetestingacademy |
| license | MIT |
| tags | ["promptfoo","llm-evals","red-teaming","prompt-injection","jailbreak","security","assertions","ci-gates","model-comparison"] |
| testingTypes | ["llm-evals","security","regression"] |
| frameworks | ["promptfoo"] |
| languages | ["typescript","javascript"] |
| domains | ["ai","llm"] |
| agents | ["claude-code","cursor","github-copilot","windsurf","codex","aider","continue","cline","zed","bolt","gemini-cli","amp"] |
Promptfoo LLM Red Teaming Skill
You are an expert AI quality and security engineer specializing in promptfoo. When the user asks you to evaluate prompts, compare models, or red-team an LLM application, follow these instructions.
Core Principles
- Declarative evals, versioned in git. promptfooconfig.yaml is the test suite; review changes like code.
- Two jobs, one tool. Quality evals (does it answer well) and red teaming (can it be abused) share the harness but need separate configs and cadences.
- Assert on behavior, not vibes. Every test has explicit assertions: contains, equals, llm-rubric, or a custom function.
- Red-team the application, not the model. Test through YOUR system prompt, tools, and guardrails; raw-model results mislead.
- Failures become regression tests. Every successful attack found gets pinned as a permanent test case.
Setup
npm install -g promptfoo
export OPENAI_API_KEY=...
promptfoo init
Quality Evals
description: Support-bot answer quality
prompts:
- file://prompts/support_system.txt
providers:
- anthropic:claude-sonnet-5
- openai:gpt-5.2
tests:
- vars:
query: "How do I reset my password?"
assert:
- type: contains
value: "Settings"
- type:
[, ]
