promptfoo-redteam-setup

name

promptfoo-redteam-setup

description

Create or refine promptfoo redteam setup configs: purpose, targets, plugins, strategies, frameworks, multi-input target inputs, policy text, grader guidance, contexts, and static-code-derived target/threat mapping. Use when preparing a red team scan plan from live probes, code evidence, or provider configs, or when generating adversarial test cases for QA. Do not use for basic provider wiring alone or for running/evaluating an already-generated redteam scan.

Promptfoo Redteam Setup

Build a small, explicit redteam config that matches the real app threat model. Start with a narrow scan that can be generated and inspected, then expand.

Read references/redteam-setup-patterns.md when you need concrete YAML patterns. For OpenAPI specs, you can run the bundled scripts/openapi-operation-to-redteam-config.mjs to draft a one-operation redteam setup config, then inspect the inferred inputs, policy, and plugins. The script ships in this skill's scripts/ directory; when the skill is installed as a plugin it lives in the plugin cache, not your project, so run it by its absolute path (or copy it in) rather than a bare scripts/... path. With --token-env, it infers Bearer/OAuth2/OpenID and header/query/cookie API-key auth; use --auth-header/--auth-prefix to override. For live connectivity QA, add --smoke-test true to include one deterministic tests row that can be run with npm run local -- eval -c ... --no-cache before redteam generation.

Inputs

Infer these from the repo, docs, or user prompt:

• Target shape: HTTP/API, model provider, custom provider, agent, RAG, MCP/tool system, or multi-input app.
• Purpose: who uses the system, what it may access or do, and what it must refuse or protect.
• Trust boundaries: identities, object IDs, documents, tools, secrets, permissions, and external content.