| name | promptfoo-config |
| description | promptfoo configuration patterns for prompt regression testing, multi-provider comparison, and assertion-based validation. Use when setting up prompt testing, comparing LLM providers, or creating eval pipelines. |
| allowed-tools | Bash, Read, Write, Edit, Grep, Glob, WebFetch |
promptfoo Configuration
Skill for configuring promptfoo prompt evaluation and testing.
Overview
promptfoo enables:
- Prompt regression testing
- Multi-provider comparison
- Assertion-based validation
- HTML report generation
Use When
This skill is automatically invoked when:
- Setting up prompt testing
- Comparing LLM providers
- Creating evaluation pipelines
- Defining test assertions
Available Scripts
| Script | Description |
|---|
scripts/init-promptfoo.sh | Initialize promptfoo project |
scripts/run-eval.sh | Run evaluation with output |
Available Templates
| Template | Description |
|---|
templates/promptfooconfig.yaml | Basic configuration |
templates/multi-provider.yaml | Provider comparison |
templates/assertions.yaml | Assertion examples |
Best Practices
- Use YAML anchors for repeated config
- Store prompts in separate files
- Use variables for dynamic tests
- Set appropriate thresholds
