Menu
Use when probing AI agents โ reads eval config YAML, orchestrates browser interaction, verification, and report generation for web-based AI agent evaluation
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Interact with Langfuse and access its documentation. Use when needing to (1) query or modify Langfuse data programmatically via the CLI โ traces, prompts, datasets, scores, sessions, and any other API resource, (2) look up Langfuse documentation, concepts, integration guides, or SDK usage, or (3) understand how any Langfuse feature works. This skill covers CLI-based API access (via npx) and multiple documentation retrieval methods.
Use when generating HTML evaluation reports for AI agent test suites โ self-contained HTML with embedded screenshots, pass/fail results grouped by functional and compliance
Use when translating natural language eval instructions into playwright-cli commands during AI agent evaluation โ clicking, typing, waiting, scrolling, screenshot capture
Use when verifying AI agent responses during evaluation โ content matching, behavioral assertions, AI-judged quality, and compliance checks with optional multi-step verification flows
Based on SOC occupation classification
| name | agent-eval |
| description | Use when probing AI agents โ reads eval config YAML, orchestrates browser interaction, verification, and report generation for web-based AI agent evaluation |
Probes an AI agent end-to-end from a YAML config. Drives a real browser, interacts like a user, judges the output across multiple dimensions, and generates an HTML report with screenshots and optional video.
playwright-cli installed (playwright-cli --version)suite: "Suite Name"
target:
url: "https://app.example.com"
credentials: # secrets โ use {{env:VAR}} to pull from .env
email: "{{env:AGENT_TARGET_EMAIL}}"
password: "{{env:AGENT_TARGET_PASSWORD}}"
auth: # login steps โ run once per suite
- "Type '{{email}}' in the email field"
- "Type '{{password}}' in the password field"
setup: # fresh conversation โ run before each scenario
- "Start a new conversation"
scenarios:
- name: "Scenario Name"
steps:
- "Natural language action steps"
verify:
- type: ai-judge
check: "Natural language assertion"
Substitution: {{env:VAR}} resolves from environment. {{key}} resolves from credentials.
Read the YAML. Resolve credentials. Classify each scenario:
The agent decides โ no YAML flag needed. Look at the steps: if they reference "approve", "fill form", "click Next", or depend on previous output โ sequential.
Log in once at the start. Save browser state for parallel sessions to reuse.
For each scenario:
Generate HTML report with:
Open report automatically when done.