Menú
Use when probing AI agents — reads eval config YAML, orchestrates browser interaction, verification, and report generation for web-based AI agent evaluation
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Interact with Langfuse and access its documentation. Use when needing to (1) query or modify Langfuse data programmatically via the CLI — traces, prompts, datasets, scores, sessions, and any other API resource, (2) look up Langfuse documentation, concepts, integration guides, or SDK usage, or (3) understand how any Langfuse feature works. This skill covers CLI-based API access (via npx) and multiple documentation retrieval methods.
Use when generating HTML evaluation reports for AI agent test suites — self-contained HTML with embedded screenshots, pass/fail results grouped by functional and compliance
Use when translating natural language eval instructions into playwright-cli commands during AI agent evaluation — clicking, typing, waiting, scrolling, screenshot capture
Use when verifying AI agent responses during evaluation — content matching, behavioral assertions, AI-judged quality, and compliance checks with optional multi-step verification flows
Basado en la clasificación ocupacional SOC
| name | agent-eval |
| description | Use when probing AI agents — reads eval config YAML, orchestrates browser interaction, verification, and report generation for web-based AI agent evaluation |
Probes an AI agent end-to-end from a YAML config. Drives a real browser, interacts like a user, judges the output across multiple dimensions, and generates an HTML report with screenshots and optional video.
playwright-cli installed (playwright-cli --version)suite: "Suite Name"
target:
url: "https://app.example.com"
credentials: # secrets — use {{env:VAR}} to pull from .env
email: "{{env:AGENT_TARGET_EMAIL}}"
password: "{{env:AGENT_TARGET_PASSWORD}}"
auth: # login steps — run once per suite
- "Type '{{email}}' in the email field"
- "Type '{{password}}' in the password field"
setup: # fresh conversation — run before each scenario
- "Start a new conversation"
scenarios:
- name: "Scenario Name"
steps:
- "Natural language action steps"
verify:
- type: ai-judge
check: "Natural language assertion"
Substitution: {{env:VAR}} resolves from environment. {{key}} resolves from credentials.
Read the YAML. Resolve credentials. Classify each scenario:
The agent decides — no YAML flag needed. Look at the steps: if they reference "approve", "fill form", "click Next", or depend on previous output → sequential.
Log in once at the start. Save browser state for parallel sessions to reuse.
For each scenario:
Generate HTML report with:
Open report automatically when done.