メニュー
Use when probing AI agents — reads eval config YAML, orchestrates browser interaction, verification, and report generation for web-based AI agent evaluation
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Interact with Langfuse and access its documentation. Use when needing to (1) query or modify Langfuse data programmatically via the CLI — traces, prompts, datasets, scores, sessions, and any other API resource, (2) look up Langfuse documentation, concepts, integration guides, or SDK usage, or (3) understand how any Langfuse feature works. This skill covers CLI-based API access (via npx) and multiple documentation retrieval methods.
Use when generating HTML evaluation reports for AI agent test suites — self-contained HTML with embedded screenshots, pass/fail results grouped by functional and compliance
Use when translating natural language eval instructions into playwright-cli commands during AI agent evaluation — clicking, typing, waiting, scrolling, screenshot capture
Use when verifying AI agent responses during evaluation — content matching, behavioral assertions, AI-judged quality, and compliance checks with optional multi-step verification flows
| name | agent-eval |
| description | Use when probing AI agents — reads eval config YAML, orchestrates browser interaction, verification, and report generation for web-based AI agent evaluation |
Probes an AI agent end-to-end from a YAML config. Drives a real browser, interacts like a user, judges the output across multiple dimensions, and generates an HTML report with screenshots and optional video.
playwright-cli installed (playwright-cli --version)suite: "Suite Name"
target:
url: "https://app.example.com"
credentials: # secrets — use {{env:VAR}} to pull from .env
email: "{{env:AGENT_TARGET_EMAIL}}"
password: "{{env:AGENT_TARGET_PASSWORD}}"
auth: # login steps — run once per suite
- "Type '{{email}}' in the email field"
- "Type '{{password}}' in the password field"
setup: # fresh conversation — run before each scenario
- "Start a new conversation"
scenarios:
- name: "Scenario Name"
steps:
- "Natural language action steps"
verify:
- type: ai-judge
check: "Natural language assertion"
Substitution: {{env:VAR}} resolves from environment. {{key}} resolves from credentials.
Read the YAML. Resolve credentials. Classify each scenario:
The agent decides — no YAML flag needed. Look at the steps: if they reference "approve", "fill form", "click Next", or depend on previous output → sequential.
Log in once at the start. Save browser state for parallel sessions to reuse.
For each scenario:
Generate HTML report with:
Open report automatically when done.