メニュー
Launch a Coval evaluation run against an AI agent. Use when user wants to start an evaluation, test an agent, or run simulations.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
End-to-end Coval adversarial / red-team testing workflow. Builds one adversarial test set (~10 bad-actor scenarios, each with an expected-behavior checklist), creates a persistent "Adversarial User" persona and a Composite Evaluation metric that scores each scenario against its own expected behaviors, launches a multi-iteration run against the agent (voice or chat), polls for completion, builds a per-scenario pass/fail scorecard, and creates a saved report grouped by Test Case. Use when a user wants to follow the Adversarial & Red-Team Testing cookbook (https://docs.coval.dev/guides/adversarial-red-team-testing) without doing each step by hand. Triggers: "adversarial test set", "red team my agent", "jailbreak / prompt-injection testing", "test my agent against bad actors".
Analyze a Coval adversarial / red-team testing report and turn it into an agent-hardening plan. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or a per-scenario scorecard from an adversarial sweep and wants evidence-backed next steps such as prompt/guardrail changes, refusal hardening, verification fixes, escalation routing, or expanded attack coverage.
Derive a SET of simulation personas for an agent from product artifacts — backend payloads, UI screenshots, journey/product docs, and sample real user messages — instead of designing one persona by hand. Identifies who actually interacts with the agent and how they behave, then creates the personas via the CLI. Best for text/chat agents and for new agents with no interaction history. Use when the user says "make personas from these screenshots/payloads", "who are my users", "create a set of personas", "derive personas from my product", "build a persona library", or "I have backend data, turn it into personas".
Turn a large dataset (an existing oversized Coval test set, an export of past conversations, or a CSV/JSON of cases) into a small, high-signal Coval test set by removing duplicates, identifying unique scenarios, and selecting a representative, failure-weighted subset — then bulk-loading it with no row cap. Use when the user says "I have thousands of cases", "dedupe my test set", "my test set is too big", "turn this dataset into a test set", "pick representative scenarios", or "my CSV import only kept 10 / uploaded everything".
Analyze a Coval accent testing report from runs across different speaker accents. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or metric summary and wants evidence-backed next steps such as prompt changes, STT/confirmation adjustments, accent-robust routing, or expanded accent coverage.
End-to-end Coval accent testing workflow. Creates one persona per accent (each using a distinct accent voice and mirroring your Standard Customer behavior), launches one run per accent against the same voice agent + test set + metrics, polls for completion, builds a per-persona comparison table from the results, and creates the saved multi-run report (grouped by Persona) via the public API. Use when a user wants to follow the Testing Across Accents cookbook (https://docs.coval.dev/guides/testing-across-accents) without doing each step by hand.
| name | launch-run |
| description | Launch a Coval evaluation run against an AI agent. Use when user wants to start an evaluation, test an agent, or run simulations. |
| argument-hint | [agent-name-or-id] [test-set-name-or-id] |
Launch an evaluation run for $ARGUMENTS.
Ensure the Coval CLI is installed and authenticated:
coval whoami
If not authenticated, run coval login first.
If agent or test set not specified, list available options:
coval agents list
coval test-sets list
coval personas list
Ask user to select:
Ask about optional parameters:
| Option | Flag | Default |
|---|---|---|
| Iterations per test case | --iterations | 1 |
| Concurrent simulations | --concurrency | 5 |
| Run name | --name | Auto-generated |
| Mutation (A/B variant) | --mutation-id | None |
coval runs launch \
--agent-id <agent_id> \
--persona-id <persona_id> \
--test-set-id <test_set_id> \
--iterations <n> \
--concurrency <n> \
--name "Descriptive Run Name"
Report the run ID and status. Offer to watch progress:
Run launched:
<run_id>Status: IN QUEUEWould you like me to watch the progress?
If yes, use coval runs watch <run_id>.
coval runs launch \
--agent-id abc123 \
--persona-id xyz789 \
--test-set-id ts456 \
--iterations 3 \
--concurrency 10 \
--name "Q1 Regression Test"