메뉴
Launch a Coval evaluation run against an AI agent. Use when user wants to start an evaluation, test an agent, or run simulations.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
End-to-end Coval adversarial / red-team testing workflow. Builds one adversarial test set (~10 bad-actor scenarios, each with an expected-behavior checklist), creates a persistent "Adversarial User" persona and a Composite Evaluation metric that scores each scenario against its own expected behaviors, launches a multi-iteration run against the agent (voice or chat), polls for completion, builds a per-scenario pass/fail scorecard, and creates a saved report grouped by Test Case. Use when a user wants to follow the Adversarial & Red-Team Testing cookbook (https://docs.coval.dev/guides/adversarial-red-team-testing) without doing each step by hand. Triggers: "adversarial test set", "red team my agent", "jailbreak / prompt-injection testing", "test my agent against bad actors".
Analyze a Coval adversarial / red-team testing report and turn it into an agent-hardening plan. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or a per-scenario scorecard from an adversarial sweep and wants evidence-backed next steps such as prompt/guardrail changes, refusal hardening, verification fixes, escalation routing, or expanded attack coverage.
Derive a SET of simulation personas for an agent from product artifacts — backend payloads, UI screenshots, journey/product docs, and sample real user messages — instead of designing one persona by hand. Identifies who actually interacts with the agent and how they behave, then creates the personas via the CLI. Best for text/chat agents and for new agents with no interaction history. Use when the user says "make personas from these screenshots/payloads", "who are my users", "create a set of personas", "derive personas from my product", "build a persona library", or "I have backend data, turn it into personas".
Turn a large dataset (an existing oversized Coval test set, an export of past conversations, or a CSV/JSON of cases) into a small, high-signal Coval test set by removing duplicates, identifying unique scenarios, and selecting a representative, failure-weighted subset — then bulk-loading it with no row cap. Use when the user says "I have thousands of cases", "dedupe my test set", "my test set is too big", "turn this dataset into a test set", "pick representative scenarios", or "my CSV import only kept 10 / uploaded everything".
Analyze a Coval accent testing report from runs across different speaker accents. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or metric summary and wants evidence-backed next steps such as prompt changes, STT/confirmation adjustments, accent-robust routing, or expanded accent coverage.
End-to-end Coval accent testing workflow. Creates one persona per accent (each using a distinct accent voice and mirroring your Standard Customer behavior), launches one run per accent against the same voice agent + test set + metrics, polls for completion, builds a per-persona comparison table from the results, and creates the saved multi-run report (grouped by Persona) via the public API. Use when a user wants to follow the Testing Across Accents cookbook (https://docs.coval.dev/guides/testing-across-accents) without doing each step by hand.
| name | launch-run |
| description | Launch a Coval evaluation run against an AI agent. Use when user wants to start an evaluation, test an agent, or run simulations. |
| argument-hint | [agent-name-or-id] [test-set-name-or-id] |
Launch an evaluation run for $ARGUMENTS.
Ensure the Coval CLI is installed and authenticated:
coval whoami
If not authenticated, run coval login first.
If agent or test set not specified, list available options:
coval agents list
coval test-sets list
coval personas list
Ask user to select:
Ask about optional parameters:
| Option | Flag | Default |
|---|---|---|
| Iterations per test case | --iterations | 1 |
| Concurrent simulations | --concurrency | 5 |
| Run name | --name | Auto-generated |
| Mutation (A/B variant) | --mutation-id | None |
coval runs launch \
--agent-id <agent_id> \
--persona-id <persona_id> \
--test-set-id <test_set_id> \
--iterations <n> \
--concurrency <n> \
--name "Descriptive Run Name"
Report the run ID and status. Offer to watch progress:
Run launched:
<run_id>Status: IN QUEUEWould you like me to watch the progress?
If yes, use coval runs watch <run_id>.
coval runs launch \
--agent-id abc123 \
--persona-id xyz789 \
--test-set-id ts456 \
--iterations 3 \
--concurrency 10 \
--name "Q1 Regression Test"