coval-external-skills

End-to-end Coval adversarial / red-team testing workflow. Builds one adversarial test set (~10 bad-actor scenarios, each with an expected-behavior checklist), creates a persistent "Adversarial User" persona and a Composite Evaluation metric that scores each scenario against its own expected behaviors, launches a multi-iteration run against the agent (voice or chat), polls for completion, builds a per-scenario pass/fail scorecard, and creates a saved report grouped by Test Case. Use when a user wants to follow the Adversarial & Red-Team Testing cookbook (https://docs.coval.dev/guides/adversarial-red-team-testing) without doing each step by hand. Triggers: "adversarial test set", "red team my agent", "jailbreak / prompt-injection testing", "test my agent against bad actors".

Analyze a Coval adversarial / red-team testing report and turn it into an agent-hardening plan. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or a per-scenario scorecard from an adversarial sweep and wants evidence-backed next steps such as prompt/guardrail changes, refusal hardening, verification fixes, escalation routing, or expanded attack coverage.

Derive a SET of simulation personas for an agent from product artifacts — backend payloads, UI screenshots, journey/product docs, and sample real user messages — instead of designing one persona by hand. Identifies who actually interacts with the agent and how they behave, then creates the personas via the CLI. Best for text/chat agents and for new agents with no interaction history. Use when the user says "make personas from these screenshots/payloads", "who are my users", "create a set of personas", "derive personas from my product", "build a persona library", or "I have backend data, turn it into personas".

Turn a large dataset (an existing oversized Coval test set, an export of past conversations, or a CSV/JSON of cases) into a small, high-signal Coval test set by removing duplicates, identifying unique scenarios, and selecting a representative, failure-weighted subset — then bulk-loading it with no row cap. Use when the user says "I have thousands of cases", "dedupe my test set", "my test set is too big", "turn this dataset into a test set", "pick representative scenarios", or "my CSV import only kept 10 / uploaded everything".

Analyze a Coval accent testing report from runs across different speaker accents. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or metric summary and wants evidence-backed next steps such as prompt changes, STT/confirmation adjustments, accent-robust routing, or expanded accent coverage.

End-to-end Coval accent testing workflow. Creates one persona per accent (each using a distinct accent voice and mirroring your Standard Customer behavior), launches one run per accent against the same voice agent + test set + metrics, polls for completion, builds a per-persona comparison table from the results, and creates the saved multi-run report (grouped by Persona) via the public API. Use when a user wants to follow the Testing Across Accents cookbook (https://docs.coval.dev/guides/testing-across-accents) without doing each step by hand.

Analyze a Coval audio-quality testing report from runs across different voice, speaking-style, volume, interruption, and background-noise scenarios. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or metric summary and wants evidence-backed next steps such as prompt changes, tool handling fixes, STT/TTS adjustments, trace setup, or expanded audio-scenario coverage.

End-to-end Coval audio-quality testing workflow. Launches one run per audio-robustness scenario against the same voice agent + test set + metrics, polls for completion, and produces the multi-run report URL grouped by Persona. Use when a user wants to follow the Testing Across Audio Qualities cookbook (https://docs.coval.dev/guides/testing-across-audio-qualities) without doing each step by hand.

Recommend, create, preview, and attach Coval trace-based metrics from OpenTelemetry spans. Use when a user has Coval traces and wants custom trace metrics, trace-aware LLM judge metrics with include_traces, latency, token usage, provider failures, tool behavior, vertical-specific workflow signals, or production monitoring based on trace attributes.

Improve Coval trace quality after basic ingestion works. Use when traces are sparse, missing useful STT/LLM/TTS/tool spans, missing attributes needed for Coval built-in metrics, or when a customer wants maximum debugging and observability value from agent traces.

Configure an AI agent to send OpenTelemetry traces to Coval. Use when a user wants to add Coval tracing, instrument an agent for simulations or conversation monitoring, make traces show up in Coval, handle SIP/PSTN/WebSocket trace correlation, or replace the one-command wizard with a security-reviewable manual setup.

Troubleshoot Coval OpenTelemetry trace ingestion, missing trace UI, sparse traces, bad simulation or conversation correlation, auth/org errors, oversized payloads, duplicate spans, and production debugging with Trace Search.

Set up and connect a new Coval agent for evaluation. Guides through agent type selection, endpoint configuration, system prompt, and default resource attachment. Use when user says "set up an agent", "connect my agent", "create an agent", "add an agent", or "configure agent endpoint".

Build or improve a Coval dashboard with metric visualizations backed by real data. Creates new dashboards from scratch or rebuilds existing ones by analyzing usage patterns, metric frequency, and data density. Use when user says "create a dashboard", "build a dashboard", "improve my dashboard", "add widgets", "visualize my metrics", "make a performance dashboard", or "dashboard for my runs".

Select and configure evaluation metrics for an AI agent. Guides through metric selection using use-case recommendations, custom LLM-based metric creation with prompt engineering, and agent default attachment. Use when user says "set up metrics", "configure metrics", "create a metric", "what metrics should I use", "add evaluation criteria", or "customize scoring".

Design and create a simulation persona for testing an AI agent. Guides through use case selection, voice and language configuration, behavior prompt crafting, and interruption calibration. Use when user says "create a persona", "design a persona", "set up a test persona", "configure simulation persona", or "build a caller profile".

Build a complete test suite with test set and test cases for evaluating an AI agent. Guides through test set type selection, scenario design using vertical-specific templates, expected behavior crafting, and bulk creation. Use when user says "create test cases", "build test suite", "add test scenarios", "set up evaluation tests", or "design test cases".

Interactively set up a first Coval AI evaluation. Guides users through installing the CLI, connecting an agent, creating personas, building test cases, selecting metrics, and launching their first eval run. Use when user says "onboard", "get started", "set up evaluation", "first eval", "new to coval", or wants help creating their first test run.

Calculate agreement between human ground truth and machine labels for a text LLM judge metric, then analyze transcripts and reviewer notes to propose an improved metric prompt. One metric at a time.

Migrate configuration from Bluejay voice AI testing platform to Coval. Use when customer says "migrate from bluejay", "bluejay migration", "import bluejay config", or needs to transfer agents, simulations, metrics, and schedules from Bluejay to Coval.

Comprehensive overview of ALL Coval platform resources, their hierarchy, relationships, API endpoints, and ID formats. Use when user asks about Coval resources, data model, how things relate, what endpoints exist, or needs context about the platform structure before making API calls.

Launch a Coval evaluation run against an AI agent. Use when user wants to start an evaluation, test an agent, or run simulations.

Full evaluation workflow - launch a run, watch progress, and summarize results. Use for end-to-end agent testing.

Monitor a Coval run's progress with live updates. Use when user wants to check run status or wait for completion.

Download audio recordings from Coval voice simulations. Use when user wants to listen to or analyze call recordings.

Retrieve and analyze simulation results from a Coval run. Use when user wants to review evaluation outcomes or debug agent behavior.

Import datasets from HuggingFace and convert them to Coval test sets. Use when the user wants to create test cases from HuggingFace dataset or repository.