القائمة
Import datasets from HuggingFace and convert them to Coval test sets. Use when the user wants to create test cases from HuggingFace dataset or repository.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
| name | huggingface-import |
| description | Import datasets from HuggingFace and convert them to Coval test sets. Use when the user wants to create test cases from HuggingFace dataset or repository. |
| argument-hint | [huggingface-repo-or-url] |
Import $ARGUMENTS from HuggingFace and convert it into Coval test sets with properly structured test cases.
Coval is an AI evaluation platform for testing voice and conversational AI agents. It runs simulations against AI agents and measures performance with configurable metrics.
| Concept | Description |
|---|---|
| Test Set | A collection of test cases, grouped by category or evaluation purpose |
| Test Case | A single evaluation scenario with input (prompt) and optional metadata |
| Persona | High-level user character (system prompt) - separate from test cases |
| Agent | The AI system being evaluated |
Key distinction:
Base URL: https://api.coval.dev/v1
Fetch the OpenAPI spec before making API calls:
# List specs (no auth)
GET https://api.coval.dev/v1/openapi
# Fetch specific spec
GET https://api.coval.dev/v1/openapi/{spec_name}
If $ARGUMENTS is provided, navigate to it. Otherwise ask:
What is the HuggingFace repository, space, or dataset you want to import?
Then:
Report to the user:
Ask these questions to map HuggingFace data to Coval format:
Q1: Input Field
Which field contains the question/prompt for the test case
input?
Q2: Categorization
How should test cases be organized into test sets?
- By existing category field
- Single test set
- Custom logic
Q3: Metadata
Which fields should be preserved in
metadataJSON? (Recommend: preserve original IDs likequestion_id)
Q4: Multi-turn (if applicable)
How to handle multi-turn conversations?
- First turn only
- Concatenate turns
- Separate test cases per turn
Create Coval-compatible CSVs:
input,metadata
"Your question here","{""question_id"": ""123"", ""source"": ""mt-bench""}"
Requirements:
input column MUST be firstmetadata as valid JSON stringNaming: {source}_{category}.csv
Manual: Upload CSVs via Coval dashboard test sets page.
API: Fetch OpenAPI spec and use test set endpoints programmatically.
| Dataset | Description |
|---|---|
cais/mmlu | 15k+ multiple-choice questions across 57 subjects (STEM, humanities, law) |
nyu-mll/glue | Sentence-level tasks: sentiment, entailment, linguistic acceptability |
tau/commonsense_qa | Reasoning tests for everyday world knowledge |
Rowan/hellaswag | Common-sense inference and completion |
| Dataset | Description |
|---|---|
openai/gsm8k | ~8k grade-school math word problems (multi-step arithmetic) |
ucinlp/drop | Reading comprehension with discrete operations |
lukaemon/bbh | BigBench Hard - challenging reasoning subset |
End-to-end Coval adversarial / red-team testing workflow. Builds one adversarial test set (~10 bad-actor scenarios, each with an expected-behavior checklist), creates a persistent "Adversarial User" persona and a Composite Evaluation metric that scores each scenario against its own expected behaviors, launches a multi-iteration run against the agent (voice or chat), polls for completion, builds a per-scenario pass/fail scorecard, and creates a saved report grouped by Test Case. Use when a user wants to follow the Adversarial & Red-Team Testing cookbook (https://docs.coval.dev/guides/adversarial-red-team-testing) without doing each step by hand. Triggers: "adversarial test set", "red team my agent", "jailbreak / prompt-injection testing", "test my agent against bad actors".
Analyze a Coval adversarial / red-team testing report and turn it into an agent-hardening plan. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or a per-scenario scorecard from an adversarial sweep and wants evidence-backed next steps such as prompt/guardrail changes, refusal hardening, verification fixes, escalation routing, or expanded attack coverage.
Derive a SET of simulation personas for an agent from product artifacts — backend payloads, UI screenshots, journey/product docs, and sample real user messages — instead of designing one persona by hand. Identifies who actually interacts with the agent and how they behave, then creates the personas via the CLI. Best for text/chat agents and for new agents with no interaction history. Use when the user says "make personas from these screenshots/payloads", "who are my users", "create a set of personas", "derive personas from my product", "build a persona library", or "I have backend data, turn it into personas".
Turn a large dataset (an existing oversized Coval test set, an export of past conversations, or a CSV/JSON of cases) into a small, high-signal Coval test set by removing duplicates, identifying unique scenarios, and selecting a representative, failure-weighted subset — then bulk-loading it with no row cap. Use when the user says "I have thousands of cases", "dedupe my test set", "my test set is too big", "turn this dataset into a test set", "pick representative scenarios", or "my CSV import only kept 10 / uploaded everything".
Analyze a Coval accent testing report from runs across different speaker accents. Use when a user provides a Coval report URL, report export, run IDs, screenshots, or metric summary and wants evidence-backed next steps such as prompt changes, STT/confirmation adjustments, accent-robust routing, or expanded accent coverage.
End-to-end Coval accent testing workflow. Creates one persona per accent (each using a distinct accent voice and mirroring your Standard Customer behavior), launches one run per accent against the same voice agent + test set + metrics, polls for completion, builds a per-persona comparison table from the results, and creates the saved multi-run report (grouped by Persona) via the public API. Use when a user wants to follow the Testing Across Accents cookbook (https://docs.coval.dev/guides/testing-across-accents) without doing each step by hand.