メニュー
Migrate AgentV eval YAML across breaking schema changes, especially workspace contract updates and portable-vs-local runtime binding cleanup.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Write, edit, review, and validate AgentV EVAL.yaml / .eval.yaml evaluation files. Use when asked to create new eval files, update or fix existing ones, add or remove test cases, configure graders (`llm-rubric`, `script`), review whether an eval is correct or complete, convert between EVAL.yaml and evals.json using `agentv convert`, or generate eval test cases from chat transcripts (markdown conversation or JSON messages). Do NOT use for creating SKILL.md files, writing skill definitions, or running evals — running and benchmarking belongs to agentv-bench.
Run AgentV evaluations and optimize agents through eval-driven iteration. Triggers: run evals, benchmark agents, optimize prompts/skills against evals, compare agent outputs across providers, analyze eval results, offline evaluation of recorded sessions, run autoresearch, optimize unattended, run overnight optimization loop. Not for: writing/editing eval YAML without running (use agentv-eval-writer), analyzing existing traces/JSONL without re-running (use agentv-trace-analyst).
Capture, optimize, and publish screenshots to Astro docs. Use when asked to take screenshots for docs, update doc images, compress PNG assets, or add visual documentation to the agentv.dev docs site. Triggers on "add screenshots to docs", "update docs images", "compress screenshots", "optimize PNG", "document with screenshots".
Use when the user asks to analyze, summarize, or extract insights from CSV data or files
Use when reviewing eval YAML files for quality issues, linting eval files before committing, checking eval schema compliance, or when asked to "review these evals", "check eval quality", "lint eval files", or "validate eval structure". Do NOT use for writing evals (use agentv-eval-writer) or running evals (use agentv-bench).
Analyze AgentV evaluation traces and result JSONL files using `agentv inspect` and `agentv results compare` CLI commands. Use when asked to inspect AgentV eval results, find regressions between AgentV evaluation runs, identify failure patterns in AgentV trace data, analyze tool trajectories, or compute cost/latency/score statistics from AgentV result files. Do NOT use for benchmarking skill trigger accuracy, analyzing skill-creator eval performance, or measuring skill description quality — those tasks belong to the skill-creator skill.
| name | agentv-eval-migrations |
| description | Migrate AgentV eval YAML across breaking schema changes, especially workspace contract updates and portable-vs-local runtime binding cleanup. |
Use this skill when updating existing AgentV eval YAML, examples, docs, or generated eval authoring guidance after a schema-breaking change.
Before editing, read references/breaking-changes.md. For stale evals from
AgentV v4.42.4, use that reference as the migration map: it lists the
v4.42.4-era shape, current shape, migration steps, verification commands, and
compatibility notes for each major breaking authoring change. Then compare the
eval file against the current portable contract:
prompts plus tests[].vars;
keep input only for documented raw-case import compatibility.expected_output to vars.expected_output, then add or keep an
explicit assertion that consumes it, usually llm-rubric with
{{ expected_output }}.id, keep provider for the backend/adapter kind,
put provider settings under config, and rewrite env references to
{{ env.NAME }}.use_target, eval_cases, evalcases, providerPromptMap, and
provider_prompt_map from authored eval/config YAML. Current AgentV rejects
these fields, so do not preserve them as compatibility aliases.workspace.scope.--workspace-path for one-off runs or .agentv/config.local.yaml with
execution.workspace_path for persistent local binding.workspace.scope: suite | attempt for portable workspace lifetime. Docker
config is not a replacement for workspace folder lifetime.snake_case and TypeScript internals in
camelCase.After migration, validate with bun apps/cli/src/cli.ts validate <file> when
working in the AgentV repo, or agentv validate <file> from an installed CLI.
Run the repo's parser/schema tests for generated examples and fixtures when the
change affects shared skill data or examples.