メニュー
Expert-level guide for using the waza CLI to automate the skill development workflow and run evaluation benchmarks. You MUST load this skill when working with the waza command.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/Cogni-AI-OU/cogni-ai-agentic-collections --skill waza-cliコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SOC 職業分類に基づく
SKILL.md を表示中
| name | waza-cli |
| description | Expert-level guide for using the waza CLI to automate the skill development workflow and run evaluation benchmarks. You MUST load this skill when working with the waza command. |
| license | MIT |
waza) must be installed.curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bashcopilot-sdk executor and needs GitHub Copilot access for running evaluations.
Run copilot login in your terminal to authenticate.waza init to initialize a new evaluation suite.waza generate to create an eval suite from an existing SKILL.md file.waza run to execute evaluation benchmarks.waza serve to view results.waza init (Creates eval.yaml, with example task, with example fixture)tasks/fixtures/waza init my-eval-suitewaza generate path/to/SKILL.md (Parses YAML frontmatter like name, description, and creates eval.yaml, starter tasks, and fixtures)waza generate SKILL.md --output-dir ./my-evalwaza run)Run an evaluation benchmark from a YAML spec file:
waza run path/to/eval.yaml --context-dir path/to/fixtureswaza run eval.yaml -c ./fixtures -v -o results.jsonwaza run eval.yaml -t "task-name-1" -t "task-name-2"waza run eval.yaml --parallel --workers 4waza run eval.yaml --transcript-dir ./transcriptswaza serve / waza compare)waza servewaza compare run1.json run2.jsonwaza compare gpt4.json claude.json gemini.jsonwaza compare run1.json run2.json --format json
(Shows per-task score deltas, pass rate differences, and aggregate statistics)waza dev)Iteratively improve SKILL.md frontmatter compliance with automated scoring:
waza dev skills/my-skillwaza dev skills/my-skill --target highwaza dev skills/my-skill --target medium --auto --max-iterations 3# Create Your First Skill
# Initialize a project and create a skill:
mkdir my-eval-suite
cd my-eval-suite
waza init
waza new skill my-skill
# Run Eval:
waza run skills/my-skill/evals/eval.yaml -v
# (The -v flag shows verbose output. Omit it for a summary.)
# Serve the interactive dashboard:
waza serve
# Run all tasks
waza run eval.yaml -v
# Run specific skill
waza run code-explainer
# Specify fixtures directory
waza run eval.yaml -c ./fixtures -v
# Save results
waza run eval.yaml -o results.json
# Filter to specific tasks
waza run eval.yaml --task "basic*" --task "edge*"
# Multiple models (parallel)
waza run eval.yaml --model gpt-4o --model claude-sonnet-4.6
# Use a different judge model for LLM-as-judge graders
waza run eval.yaml --model gpt-4o --judge-model claude-opus-4.6
# Parallel execution with 8 workers
waza run eval.yaml --parallel --workers 8
# With caching
waza run eval.yaml --cache --cache-dir .waza-cache
# Generate JUnit XML for CI test reporting
waza run eval.yaml --reporter junit:results.xml
# A/B testing: baseline vs skill performance
waza run eval.yaml --baseline -o results.json
# Output includes improvement breakdown (quality, tokens, turns, time, completion)
# Auto-update diff grader snapshots
waza run eval.yaml --update-snapshots
# Auto skill discovery
waza run --discover ./skills/
# Auto discovery with strict mode (fail if any SKILL.md lacks eval coverage)
waza run --discover --strict ./skills/
# Skip grading, then grade separately
waza run eval.yaml --skip-graders -o results.json
waza grade eval.yaml --results results.json
# Session event logging
waza run eval.yaml --session-log --session-dir ./logs
# Multi-model comparison with recommendation
waza run eval.yaml --model gpt-4o --model claude-sonnet-4.6 --recommend
# Keep temp workspaces for debugging fixture issues
waza run eval.yaml --keep-workspace -v