菜单
Run Microsoft's eval-recipes benchmarks to validate amplihack improvements against baseline agents. Activates when testing with eval-recipes, running evals, or benchmarking changes.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | eval-recipes-runner |
| version | 1.0.0 |
| description | Run Microsoft's eval-recipes benchmarks to validate amplihack improvements against baseline agents. Activates when testing with eval-recipes, running evals, or benchmarking changes. |
Run Microsoft's eval-recipes benchmarks to validate amplihack improvements against baseline agents.
I can run eval-recipes benchmarks to:
# Clone eval-recipes from Microsoft
git clone https://github.com/microsoft/eval-recipes.git ~/eval-recipes
cd ~/eval-recipes
# Copy our agent configs
cp -r $(pwd)/.claude/agents/eval-recipes/* data/agents/
# Install dependencies
uv sync
Test a specific branch:
# Update install.dockerfile to use specific branch
# Then run benchmark
cd ~/eval-recipes
uv run eval_recipes/main.py --agent amplihack --task linkedin_drafting --trials 3
Compare before/after:
# Test baseline (main)
uv run eval_recipes/main.py --agent amplihack --task linkedin_drafting
# Test PR branch (edit install.dockerfile to checkout PR branch)
uv run eval_recipes/main.py --agent amplihack_pr1443 --task linkedin_drafting
# Compare scores
Common tasks from eval-recipes:
linkedin_drafting - Create tool for LinkedIn posts (scored 6.5/100 before PR #1443)email_drafting - Create CLI tool for emails (scored 26/100 before)arxiv_paper_summarizer - Research toolgithub_docs_extractor - Documentation tool~/eval-recipes/data/tasks/When user says "test this change with eval-recipes":
# In .claude/agents/eval-recipes/amplihack/install.dockerfile
RUN git clone https://github.com/rysweet/...git /tmp/amplihack && \
cd /tmp/amplihack && \
git checkout BRANCH_NAME && \
pip install -e .
cp -r .claude/agents/eval-recipes/* ~/eval-recipes/data/agents/
cd ~/eval-recipes
uv run eval_recipes/main.py --agent amplihack --task TASK_NAME --trials 3
Baseline (main branch):
With PR #1443 (task classification):
User says: "Test PR #1443 with eval-recipes on the LinkedIn task"
I do:
feat/issue-1435-task-classificationcp -r .claude/agents/eval-recipes/* ~/eval-recipes/data/agents/cd ~/eval-recipes && uv run eval_recipes/main.py --agent amplihack --task linkedin_drafting --trials 3~/eval-recipesexport ANTHROPIC_API_KEY=sk-ant-...curl -LsSf https://astral.sh/uv/install.sh | sh.benchmark_results/ in eval-recipes repoFor fully autonomous testing:
# Test suite for a PR
tasks="linkedin_drafting email_drafting arxiv_paper_summarizer"
for task in $tasks; do
uv run eval_recipes/main.py --agent amplihack --task $task --trials 3
done
# Compare results
cat .benchmark_results/*/amplihack/*/score.txt
Expert guidance for Azure DevOps CLI (az devops) - automation, pipelines, repos, boards, and artifacts management. Use when working with Azure DevOps, managing pipelines, creating work items, or when user mentions ADO, builds, releases, or Azure repos.
Expert backlog manager that prioritizes work using multi-criteria scoring, analyzes dependencies, and recommends optimal next tasks. Activates when managing backlogs, prioritizing work, adding items, or analyzing what to work on next.
Expert knowledge of Microsoft Agent Framework for building production AI agents and workflows. Use when building agents with Microsoft's framework, multi-agent orchestration, or tool integration.
Expert project manager orchestrating backlog-curator, work-delegator, workstream-coordinator, and roadmap-strategist sub-skills. Coordinates complex software projects through delegation and strategic oversight. Activates when managing projects, coordinating work, or tracking overall progress.
Expert strategist managing project roadmaps, goals, milestones, and strategic direction. Tracks goal progress, ensures alignment, and provides strategic recommendations. Activates when planning roadmaps, setting goals, tracking milestones, or discussing strategic direction.
Expert delegation specialist that creates comprehensive context packages for coding agents, analyzes requirements, identifies relevant files, and generates clear instructions. Activates when delegating work, assigning tasks, creating delegation packages, or preparing agent instructions.