菜单
Compare metrics between two wandb runs (new vs old format). Use when comparing DPO training runs to verify metric consistency.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Suggest the minimal permission rule for the most recent denied tool call. Use when Claude was prompted for permission and you want to allow it permanently.
Read review comments on a GitHub PR and fix the issues raised. Use when the user asks to address, fix, or resolve PR comments or review feedback.
Review PyTorch interview prep solutions on the current branch. Use when the user asks to review interview code, review PyTorch solutions, or invokes /review-interview-code.
Remove AI-generated comment slop from a branch diff. Use when the user asks to clean up AI comments, remove slop, or strip unnecessary comments from a branch.
Update the body of a GitHub pull request. Use when the user asks to update, edit, or modify a PR description/body.
Monitor Beaker experiments until completion. Use when the user asks to monitor, check, or track a Beaker experiment.
| name | compare-wandb-runs |
| description | Compare metrics between two wandb runs (new vs old format). Use when comparing DPO training runs to verify metric consistency. |
| allowed-tools | Bash(uv run python scripts/compare_wandb_runs.py*) |
Compare metrics between two wandb runs using the compare_wandb_runs.py script.
uv run python scripts/compare_wandb_runs.py --new-run <new_run_path> --old-run <old_run_path>
--new-run: The new run path (e.g., ai2-llm/open_instruct_internal/mmkz9dca)--old-run: The old run path (e.g., ai2-llm/open_instruct_internal/fqtszuhd)--rtol: Relative tolerance for comparison (default: 1e-5)--atol: Absolute tolerance for comparison (default: 1e-8)Compare two runs:
uv run python scripts/compare_wandb_runs.py --new-run "ai2-llm/open_instruct_internal/mmkz9dca" --old-run "ai2-llm/open_instruct_internal/fqtszuhd"
Compare with custom tolerances:
uv run python scripts/compare_wandb_runs.py --new-run "ai2-llm/open_instruct_internal/abc123" --old-run "ai2-llm/open_instruct_internal/def456" --rtol 1e-3 --atol 1e-6