Menu
Compare metrics between two wandb runs (new vs old format). Use when comparing DPO training runs to verify metric consistency.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Suggest the minimal permission rule for the most recent denied tool call. Use when Claude was prompted for permission and you want to allow it permanently.
Read review comments on a GitHub PR and fix the issues raised. Use when the user asks to address, fix, or resolve PR comments or review feedback.
Review PyTorch interview prep solutions on the current branch. Use when the user asks to review interview code, review PyTorch solutions, or invokes /review-interview-code.
Remove AI-generated comment slop from a branch diff. Use when the user asks to clean up AI comments, remove slop, or strip unnecessary comments from a branch.
Update the body of a GitHub pull request. Use when the user asks to update, edit, or modify a PR description/body.
Monitor Beaker experiments until completion. Use when the user asks to monitor, check, or track a Beaker experiment.
| name | compare-wandb-runs |
| description | Compare metrics between two wandb runs (new vs old format). Use when comparing DPO training runs to verify metric consistency. |
| allowed-tools | Bash(uv run python scripts/compare_wandb_runs.py*) |
Compare metrics between two wandb runs using the compare_wandb_runs.py script.
uv run python scripts/compare_wandb_runs.py --new-run <new_run_path> --old-run <old_run_path>
--new-run: The new run path (e.g., ai2-llm/open_instruct_internal/mmkz9dca)--old-run: The old run path (e.g., ai2-llm/open_instruct_internal/fqtszuhd)--rtol: Relative tolerance for comparison (default: 1e-5)--atol: Absolute tolerance for comparison (default: 1e-8)Compare two runs:
uv run python scripts/compare_wandb_runs.py --new-run "ai2-llm/open_instruct_internal/mmkz9dca" --old-run "ai2-llm/open_instruct_internal/fqtszuhd"
Compare with custom tolerances:
uv run python scripts/compare_wandb_runs.py --new-run "ai2-llm/open_instruct_internal/abc123" --old-run "ai2-llm/open_instruct_internal/def456" --rtol 1e-3 --atol 1e-6