菜单
Find straggler ranks via collective latency imbalance
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
NCCL proxy wait decomposition — culprit (send_gpu_wait) vs victim (recv_wait)
One-shot health check: CPU, GPU, tables, torch step progress, TorchProbe overhead, cluster nodes, NCCL profiler health
SRE first-response triage for distributed training incidents. Automates the manual checks from PyTorch/NCCL debugging runbooks.
Diagnose PyTorch NCCL watchdog timeouts with Flight Recorder collective sequence alignment.
GPU memory and utilization headroom
Detect monotonic GPU memory growth across training steps
基于 SOC 职业分类
| name | slow_rank |
| description | Find straggler ranks via collective latency imbalance |
| category | distributed |
| tables | ["python.comm_collective","global.python.comm_collective","nccl.proxy_ops"] |
| tags | ["rank","straggler","distributed","collective","拖后腿"] |
| keywords | {"en":["slow rank","straggler","lagging rank","which rank"],"zh":["慢 rank","拖后腿","straggler","掉队","哪个 rank","rank 慢"]} |
| parameters | {"step_window":{"type":"integer","default":20},"use_global":{"type":"boolean","default":true}} |
对比各 rank 的 collective 延迟，找出明显偏慢的 straggler。 单机多卡时 rank 列来自 torch.distributed；多机时使用 global.* 并带 _host/_rank 标签。
step_window (integer, default 20): Include collectives from the last N global_stepsuse_global (boolean, default True): Query global.python.comm_collective for cross-node fan-out