메뉴
SkillsMP는 NVIDIA/nvidia-resiliency-ext에서 4개의 skill을 수집했습니다. skill을 열어 소스와 세부 정보를 확인하세요.
수집된 skill 4개 중 4개를 표시합니다.
Closed-loop fault injection and attribution accuracy benchmark. Draws from a prioritized pool of (fault_type, rank, iter, nodes) experiments and submits them 2 at a time via sbatch — waiting for each pair to finish before submitting the next — to bound…
원문 언어: 영어
Analyze a SLURM job log file for failure root-cause attribution and restart decisions using NVRxLogAnalyzer. Use when you have a SLURM training job log and need to determine why the job failed and whether it should be restarted. Performs per-cycle chunking,…
원문 언어: 영어
Orchestration layer over nvidia_resiliency_ext attribution modules. Provides log-analysis, fr-analysis, and a Megatron-LM-oriented fault-injection feedback loop for benchmarking attribution quality on SLURM workloads.
원문 언어: 영어
Analyze PyTorch NCCL flight-recorder (FR) dumps to identify collective operation hangs and isolate the responsible ranks using CollectiveAnalyzer. Use when a distributed training job hangs due to an NCCL collective timeout and FR dump files are available.…
원문 언어: 영어