mcore-determinism-debug
Root-cause method when two Megatron runs that should be bit-identical diverge: fingerprint every op, diff the two traces, first mismatch is the culprit. Use when two identical training runs give different loss; a run is bit-exact at N nodes but not 2N; you need to know which op, layer or kernel first diverged; 'nondeterminism', 'not bit-exact', 'runs diverge', 'determinism trace', 'first divergence', 'which kernel is nondeterministic'.
Source facts
- Repository
- NVIDIA/Megatron-LM
- Last source activity
- October 30, 2026 at 05:54
- Detected SKILL.md language
- English
- Stars
- 18,378
- Forks
- 4,724
Install options
The review-first prompt is selected by default. You can switch to a direct command or download a local copy.
Review the source files
Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.
File Explorer
3 filesShowing SKILL.md
SKILL.md
Source instructions · Read-only preview| name | mcore-determinism-debug |
|---|---|
| description | Root-cause method when two Megatron runs that should be bit-identical diverge: fingerprint every op, diff the two traces, first mismatch is the culprit. Use when two identical training runs give different loss; a run is bit-exact at N nodes but not 2N; you need to know which op, layer or kernel first diverged; 'nondeterminism', 'not bit-exact', 'runs diverge', 'determinism trace', 'first divergence', 'which kernel is nondeterministic'. |
