nemo-mbridge-recipe-recommender

# Auto Recipe — Recipe Index & Recommendation This skill indexes every shipped recipe and helps users pick the right starting config, adjust parallelism, and avoid common pitfalls. ## How to Use This Skill 1. Ask the user for: **model name/size**, **GPU count & type**, **training goal** (pretrain / SFT / PEFT), and **sequence length** (if non-default). 2. Look up the best-match recipe in the index below. 3. Recommend the recipe function name + entry-point command. 4. Provide adjustment advice (parallelism resizing, batch tuning, pitfalls). ## First Answer Checklist When recommending recipes, always include these distinctions before the long index details: 1. **Library recipes** under `src/megatron/bridge/recipes/` are for functional training and use `scripts/training/run_recipe.py`. 2. **Performance recipes** under `scripts/performance/` are for upper-bound throughput benchmarks. They use mock data and should not be presented as production training recipes. 3. For a first-time Bridge smoke test, recommend `llama3_8b_sft_config` with mock data via `--dataset llm-pretrain-mock`. Do not use `llm-finetune` for the setup-only tryout unless the user specifically asks for an SFT data path. 4. For normal SFT recommendations, use `--dataset llm-finetune`; for pretrain and mock validation recommendations, use `--dataset llm-pretrain-mock`. 5. After the recipe and dataset, give the required resizing rules: TP must divide `num_key_value_heads`, keep TP within one node unless using NVL72-class interconnect, enable SP when TP > 1, configure CP for long context, DP is implicit, and reduce `micro_batch_size` first on OOM. --- ## Entry Points ### Library recipes (functional training) ```bash # Pretrain with mock data uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ --recipe <recipe_function_name> \ --dataset llm-pretrain-mock # SFT with SQuAD uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ --recipe <recipe_function_name> \ --dataset llm-finetune # Override any field via CLI uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ --recipe llama3_8b_pretrain_config \ --dataset llm-pretrain-mock \ 'model.tensor_model_parallel_size=2' \ 'training.global_batch_size=64' ``` ### Performance recipes (throughput benchmarks) ```bash python scripts/performance/run_script.py \ --recipe <model_family> \ --gpu_type h100 \ --num_gpus 64 \ --data mock ``` See the Performance Recipe Index for important caveats before using these for anything beyond throughput benchmarking. --- ## Recipe Unification (Coming Soon — PR #2803) PR [#2803](https://github.com/NVIDIA-NeMo/Megatron-Bridge/pull/2803) is unifying performance recipes into the same **Python function** format used by library recipes. Key changes: - Perf recipes move from `scripts/performance/configs/` → `src/megatron/bridge/recipes/<family>/<model>_perf.py` - Each perf recipe becomes a **self-contained Python function** (e.g. `llama3_8b_h100_bf16_pretrain_config()`) - The old `WorkloadBaseConfig` → `set_workload_base_configs` → `get_perf_optimized_recipe` pipeline is removed - Shared helpers: `_benchmark_common()` (50 iters, timing, TE RNG), `_perf_precision()` (bf16 / fp8_cs / fp8_mx / nvfp4) **Why Python, not YAML?** Previous YAML-based approaches had problems: recipe logic was split across multiple indirection layers, configs were not self-contained, and the two-level pipeline made maintenance and debugging difficult. Python functions are explicit, greppable, and composable. After #2803 lands, both library and perf recipes will be invocable through the same `run_recipe.py` entry point. --- ## Library Recipe Index All recipes live under `src/megatron/bridge/recipes/`. Each function returns a `ConfigContainer` with model, training, optimizer, and data settings. ### Llama | Recipe | Mode | TP | PP | CP | SP | GPUs (min) | Seq Len | |--------|------|----|----|----|----|------------|---------| | `llama2_7b_pretrain_config` | Pretrain | 2 | 1 | — | — | 2 | 4K | | `llama3_8b_pretrain_config` | Pretrain | 2 | 1 | — | ✓ | 2 | 8K | | `llama3_8b_16k_pretrain_config` | Pretrain | 2 | 1 | 2 | ✓ | 4 | 16K | | `llama3_8b_64k_pretrain_config` | Pretrain | 2 | 1 | 4 | ✓ | 8 | 64K | | `llama3_8b_128k_pretrain_config` | Pretrain | 2 | 1 | 8 | ✓ | 16 | 128K | | `llama3_70b_pretrain_config` | Pretrain | 8 | 4 | — | ✓ | 32 | 8K | | `llama3_70b_16k_pretrain_config` | Pretrain | 8 | 4 | 2 | ✓ | 64 | 16K | | `llama3_70b_64k_pretrain_config` | Pretrain | 8 | 4 | 4 | ✓ | 128 | 64K | | `llama31_405b_pretrain_config` | Pretrain | 8 | 16 | — | ✓ | 128 | 8K | | `llama3_8b_sft_config` | SFT | 2 | 1 | — | ✓ | 2 | 8K | | `llama3_70b_sft_config` | SFT | 4 | 4 | — | ✓ | 16 | 8K | | `llama31_405b_sft_config` | SFT | 8 | 8 | — | ✓ | 64 | 8K | | `llama3_8b_peft_config` | PEFT | 1 | 1 | — | — | 1 | 8K | | `llama3_70b_peft_config` | PEFT | 2 | 4 | — | ✓ | 8 | 8K | | `llama31_405b_peft_config` | PEFT | 4 | 8 | — | ✓ | 32 | 8K | ### Qwen2 / Qwen2.5 | Recipe | Mode | TP | PP | Sizes | |--------|------|----|----|-------| | `qwen2_*_{pretrain,sft,peft}_config` | All | 1–8 | 1–4 | 500M, 1.5B, 7B, 14B, 32B, 72B | | `qwen25_*_{pretrain,sft,peft}_config` | All | 1–8 | 1–4 | 500M, 1.5B, 3B, 7B, 14B, 32B, 72B | ### Qwen3 (Dense) | Recipe | Mode | TP | PP | CP | Sizes | |--------|------|----|----|-----|-------| | `qwen3_*_pretrain_config` | Pretrain | 1–8 | 1–2 | — | 600M–32B | | `qwen3_*_sft_config` | SFT | 1–8 | 1–2 | — | 600M–32B | | `qwen3_600m_sft_128k_config` | SFT | 1 | 1 | 8 | 600M (128K seq) | | `qwen3_*_peft_config` | PEFT | 1 | 1 | — | 600M–32B | ### Qwen3 MoE | Recipe | Mode | TP | PP | EP | CP | GPUs | |--------|------|----|----|----|----|------| | `qwen3_30b_a3b_pretrain_config` | Pretrain | 1 | 1 | 8 | — | 8 | | `qwen3_30b_a3b_sft_config` | SFT | 1 | 1 | 8 | — | 8 | | `qwen3_30b_a3b_peft_config` | PEFT | 1 | 1 | 1 | — | 1 | | `qwen3_235b_a22b_pretrain_config` | Pretrain | 4 | 16 | 8 | 2 | 512+ | | `qwen3_235b_a22b_sft_config` | SFT | 4 | 8 | 8 | — | 256 | | `qwen3_235b_a22b_peft_config` | PEFT | 1 | 4 | 4 | — | 16 | ### Qwen3-Next | Recipe | Mode | TP | PP | EP | |--------|------|----|----|-----| | `qwen3_next_80b_a3b_pretrain_config` | Pretrain | 1 | 4 | 8 | | `qwen3_next_80b_a3b_sft_config` | SFT | 1 | 2 | 8 | | `qwen3_next_80b_a3b_peft_config` | PEFT | 1 | 1 | 4 | ### DeepSeek | Recipe | Mode | TP | PP | EP | GPUs | |--------|------|----|----|-----|------| | `deepseek_v2_lite_pretrain_config` | Pretrain | 1 | 1 | 8 | 8 | | `deepseek_v2_pretrain_config` | Pretrain | 1 | 4 | 32 | 128 | | `deepseek_v3_pretrain_config` | Pretrain | 2 | 16 | 64 | 2048 | | `deepseek_v3_pretrain_config_32nodes` | Pretrain | 2 | 8 | 32 | 256 | ### GLM-4.5 | Recipe | Mode | TP | PP | EP | GPUs | |--------|------|----|----|-----|------| | `glm45_355b_pretrain_config` | Pretrain | 2 | 8 | 16 | 256 | | `glm45_air_106b_pretrain_config` | Pretrain | 1 | 4 | 8 | 32 | | `glm45_355b_sft_config` | SFT | 2 | 8 | 16 | 256 | | `glm45_air_106b_sft_config` | SFT | 1 | 4 | 8 | 32 | | `glm45_355b_peft_config` | PEFT | 2 | 4 | 4 | 32 | | `glm45_air_106b_peft_config` | PEFT | 1 | 2 | 4 | 8 | ### Gemma | Recipe | Mode | TP | PP | Sizes | |--------|------|----|----|-------| | `gemma2_*_{pretrain,sft,peft}_config` | All | 2–8 | 1–2 | 2B, 9B, 27B | | `gemma3_1b_{pretrain,sft,peft}_config` | All | 1 | 1 | 1B (32K seq) | ### NemotronH / Nemotron | Recipe | Mode | TP | PP | EP | Notes | |--------|------|----|----|-----|-------| | `nemotronh_{4b,8b,47b,56b}_*_config` | P/S/PEFT | 1–8 | 1–4 | — | Dense SSM-hybrid | | `nemotron_3_nano_*_config` | P/S/PEFT | varies | 1 | 8 | MoE + Mamba | | `nemotron_3_super_*_config` | P/S/PEFT | 4 | 1 | 8 | MoE + Mamba, ~40% CUDA graph gain | | `nemotron_nano_{9b,12b}_v2_*_config` | P/S/PEFT | varies | 1 | — | Dense | ### Other Models | Recipe | Mode | Notes | |--------|------|-------| | `moonlight_16b_{pretrain,sft,peft}_config` | All | MoE EP=8 | | `olmoe_7b_{pretrain,sft,peft}_config` | All | MoE EP=8 | | `ministral3_{3b,8b,14b}_{sft,peft}_config` | SFT/PEFT | Dense | | `gpt_oss_20b_*_config` | All | MoE + FP8/MXFP8 variants | | `gpt_oss_120b_*_config` | All | MoE | | `vanilla_gpt_pretrain_config` | Pretrain | MLM/Bridge parity baseline | | `gpt3_175b_pretrain_config` | Pretrain | TP=4, PP=8, VP=6 | | `kimi_k2_pretrain_config` | Pretrain | 1T MoE, TP=2 PP=16 EP=32 | ### VLM Recipes | Recipe | Mode | TP | PP | EP | GPUs | |--------|------|----|----|-----|------| | `gemma3_vl_{4b,12b,27b}_{sft,peft}_config` | SFT/PEFT | 1–8 | 1–2 | — | 1–16 |