nemo-customizer

Fine-tune models on NeMo Platform with `automodel`, `unsloth`, or `rl` (all `submit`-only): HF dataset conversion, filesets, model entities, and job JSON (hyperparameters, batch, schedule, optimizer) + job polling. `automodel`/`unsloth` run SFT/LoRA as Docker GPU jobs; `rl` runs DPO (preference optimization) on a Ray cluster (Kubernetes). Use for train, fine-tune, customize, SFT, LoRA, DPO, preference optimization, learning rate, epochs, or nemo customization.

跳到安装

仓库: NVIDIA-NeMo/nemo-platform
最近来源活动: 2026年8月14日 17:32
检测到的 SKILL.md 语言: 英语
星标: 67
分支: 19

安装方式

默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

直接命令不会经过审查 Prompt；运行前请先检查来源。

npx skills add https://github.com/NVIDIA-NeMo/nemo-platform --skill nemo-customizer

命令会保持在同一行。复制前请横向滚动并检查完整内容。

想先保存到本地？可下载 SkillsMP 当前能够提供的文件。

检查来源文件

决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。

文件资源管理器

16 个文件

SKILL.md

来源说明 · 只读预览

返回搜索返回安装

NeMo Customizer

End-to-end SFT + LoRA (automodel/unsloth), DPO, and GRPO (rl) on NeMo Platform. Three backend plugins ship in this repo — all are submit-only (local run is hard-disabled on each):

Backend	Verb	Trains	Where it runs	Pick when
`automodel` (default)	`submit`	SFT / LoRA	Platform Docker GPU executor (Jobs service schedules containers on the platform host's daemon)	General SFT/LoRA; multi-GPU (data/tensor parallel); distillation; full-weight SFT
`unsloth`	`submit`	SFT / LoRA	Same — Docker GPU job with 4 steps (download → train → upload → model-entity)	User asks for Unsloth, or wants Unsloth's 4-bit LoRA path / optimizer defaults on a single GPU
`rl`	`submit`	DPO (preference) or GRPO (Gym env)	Platform Kubernetes executor — provisions a Ray cluster; 4 steps (download → train → upload → model-entity)	Preference DPO (full-weight), or GRPO with an environment FileSet (`native-v1` / `wheels-v1` / `adapter-wheels-v1`) + Gym rollout rows; GRPO also supports LoRA

nemo-customizer is the router (nemo customization …); training backends are separate plugins (nemo-automodel, nemo-unsloth, nemo-rl). submit posts to the platform API; the platform runs training in container steps — not in the CLI shell. Heavy ML deps live in container images only.

Runtime split: automodel/unsloth need platform.runtime: docker; rl needs platform.runtime: kubernetes (no local Docker fallback — it schedules a Ray cluster on the remote cluster). A given platform is usually one or the other — confirm with execution profiles before picking rl.

Decision rule below in Plugin pick. Batch shell work; reuse resources with --exist-ok; skip CLI --help unless a command fails.

Pre-flight — CLI resolution

Run from the nemo-platform git root (top-level pyproject.toml), not a plugin subfolder. Example commands below use nemo … — resolve the invocation once before any other step:

cd /path/to/nemo-platform
if command -v nemo >/dev/null 2>&1; then
  echo "nemo"
elif command -v uv >/dev/null 2>&1 && uv run nemo --help >/dev/null 2>&1; then
  echo "uv run nemo"
else
  echo "CLI_NOT_FOUND"
fi

Result	Action
`nemo`	Use `nemo …` for all commands in this workflow
`uv run nemo`	Prefix every command with `uv run` (repo dev checkout without `nemo` on `PATH`)
`CLI_NOT_FOUND`	Stop. Route to nemo-setup (`make bootstrap` then `nemo setup` from the nemo-platform repo root). Do not continue.

Authentication (optional)

Platform auth is not required to run customization when the cluster has authentication disabled. Check with nemo auth status — if it reports authentication is disabled, skip login and proceed.

When auth is enabled on the connected platform, API calls need credentials:

Situation	Action
Auth disabled	Skip login
Auth enabled, unsigned JWT allowed (typical local dev: `auth.allow_unsigned_jwt: true`)	`nemo auth login --unsigned-token --email <user email or admin@example.com>`
Auth enabled, OIDC configured	`nemo auth login` (or `--username` / `--password` for non-interactive)
401/403 on any platform call	Run the matching login above, then retry

Use admin@example.com unless the user specifies another email. Run nemo auth status after login to confirm.

HuggingFace token (gated models)

Gated HF repos (Llama, Gemma, Mistral instruct, …) need a platform secret (convention: hf-token) referenced as token_secret on the model fileset — not in job JSON (unlike W&B's api_key_secret). The Files service does not read your local ~/.cache/huggingface or shell HF_TOKEN.

Model access	Action
Public (e.g. `Qwen/Qwen3-1.7B`)	Skip; omit `token_secret` on the fileset
Gated / private HF repo	Before model fileset creation or job submit: `nemo secrets list --workspace default` and confirm `hf-token` exists. If missing, ask the user for their HF token and stop — do not create the fileset or submit until wired up.

Full create/update commands, fileset token_secret, license acceptance, and download-phase errors: references/troubleshooting.md § Gated HuggingFace models.

Plugin pick

Run nemo jobs list-execution-profiles -f json (login first only if auth is enabled — see Authentication; see references/troubleshooting.md for parsing).
If the task is DPO / preference optimization (a {prompt, chosen, rejected} dataset, "align", "preference", "RLHF-style"), GRPO / Gym / Prime Intellect env, or the user explicitly asked for NeMo-RL → rl (requires a GPU profile and platform.runtime: kubernetes).
Else if the user explicitly asked for Unsloth → unsloth.
Else if the user explicitly asked for Automodel → automodel.
Else if any profile has provider: gpu or gpu_distributed → automodel (default, SFT/LoRA).
Else stop and tell the user GPU customization is unavailable (all backends need a GPU execution profile; automodel/unsloth also need platform.runtime: docker, rl needs platform.runtime: kubernetes).

rl runtime gate: rl submit fails fast unless the platform runs platform.runtime: kubernetes (require_distributed_runtime). rl job steps execute as Kubernetes pods via the kubernetes_job execution backend — the docker job backend cannot run rl. Before submitting rl, confirm with nemo jobs list-execution-profiles -f json that the cpu/gpu profiles report backend: kubernetes_job (or volcano_job). If they report backend: docker/subprocess, the platform is not configured for rl: stop and tell the user DPO and GRPO need a Kubernetes-runtime platform — do not start/reuse a docker-runtime platform, and do not fall back to automodel/unsloth (those are SFT/LoRA, neither DPO nor GRPO). To stand up or configure one, see references/rl-kubernetes-runtime.md.

For automodel/unsloth, training never runs inside the nemo CLI process. After submit, the platform's local Docker executor launches GPU container steps on the daemon attached to that platform host (often the same machine as http://127.0.0.1:8080, but always query the platform — not the agent's shell GPU or a separate docker info on another box). rl does not use the Docker executor — its steps run on the Kubernetes cluster the platform is configured against.

Gotchas

Resolve the CLI per Pre-flight — CLI resolution before any nemo … command; run from the nemo-platform git root, not a plugin subfolder.
Set NMP_BASE_URL only when the user gives a platform URL; default http://127.0.0.1:8080 (same as http://localhost:8080). The nemo CLI reads this env var (see SDK NMP_BASE_URL). Track whether the user overrode the base URL — see Platform unreachable below.
Platform unreachable — if any platform API call fails with a connection error (Connection error, timeout, refused):
- User gave a custom URL or you exported a non-default NMP_BASE_URL: stop and tell the user the platform is not reachable at that address. Do not offer to start local services.
- Default URL only (no user override): ask whether to start the platform locally. If they agree, from the nemo-platform git root run in the background:
```
nemo services run \
  --host 0.0.0.0 \
  --port 8080 \
  --controllers jobs,entities,models \
  --service-group all
```
  Poll until healthy (curl -sf http://127.0.0.1:8080/health/ready or retry nemo jobs list-execution-profiles -f json), then continue the workflow. Do not start services without asking.
  - ⚠️ This default start is a DOCKER-runtime platform — valid for automodel/unsloth only. It is NOT valid for rl: rl needs platform.runtime: kubernetes with a kubernetes_job execution backend. Starting this default and submitting rl will fail the runtime gate. For rl, configure/point at a Kubernetes-runtime platform instead — see references/rl-kubernetes-runtime.md. Never start or reuse a docker-runtime platform for rl.
All backends are submit only — nemo customization <plugin> run … hard-fails with a pointer to submit (automodel, unsloth, and rl each disable local run). Do not improvise verbs or pass .

rl (DPO / GRPO) gotchas

rl is DPO or GRPO, not SFT — DPO trains on preference pairs {prompt, chosen, rejected}; GRPO needs an environment FileSet + a Gym rollout-row dataset. Don't route SFT/LoRA work here, and don't route DPO/GRPO to automodel/unsloth.
DPO is full-weight only; GRPO does LoRA too — set finetuning_type: "lora" on the GRPO training block (plus an optional lora block). The output type is inferred, so output still carries only name. Three things the schema enforces: lora must be omitted when finetuning_type is all_weights; lora_merged is rejected outright (no merge at export — train full-weight if merged weights are the goal); and use_triton is forced off when tensor_parallel_size > 1, so leave it alone. Fields and module-selection rules: references/hyperparameters-rl.md § LoRA (GRPO only).
There is no grpo subcommand — GRPO submits through nemo customization rl submit like DPO, selected by training.type: "grpo" in the job JSON. training.type is the union discriminator and is required: omitting it fails with union_tag_not_found rather than defaulting.
GRPO needs TWO FileSets — an environment (code + config, purpose=environment) and a dataset (prompt rows, purpose=dataset). Both are plain string refs in the job JSON. Three environment formats are supported — native-v1, wheels-v1, adapter-wheels-v1 — and picking one is the first question to settle. Full guide: references/gym-environments.md.
Never put .jsonl in the environment package — validation rejects it outright. This bites the native-v1 path especially: Gym's own configs point datasets[].jsonl_fpath at an in-tree file, so an environment copied straight from the Gym source tree fails until the data dir is stripped and the prompts move to the dataset FileSet.
Validate the environment package before uploading — prints or exits 1 with the exact violation. The same checks run at submit against the FileSet listing, so this catches the failure minutes earlier and for free.

Workflow

Common steps then branch by plugin pick:

- [ ] Resolve CLI (Pre-flight — CLI resolution); cd nemo-platform
- [ ] export NMP_BASE_URL (if user provided endpoint); note whether base URL is user-overridden
- [ ] nemo auth status — skip login if auth disabled; if auth enabled and unsigned JWT allowed, `nemo auth login --unsigned-token --email <…>`; if OIDC, `nemo auth login`
- [ ] nemo jobs list-execution-profiles -f json — apply Plugin pick rules above (retry login on 401/403)
- [ ] On connection error: default URL → ask to start platform (see Platform unreachable); custom URL → report unreachable and stop
- [ ] Convert HF dataset → /tmp/train-data/*.jsonl (see references/hf-conversion.md)
- [ ] Create dataset fileset (--exist-ok), upload the JSONL files, nemo files list to verify — automodel/unsloth: train.jsonl (+ validation.jsonl); rl: training.jsonl + validation.jsonl (see rl branch)
- [ ] Gated HF base model? → confirm `hf-token` exists; ask user and stop if missing (see HuggingFace token + troubleshooting § Gated HuggingFace models)
- [ ] Create HF weights fileset + model entity if missing (--exist-ok; gated repos need `token_secret` on fileset — see troubleshooting)

# automodel branch (submit → Docker GPU job)
- [ ] Write /tmp/job.json (batch sizing for ≥48 GB GPU; else Defaults table)
- [ ] nemo customization automodel submit /tmp/job.json --workspace default
- [ ] Poll until top-level terminal (`poll_customization_job.sh`; default 15s interval, or 30–60s manual polls)
- [ ] Report using the template in `references/reporting.md`
- [ ] Optional: compare base vs adapter on validation — `references/eval_helpers.py …` (LoRA only; CHAT format; adapters hot-reload automatically; see `references/post-training-eval.md`)

# unsloth branch (submit → Docker GPU job)
- [ ] Write /tmp/job.json using the UnslothJobInput shape (see Fast path — unsloth)
- [ ] nemo customization unsloth submit /tmp/job.json --workspace default [--profile <gpu-profile>]
- [ ] Poll until top-level terminal (`poll_customization_job.sh unsloth-<job-id>`; default 15s interval)
- [ ] Report using the template in `references/reporting.md`
- [ ] Optional: compare base vs adapter on validation — `references/eval_helpers.py …` (LoRA only; CHAT format; adapters hot-reload automatically; see `references/post-training-eval.md`)

# rl branch (DPO; submit → Kubernetes/Ray job) — requires platform.runtime: kubernetes
- [ ] Verify execution backend: `nemo jobs list-execution-profiles -f json` shows cpu/gpu at `backend: kubernetes_job` (NOT docker/subprocess). If not → stop; do not start a docker platform; configure per references/rl-kubernetes-runtime.md
- [ ] Dataset is PREFERENCE data: upload training.jsonl + validation.jsonl ({prompt,chosen,rejected}) to ONE fileset
- [ ] Write /tmp/job.json using the RlJobInput shape (see Fast path — rl (DPO))
- [ ] nemo customization rl submit /tmp/job.json --workspace default [--profile <gpu-profile>]
- [ ] Read job id from the "name" field in submit stdout (JSON) — submit has no --name flag; do NOT pick the newest rl-* from `nemo jobs list`
- [ ] Poll until top-level terminal (`poll_customization_job.sh rl-<job-id>`; default 15s interval)
- [ ] Report using the template in `references/reporting.md`

# rl branch (GRPO; submit → Kubernetes/Ray job) — requires platform.runtime: kubernetes
- [ ] Verify execution backend (same gate as DPO above)
- [ ] Pick the ENVIRONMENT format — verifiers/hub env → adapter-wheels-v1; Gym source tree → native-v1; own code → wheels-v1 (see references/gym-environments.md)
- [ ] Build the environment package: hub env → `pi-to-gym-conversion --hub-id … --hub-version … --out-dir …` on an internet-capable host; otherwise hand-build the manifest + configs (+ wheels for the two wheels formats)
- [ ] Validate BEFORE upload: `pi-to-gym-conversion --validate-only <pkg-dir>` — cheapest possible failure
- [ ] Upload environment (--purpose environment) and dataset (--purpose dataset) as TWO filesets; preserve relative paths; no .jsonl inside the env package
- [ ] Dataset rows are GYM ROLLOUT ROWS (prompt under responses_create_params.input + agent_ref object) — see references/dataset-formats.md § NeMo-RL (GRPO)
- [ ] Write /tmp/job.json with training.type "grpo" + the `environment` string ref (see Fast path — rl (GRPO))
- [ ] nemo customization rl submit /tmp/job.json --workspace default [--profile <gpu-profile>]
- [ ] Read job id from the "name" field in submit stdout (JSON) — submit has no --name flag
- [ ] Poll until top-level terminal (`poll_customization_job.sh rl-<job-id>`)
- [ ] Report on REWARD, not loss (references/reporting.md)

Fast path — automodel

Substitute <hf-repo>, <hf-dataset>, <model-entity>, <weights-fileset>, <dataset-fileset>, <output-name>.

Setup

export NMP_BASE_URL=http://127.0.0.1:8080   # user override only
cd /path/to/nemo-platform
nemo auth status   # skip login if auth disabled; if enabled + unsigned JWT allowed → login --unsigned-token --email admin@example.com
nemo jobs list-execution-profiles -f json   # platform GPU profiles → automodel; set training.execution_profile if needed

1. Dataset — convert per references/hf-conversion.md, then:

DATASET=<dataset-fileset>   # e.g. commonsense_qa
nemo files filesets create "$DATASET" --workspace default --purpose dataset --exist-ok
nemo files upload /tmp/train-data/train.jsonl "$DATASET" --workspace default --remote-path train.jsonl
# validation.jsonl if present
nemo files list "$DATASET" --workspace default

2. Model — skip if entity exists (nemo models list --workspace default). For gated HF repos, complete HuggingFace token (gated models) first — see references/troubleshooting.md § Gated HuggingFace models for token_secret on the fileset.

WEIGHTS=<weights-fileset>   # e.g. qwen3-1.7b
MODEL_ENTITY=<model-entity>   # Models API entity (not dataset fileset, not HF id)
HF_REPO=<hf-repo>           # e.g. Qwen/Qwen3-1.7B

nemo files filesets create "$WEIGHTS" --workspace default --purpose model --exist-ok \
  --storage '{"type":"huggingface","repo_id":"'"$HF_REPO"'","repo_type":"model","revision":"main"}'

nemo models create "$MODEL_ENTITY" --workspace default --exist-ok \
  --input-data '{"name":"'"$MODEL_ENTITY"'","fileset":"default/'"$WEIGHTS"'","custom_fields":{"hf_model_id":"'"$HF_REPO"'"}}'

For gated repos, add "token_secret":"hf-token" to the --storage JSON (after creating the secret). See troubleshooting § Gated HuggingFace models.

3. Job JSON — write /tmp/job.json. model is the registered model entity (default/<model-entity>), not an HF repo id or dataset fileset. Full hyperparameter reference: references/hyperparameters-automodel.md.

{
  "model": "default/<model-entity>",
  "dataset": {
    "training": "default/<dataset-fileset>",
    "validation": "default/<dataset-fileset>"
  },
  "training": {
    "training_type": "sft",
    "finetuning_type": "lora",
    "lora": { "rank": 16, "alpha": 32 },
    "max_seq_length": 2048
  },
  "schedule": { "epochs": 1 },
  "batch": { "global_batch_size":

4. Submit and poll

nemo customization automodel submit /tmp/job.json --workspace default
bash plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/scripts/poll_customization_job.sh automodel-<job-id>

Read <job-id> from the "name" field in submit stdout (JSON). Do not use 2>&1 before json.load — warnings on stderr break parsing; see Gotchas. Optional interval override: append seconds (e.g. … 30). Or poll manually: nemo jobs get-status automodel-<job-id> every 30–60s.

Fast path — unsloth

Same substitutions as automodel. Steps 1 (dataset) and 2 (model entity) are identical — the differences are the job JSON shape (UnslothJobInput) and the unsloth submit command.

1. Dataset — same as automodel Fast path step 1.

2. Model — same as automodel Fast path step 2.

3. Job JSON — write /tmp/job.json using the UnslothJobInput shape (see references/hyperparameters-unsloth.md). model is an object (not a string), dataset.path is a single fileset ref, hardware.gpus replaces the parallelism block (single GPU in the training container). nemo customization unsloth explain prints the live schema.

{
  "name": "<job-name>",
  "model": {
    "name": "default/<model-entity>",
    "max_seq_length": 2048,
    "load_in_4bit": true,
    "dtype": "auto"
  },
  "dataset": {
    "path": "default/<dataset-fileset>",
    "text_field": "text",
    "apply_chat_template": true
  },
  "training": {
    "training_type": "sft",
    "finetuning_type": "lora",
    "lora": { "rank"

If the model uses messages chat format (preferred when the tokenizer has a chat template), keep dataset.apply_chat_template: true. Otherwise emit a single text column from your converter and set apply_chat_template: false.

4. Submit and poll

nemo customization unsloth submit /tmp/job.json --workspace default
bash plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/scripts/poll_customization_job.sh unsloth-<job-id>

If you try nemo customization unsloth run …, the CLI hard-fails with a pointer to submit.

Fast path — rl (GRPO)

GRPO on a Ray cluster — Kubernetes runtime only, full-weight. Same runtime gate as DPO: confirm nemo jobs list-execution-profiles -f json shows cpu/gpu at backend: kubernetes_job before anything else.

GRPO differs from every other backend in one structural way: it needs two FileSets, an environment (code that runs a rollout and returns a reward) and a dataset of prompt rows. There are no labelled completions — the reward comes from the environment.

1. Environment — the bulk of the work, and it has its own reference: references/gym-environments.md. Pick the format first:

The user has…	Format	How
A Prime Intellect hub env, or any `verifiers` env	`adapter-wheels-v1`	`pi-to-gym-conversion` (below) — the supported path
An env already in the NeMo Gym source tree	`native-v1`	Package the server dir + add a manifest; strip any `.jsonl`
Their own environment code	`wheels-v1`	Hand-build: manifest + configs + vendored wheel closure

For a hub env, run the converter on an internet-capable host — training clusters have no hub egress:

uv run --package nmp-rl pi-to-gym-conversion \
  --hub-id primeintellect/ascii-tree --hub-version 0.1.5 \
  --out-dir ./ascii-tree-pkg --dataset-dir ./ascii-tree-data \
  --validation-fraction 0.1 --upload --workspace default

--upload creates both FileSets and uploads them. Without it, upload by hand (--purpose environment for the package, --purpose dataset for the JSONL) — see references/gym-environments.md § Upload. Always validate first: pi-to-gym-conversion --validate-only ./ascii-tree-pkg.

2. Dataset — the converter writes it for a hub env. For the user's own prompts, rows are Gym rollout rows with the prompt under responses_create_params.input and an agent_ref object — not messages[], not prompt/completion, not preference triples. Schema and a conversion snippet: references/dataset-formats.md § NeMo-RL (GRPO).

3. Model — same as automodel Fast path step 2 (HF weights fileset + model entity; gated repos need token_secret).

4. Job JSON — model, dataset, and environment are all strings; the method is training.type: "grpo". Full field reference: references/hyperparameters-rl.md.

{
  "model": "default/<model-entity>",
  "dataset": "default/<gym-dataset-fileset>",
  "environment": "default/<environment-fileset>",
  "training": {
    "type": "grpo",
    "epochs": 1,
    "learning_rate": 1e-6,
    "max_seq_length": 2048,
    "batch_size": 32,
    "micro_batch_size": 1,
    "num_generations_per_prompt": 8,
    "temperature": 1.0,
    "parallelism": { "num_nodes": 1, "num_gpus_per_node":

LoRA variant. GRPO trains full-weight by default. To train an adapter instead, add finetuning_type and an optional lora block — nothing else changes:

"training": {
  "type": "grpo",
  "finetuning_type": "lora",
  "lora": { "rank": 16, "alpha": 32 },
  "...": "rest as above"
}

The output type is inferred — lora registers an HF PEFT adapter entity against the base model, and output still carries only name. Do not set lora alongside all_weights (hard error), and do not ask for lora_merged (rejected; GRPO does not merge at export). Field table and module-selection rules: references/hyperparameters-rl.md § LoRA (GRPO only).

5. Submit and poll — identical to DPO (no --name; read the rl-<hex> id from submit stdout):

nemo customization rl submit /tmp/job.json --workspace default > /tmp/rl-submit.json
JOB=$(python3 -c "import json;print(json.load(open('/tmp/rl-submit.json'))['name'])")
bash plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/scripts/poll_customization_job.sh "$JOB"

Read GRPO progress on reward, not loss. The GRPO surrogate loss oscillates near zero and says nothing about run quality. train_reward and val_accuracy are the curves that matter — val_accuracy is the validation pass's mean reward under NeMo-RL's name for it. train_truncation_rate rising or train_baseline_reward/pct_mixed falling toward zero explain a reward curve that stopped moving. See references/reporting.md § GRPO metrics.

Fast path — rl (DPO)

DPO on a Ray cluster — Kubernetes runtime only, full-weight. Before anything else, confirm the platform dispatches jobs to Kubernetes: nemo jobs list-execution-profiles -f json must show cpu/gpu at backend: kubernetes_job (not docker/subprocess). If it doesn't, stop — do not start/use a docker-runtime platform; configure a Kubernetes-runtime one per references/rl-kubernetes-runtime.md. Model-entity setup (step 2) is identical to automodel; the dataset is preference data and the job JSON is the RlJobInput shape.

1. Preference dataset — rows are {prompt, chosen, rejected} (see references/dataset-formats.md § NeMo-RL). Upload both files to one fileset:

DATASET=<preference-fileset>   # e.g. dpo-data
nemo files filesets create "$DATASET" --workspace default --purpose dataset --exist-ok
nemo files upload /tmp/dpo-train.jsonl "$DATASET" --workspace default --remote-path training.jsonl
nemo files upload /tmp/dpo-val.jsonl   "$DATASET" --workspace default --remote-path validation.jsonl
nemo files list "$DATASET" --workspace default

2. Model — same as automodel Fast path step 2 (HF weights fileset + model entity; gated repos need token_secret).

3. Job JSON — write /tmp/job.json. model and dataset are strings; the method is under training with type: "dpo". Full field reference: references/hyperparameters-rl.md.

{
  "model": "default/<model-entity>",
  "dataset": "default/<preference-fileset>",
  "training": {
    "type": "dpo",
    "epochs": 1,
    "learning_rate": 5e-6,
    "max_seq_length": 1024,
    "batch_size": 32,
    "micro_batch_size": 1,
    "ref_policy_kl_penalty": 0.05,
    "parallelism": { "num_nodes": 1, "num_gpus_per_node": 1 }
  },
  "output": { "name":

4. Submit and poll — rl submit has no --name flag (the platform auto-generates the rl-<hex> job id), so read it from submit stdout:

nemo customization rl submit /tmp/job.json --workspace default > /tmp/rl-submit.json   # add --profile <name> if the default gpu profile is wrong
JOB=$(python3 -c "import json;print(json.load(open('/tmp/rl-submit.json'))['name'])")
bash plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/scripts/poll_customization_job.sh "$JOB"

Do not derive the job id by picking the newest rl-* from nemo jobs list — a concurrent job or an earlier failed submit selects the wrong one. If /tmp/rl-submit.json does not parse, the submit result is unknown: stop and inspect it (re-submitting risks a duplicate job).

Do not use 2>&1 before json.load — warnings on stderr break parsing; see Gotchas. Or poll manually: nemo jobs get-status rl-<job-id> every 30–60s. If submit fails on an unknown profile, re-list execution profiles and pass --profile <name>. nemo customization rl run … is disabled (no local execution — it provisions a Ray cluster); nemo customization rl explain prints the live schema.

Defaults

Shared:

Field	Value
Workspace	`default`
Plugin	`automodel` (override per Plugin pick)
Training	SFT + LoRA, `max_seq_length` 2048
Schedule	`epochs` ≥ 1; omit `max_steps`
Auth email (when login required)	`admin@example.com` unless user specifies

Automodel-specific:

Field	Value
Parallelism	1 node, 1 GPU, TP=1
Batch	`global_batch_size` 4, `micro_batch_size` 1 (unknown VRAM; see `references/batch-sizing.md` for ≥48 GB)
Optimizer	`learning_rate` 5e-5

Unsloth-specific:

Field	Value
Hardware	`hardware.gpus` `"0"`, `hardware.precision` `bf16` (selection only, single GPU)
Model load	`load_in_4bit: true`, `dtype: "auto"`
Batch	`batch.per_device_train_batch_size` 2, `batch.gradient_accumulation_steps` 4 (effective batch 8; see `references/batch-sizing.md` for ≥48 GB ramp)
Optimizer	`learning_rate` 5e-5, `optim` `adamw_8bit`
Output	`save_method: "lora"` (adapter-only) unless user asks for merged checkpoint
Gradient checkpointing	`training.use_gradient_checkpointing: "unsloth"`

rl-specific (DPO):

Field	Value
Training	DPO, full-weight (`type: "dpo"`; no LoRA)
Model (if user gives none)	`Qwen/Qwen3-0.6B`
Dataset (if user gives none)	`nvidia/HelpSteer3` (preference subset; uploaded raw — see `references/dataset-formats.md` § NeMo-RL)
Schedule (if user gives none)	small demo run: `max_steps` 20 (completes fast, proves the pipeline). For a real run, set `epochs` and omit `max_steps`.
Parallelism	1 node, 1 GPU (`parallelism.num_nodes`/`num_gpus_per_node`)
Batch	`batch_size` 32, `micro_batch_size` 1
Optimizer	`learning_rate` 5e-6 (DPO uses a low LR), AdamW + cosine
DPO	`ref_policy_kl_penalty` (β) 0.05, `sft_loss_weight` 0.0
Max sequence length	1024
Output	full-weight model entity (`output.name`); no adapter

When the user asks for a DPO job without specifics, default to the above: a 20-step run of Qwen/Qwen3-0.6B on nvidia/HelpSteer3 — small enough to finish quickly and confirm the pipeline end-to-end.

rl-specific (GRPO):

Field	Value
Training	GRPO (`type: "grpo"`), full-weight by default (`finetuning_type: "all_weights"`). Use `finetuning_type: "lora"` when the user asks for an adapter or the full-weight run will not fit; `lora_merged` is rejected.
Model (if user gives none)	`Qwen/Qwen3-0.6B`
Environment (if user gives none)	`primeintellect/ascii-tree` converted with `pi-to-gym-conversion` — small, self-contained, no judge model needed
Environment format	`adapter-wheels-v1` (converter output) unless the user's env dictates otherwise
Schedule (if user gives none)	small demo run: `max_steps` 20. For a real run, set `epochs` and omit `max_steps`.
Parallelism	1 node, 1 GPU
Batch	`batch_size` 32, `micro_batch_size` 1
Rollout	`num_generations_per_prompt` 8 (must divide `batch_size`), `temperature` 1.0
Optimizer	`learning_rate` 1e-6 (GRPO uses a very low LR — lower than DPO)
Max sequence length	2048
Validation	`val_at_start: true` when the user wants to see uplift; costs one extra rollout pass
Output	full-weight model entity (`output.name`); with `finetuning_type: "lora"`, an HF PEFT adapter entity instead — the type is inferred, never set
LoRA (when used)	`rank` 16, `alpha` 32, `dropout` 0.0, all linear layers (omit `target_modules`/`exclude_modules`)

When the user asks for a GRPO job without specifics, offer the ascii-tree demo above and say plainly that GRPO needs an environment — it cannot be run from a dataset alone, unlike every other backend here.

Batch sizing

micro_batch_size / global_batch_size (automodel) and per_device_train_batch_size × gradient_accumulation_steps (unsloth) on ≥48 GB GPUs, multi-GPU (data vs tensor parallel), and OOM / throughput tuning live in references/batch-sizing.md. On unknown VRAM the Defaults above are safe — read batch-sizing before raising batch on a known ≥48 GB card. rl (DPO) batch knobs (batch_size / micro_batch_size) are in references/hyperparameters-rl.md.

Worked example

Automodel: Qwen/Qwen3-1.7B + tau/commonsense_qa → CHAT JSONL, fileset commonsense_qa, entity qwen3-1.7b, output qwen3-1.7b-commonsense-qa-lora, epochs: 1 (no max_steps). On ≥48 GB GPU use LoRA ≤4B default: micro 32, GBS 128, learning_rate 1e-4 (high-util: 64 / 256).

Unsloth: same model + dataset + entity + fileset, but nemo customization unsloth submit /tmp/job.json -w default. Job JSON ≤4B row: batch.per_device_train_batch_size 8, batch.gradient_accumulation_steps 16 (effective 128), learning_rate 1e-4, hardware.gpus "0", output.save_method "lora". Poll unsloth-<job-id> to completion. For payload shape only (not the field set or values): plugins/nemo-unsloth/tests/fixtures/minimal_unsloth_sft.json — a smoke-test input, so confirm fields against unsloth explain.

rl (DPO): the no-details default — Qwen/Qwen3-0.6B + nvidia/HelpSteer3 (preference subset, uploaded raw), output qwen3-0.6b-dpo. First confirm kubernetes_job backend (see Plugin pick → rl runtime gate). Upload training.jsonl + validation.jsonl to one fileset, register the model entity, then submit a small 20-step demo job:

{
  "model": "default/qwen3-0.6b",
  "dataset": "default/helpsteer3-dpo",
  "training": { "type": "dpo", "max_steps": 20, "batch_size": 32, "micro_batch_size": 1,
                "learning_rate": 5e-6, "max_seq_length": 1024, "ref_policy_kl_penalty": 0.05,
                "parallelism": { "num_nodes": 1, "num_gpus_per_node": 1 } },
  "output": { "name":

nemo customization rl submit /tmp/job.json -w default, derive the rl-<hex> id (submit has no --name), poll to completion. For payload shape only: plugins/nemo-rl/tests/fixtures/minimal_dpo.json — a smoke-test input, so confirm fields against rl explain. For a real run, replace max_steps: 20 with epochs.

Report to user

After polling reaches a terminal status (completed, error, or cancelled), report using the template in references/reporting.md — one format for all backends. It covers the Fine-tune result header, the Training configuration table (with per-backend examples: automodel, unsloth, rl/DPO), and Using the adapter (automodel/unsloth LoRA) vs Using the fine-tuned model (full SFT / merged / rl DPO), plus metrics extraction, notes by status, /tmp report saving, and error follow-ups.

Reference files

When	Read
HF conversion or MCQA shaping	`references/hf-conversion.md`
CHAT vs SFT vs CUSTOM (automodel); text vs messages (unsloth); preference triples (rl/DPO); Gym rollout rows (rl/GRPO)	`references/dataset-formats.md`
GRPO environments — the three formats (`native-v1`, `wheels-v1`, `adapter-wheels-v1`), converting a verifiers / Prime Intellect env, packaging a Gym source-tree env, bringing your own code, validation, upload, packaging errors	`references/gym-environments.md`
Field glossary, full JSON template, distillation/KD, live-schema pointers (index routes per backend)	`references/hyperparameters.md` → `hyperparameters-automodel.md` / `hyperparameters-unsloth.md` / `hyperparameters-rl.md`
Batch sizing (≥48 GB), OOM / throughput (automodel + unsloth)	`references/batch-sizing.md`
Multi-GPU same node	`references/batch-sizing.md` § Multi-GPU (same node) (unsloth is single-GPU)
Reporting: result template, Training configuration, Using the adapter / fine-tuned model	`references/reporting.md`
Backend choice, execution profiles, submit failure, container images, missing image on remote platform, gated HF auth / download 502, CLI, connection errors	`references/troubleshooting.md` (§ Parsing CLI JSON for `2>&1` / `json.load`; § Gated HuggingFace models for `hf-token`)
rl (DPO) needs Kubernetes job execution — verifying / configuring `runtime: kubernetes` + `kubernetes_job` executors (local platform → remote cluster, launcher image, PVC, loopback)	`references/rl-kubernetes-runtime.md`
Live JSON schema — authoritative, check here first	`uv run nemo customization automodel explain` / `uv run nemo customization unsloth explain` / `uv run nemo customization rl explain`

Related: plugins/nemo-automodel/README.md, plugins/nemo-unsloth/README.md, plugins/nemo-rl/README.md, plugins/nemo-customizer/docs/CUSTOMIZATION.md, skills nemo-files, nemo-status, nemo-secrets.

--venv

uv run --package nmp-rl pi-to-gym-conversion --validate-only <dir>

{"valid": true, ...}

name	nemo-customizer
description	Fine-tune models on NeMo Platform with `automodel`, `unsloth`, or `rl` (all `submit`-only): HF dataset conversion, filesets, model entities, and job JSON (hyperparameters, batch, schedule, optimizer) + job polling. `automodel`/`unsloth` run SFT/LoRA as Docker GPU jobs; `rl` runs DPO (preference) or GRPO (NeMo Gym environment + reward) on a Ray cluster (Kubernetes). Covers building and converting NeMo Gym environment packages (native-v1, wheels-v1, adapter-wheels-v1) and Gym rollout-row datasets. Use for train, fine-tune, customize, SFT, LoRA, DPO, GRPO, RLHF, reinforcement learning, reward environment, NeMo Gym, verifiers, Prime Intellect, preference optimization, learning rate, epochs, or nemo customization.
triggers	["nemo-customizer","nemo customizer","fine-tune","fine tune","finetune","train a model","customize a model","sft","lora","dpo","grpo","rlhf","reinforcement learning","reward environment","nemo gym","nemo-gym","gym environment","verifiers env","prime intellect","environment fileset","direct preference optimization","preference optimization","preference tuning","automodel","unsloth","nemo-rl","nemo rl","nemo customization","nemo-customization","customizer","customization training","automodel submit","unsloth submit","rl submit"]
not-for	["nemo-build-agent (agent scaffold/deploy, not weight training)","nemo-explore (agent design only)","nemo-setup (platform install; route here when CLI resolution fails)","safe-synthesizer (tabular synthetic data training)"]
compatibility

name	nemo-customizer
description	Fine-tune models on NeMo Platform with `automodel`, `unsloth`, or `rl` (all `submit`-only): HF dataset conversion, filesets, model entities, and job JSON (hyperparameters, batch, schedule, optimizer) + job polling. `automodel`/`unsloth` run SFT/LoRA as Docker GPU jobs; `rl` runs DPO (preference) or GRPO (NeMo Gym environment + reward) on a Ray cluster (Kubernetes). Covers building and converting NeMo Gym environment packages (native-v1, wheels-v1, adapter-wheels-v1) and Gym rollout-row datasets. Use for train, fine-tune, customize, SFT, LoRA, DPO, GRPO, RLHF, reinforcement learning, reward environment, NeMo Gym, verifiers, Prime Intellect, preference optimization, learning rate, epochs, or nemo customization.
triggers	["nemo-customizer","nemo customizer","fine-tune","fine tune","finetune","train a model","customize a model","sft","lora","dpo","grpo","rlhf","reinforcement learning","reward environment","nemo gym","nemo-gym","gym environment","verifiers env","prime intellect","environment fileset","direct preference optimization","preference optimization","preference tuning","automodel","unsloth","nemo-rl","nemo rl","nemo customization","nemo-customization","customizer","customization training","automodel submit","unsloth submit","rl submit"]
not-for	["nemo-build-agent (agent scaffold/deploy, not weight training)","nemo-explore (agent design only)","nemo-setup (platform install; route here when CLI resolution fails)","safe-synthesizer (tabular synthetic data training)"]
compatibility