Gym

Guide for adding a new benchmark or training environment to NeMo-Gym. Use when the user asks to add, create, or integrate a benchmark, evaluation, training environment, or resources server into NeMo-Gym. Also use when wrapping an existing 3rd-party benchmark library. Covers the full workflow: data preparation, resources server implementation, agent wiring, YAML config, testing, and reward profiling (baselining). Triggered by: "add benchmark", "new resources server", "integrate benchmark", "wrap benchmark", "add training environment", "add eval".

Use when debugging a Nemo Gym run or reward profiling job. Covers rollout collection failures, empty or partial JSONL outputs, stale materialized inputs, verifier/schema errors, Ray or Slurm issues, vLLM readiness, judge failures, tool/sandbox failures, cache problems, and throughput bottlenecks.

Use to help users get started with Nemo Gym reward profiling. Covers the basic gym env start, gym eval run, and gym eval profile workflow, repeated rollouts, materialized inputs, rollout JSONL artifacts, task and rollout identity, output inspection, partial profiling, and rollout_infos. For failed jobs, prefer nemo-gym-debugging.

Use when debugging a Nemo Gym run or reward profiling job. Covers rollout collection failures, empty or partial JSONL outputs, stale materialized inputs, verifier/schema errors, Ray or Slurm issues, vLLM readiness, judge failures, tool/sandbox failures, cache problems, and throughput bottlenecks.

Use to help users get started with Nemo Gym reward profiling. Covers the basic gym env start, gym eval run, and gym eval profile workflow, repeated rollouts, materialized inputs, rollout JSONL artifacts, task and rollout identity, output inspection, partial profiling, and rollout_infos. For failed jobs, prefer nemo-gym-debugging.

Manage stacked branches and pull requests with the gh-stack GitHub CLI extension. Use when the user wants to create, push, rebase, sync, navigate, or view stacks of dependent PRs. Triggers on tasks involving stacked diffs, dependent pull requests, branch chains, or incremental code review workflows.

Use when analyzing NeMo Gym benchmark rollouts for BLADE-style reports, writing benchmark methodology notes, checking whether a benchmark is BLADE-ready, comparing model runs, or explaining why a benchmark report passed, failed, or changed. Covers aggregate metrics, rollout evidence, report structure, root-cause taxonomy, judge expectations, and improvement recommendations. For generic reward profiling commands, prefer nemo-gym-reward-profiling; for failed infrastructure jobs, prefer nemo-gym-debugging.

Use when analyzing NeMo Gym benchmark rollouts for BLADE-style reports, writing benchmark methodology notes, checking whether a benchmark is BLADE-ready, comparing model runs, or explaining why a benchmark report passed, failed, or changed. Covers aggregate metrics, rollout evidence, report structure, root-cause taxonomy, judge expectations, and improvement recommendations. For generic reward profiling commands, prefer nemo-gym-reward-profiling; for failed infrastructure jobs, prefer nemo-gym-debugging.

Use when creating, validating, or documenting Nemo Gym pivot datasets from rollout, trajectory, chat-completion, Responses API, or tool-call artifacts. Covers Gym Responses-style row conversion, pivot selection, single-step tool-use configs, agent_ref alignment, verifier knobs, expected-action row contracts, and train/eval usage.

Use when creating, validating, or documenting Nemo Gym pivot datasets from rollout, trajectory, chat-completion, Responses API, or tool-call artifacts. Covers Gym Responses-style row conversion, pivot selection, single-step tool-use configs, agent_ref alignment, verifier knobs, expected-action row contracts, and train/eval usage.

Maintain the NeMo Gym Fern docs site — add, update, move, or remove pages under fern/. Use for any documentation change. Triggered by: "edit docs", "add doc page", "update docs", "rename page", "fix broken link", "add redirect", "preview docs", "publish docs", any request that touches `fern/`.