NVIDIA-NeMo/Gym

add-benchmark

Guide for adding a new benchmark or training environment to NeMo-Gym. Use when the user asks to add, create, or integrate a benchmark, evaluation, training environment, or resources server into NeMo-Gym. Also use when wrapping an existing 3rd-party benchmark…

Idioma do texto original: inglês

add-harness

Guide for adding or updating an agent harness in NeMo-Gym: a CLI or SDK agent runtime (for example Hermes, Pi, Codex, OpenCode, or an agent framework) wrapped as an Agent Server adapter that runs through Gym's Environment Server and agent sessions. Use when…

Idioma do texto original: inglês

run-and-babysit-eval

Take a NeMo Gym evaluation from a ready config to a trusted result: submit it, let the resumable Slurm chain continue through walltime limits, preemption and node loss, inspect and recover the run with `gym eval resubmit --recover`, check health and the…

Idioma do texto original: inglês

nemo-gym-score-triage

Use when a NeMo Gym score does not match a reference number (tech report, model card, Artificial Analysis, a leaderboard, an earlier run), when two runs or two harnesses disagree, when repeat-to-repeat variance looks too high, when the same output gets…

Idioma do texto original: inglês

nemo-gym-docs

Maintain the NeMo Gym Fern docs site — add, update, move, or remove pages under fern/. Use for any documentation change. Triggered by: "edit docs", "add doc page", "update docs", "rename page", "fix broken link", "add redirect", "preview docs", "publish…

Idioma do texto original: inglês

add-model-eval-config

Onboard a model for NeMo Gym evaluations and compose the complete eval config: the vLLM policy service, its wiring as the policy, the reasoning toggle, and each benchmark's own config from Gym's `benchmarks/<name>/`, then validate and smoke it. Use when a new…

Idioma do texto original: inglês

nemo-gym-pivot-datasets

Use when creating, validating, or documenting Nemo Gym pivot datasets from rollout, trajectory, chat-completion, Responses API, or tool-call artifacts. Covers Gym Responses-style row conversion, reconstructing model calls from flattened rollout output,…

Idioma do texto original: inglês

nemo-gym-debugging

Use when debugging a Nemo Gym run or reward profiling job. Covers rollout collection failures, empty or partial JSONL outputs, stale materialized inputs, verifier/schema errors, Ray or Slurm issues, vLLM readiness, judge failures, tool/sandbox failures, cache…

Idioma do texto original: inglês

nemo-gym-reward-profiling

Use to help users get started with Nemo Gym reward profiling. Covers the basic gym env start, gym eval run, and gym eval profile workflow, repeated rollouts, materialized inputs, rollout JSONL artifacts, task and rollout identity, output inspection, partial…

Idioma do texto original: inglês

pr-review

NeMo Gym pull-request review rubric used by the formal /review service, including automatic reviews.

Idioma do texto original: inglês

nemo-gym-pr-checks-and-labels

Inspect and prepare NVIDIA-NeMo/Gym pull requests by reviewing relevant file history, selecting repository-specific local and CI checks, diagnosing missing or failed checks, and choosing current type, area, and state labels. Use for PR readiness, check…

Idioma do texto original: inglês

nemo-gym-release-publish

Publish a NeMo Gym release by running the "Build, validate, and release NeMo-Gym" workflow (release.yaml) as a dry run, checking that the wheel installs, then publishing to PyPI and GitHub and verifying the result. Use when asked to release, publish, cut, or…

Idioma do texto original: inglês

nemo-gym-release-notes

Write and review user-focused NeMo Gym release notes and README news entries. Use when drafting, editing, prioritizing, or validating a release summary, release-note section, changelog highlights, or release announcement.

Idioma do texto original: inglês

gh-stack

Manage stacked branches and pull requests with the gh-stack GitHub CLI extension. Use when the user wants to create, push, rebase, sync, navigate, or view stacks of dependent PRs. Triggers on tasks involving stacked diffs, dependent pull requests, branch…

Idioma do texto original: inglês

nemo-gym-blade-analysis

Use when analyzing NeMo Gym benchmark rollouts for BLADE-style reports, writing benchmark methodology notes, checking whether a benchmark is BLADE-ready, comparing model runs, or explaining why a benchmark report passed, failed, or changed. Covers aggregate…

Idioma do texto original: inglês