cicd

# CI/CD Guide --- ## How CI Works NeMo-RL CI runs on GitHub Actions. Workflows live in `.github/workflows/`. The main test workflow triggers on pushes to `pull-request/<number>` branches. These branches are created automatically by **copy-pr-bot** when a contributor pushes to their fork and the PR receives a trust signal: - The commit is GPG-signed by a maintainer, **or** - A maintainer posts `/ok to test <full-sha>` as a PR comment. --- ## Triggering CI After pushing a new commit to your PR, trigger CI with: ``` /ok to test <full-commit-sha> ``` Use `git rev-parse HEAD` (not the short form) to get the full SHA. > **Re-triggering after new commits**: each `/ok to test <sha>` is specific to > that SHA. After pushing additional commits, post a new comment with the > updated SHA. --- ## CI Labels Every PR **must** have exactly one `CI:*` label — the quality-check job stays red until one is attached. Labels control which test tier runs and whether a new container image is built. | Label | What runs | Container | |-------|-----------|-----------| | `CI:docs` | Doc tests only | Reuses main container (PRs to a release branch such as `r0.8.0` reuse that branch's container, same as `CI:Lfast`) | | `CI:Lfast` | Fast test subset | Reuses main container (PRs to a release branch such as `r0.8.0` reuse that branch's container, published only by pushes to that branch after merges) | | `CI:L0` | Unit tests + docs + lint | Builds new image | | `CI:L1` | L0 + functional tests | Builds new image | | `CI:L2` | Same as L1 today (plan to add convergence tests) | Builds new image | | `Skip CICD` | Nothing (skips all tests) | — | **Default on merge group / push to main**: L1. **Which label to attach when opening a PR:** | Changed paths / nature of change | Label | |----------------------------------|-------| | Docs only (`docs/`, `*.md`, docstrings) | `CI:docs` | | Default, including new features and bug fixes | `CI:Lfast` | | (Rarely needed) | `CI:L0` | | Large refactor, broad changes to core code, or dependency changes | `CI:L1` | | (Not needed: it runs the same jobs as `CI:L1` today) | `CI:L2` | --- ## CI Failure Investigation ```bash # List recent workflow runs for the PR gh run list --repo NVIDIA-NeMo/RL --branch "pull-request/<pr-number>" # View failing run summary gh run view <run-id> --repo NVIDIA-NeMo/RL # Stream failing job output gh run view <run-id> --repo NVIDIA-NeMo/RL --log-failed ``` Common failure patterns: | Symptom | Likely cause | Fix | |---------|-------------|-----| | CI never starts | No trust signal or copy-pr-bot not triggered | Post `/ok to test <sha>` | | `semantic-pull-request` fails | PR title doesn't follow Conventional Commits | Fix PR title; see `contributing` skill | | Linting fails | Style violation | Run `uv run ruff check --fix . && uv run ruff format .` | | Unit test failure | Code regression or missing dependency | Reproduce locally; see `testing` skill |