configure-pipeline

# Configure Nemotron Voice Agent Pipeline ## Purpose Edit the runtime configuration of the voice agent (built-in catalogs, prompts, feature flags) and re-apply Compose without rebuilding images. ## Prerequisites - An existing deployment created by `deploy` (root compose or one of its per-example references). ## Scope - Run commands from the repository root. - Limit repository-backed changes to `.env`, `examples_registry.yaml`, example-local `prompts.yaml`, and per-example service catalogs. - UI-only prompt or service tests stay in browser localStorage. Redeployment is not required. - Exposed UI examples and transports live in `examples_registry.yaml` (`selection` and `transports` fields). Use the `EXAMPLE_SELECTION` env var only to override the registry at runtime (e.g., for one-off benchmarks). - Use `deploy` for initial deployment, profile selection, or auth troubleshooting. ## Instructions 1. Identify the active example by inspecting the running app container (`generic-assistant`, `multilingual-assistant`, `omni-assistant`, `omni-assistant-subagents`, or `frontend-backend-agent`). Each example has its own catalog under its package directory in `src/examples/` (the example id maps to a package dir: `generic-assistant` → `src/examples/generic`, `multilingual-assistant` → `src/examples/multilingual`, `omni-assistant` → `src/examples/omni_assistant`, `omni-assistant-subagents` → `src/examples/omni_assistant_subagents`, `frontend-backend-agent` → `src/examples/frontend_backend_agent`). 2. Edit the smallest configuration surface that satisfies the request: - `.env`: feature flags, tracing, chat history, audio debugging, and buffering. - `examples_registry.yaml`: visible examples (`selection`), allowed transports (`transports`), and per-example slot defaults (`defaults`). - `<example-package-dir>/services.cloud.yaml` (remote / NVCF) and `<example-package-dir>/services.local.yaml` (Compose-managed local NIMs nested under `workstation` / `dgxspark` / `jetson`, matching the example's supported `<example-id>/<hardware>` recipes): built-in LLM, ASR, TTS, and example-specific role catalogs. - `<example-package>/prompts.yaml`: built-in prompt presets and prompt content for the active example. 3. Validate: - Multilingual prompts must use multilingual-capable ASR and TTS from the active catalog (e.g. `parakeet-rnnt`, `nemotron-asr-streaming-multilingual`, `magpie-multilingual-tts`, `chatterbox-multilingual-tts`). Verify the keys exist before referencing them. - Local catalog endpoints must use Compose service names (`nemotron-asr-streaming-english:50052`, `nemotron-asr-streaming-multilingual:50052`, `parakeet-ctc-asr:50052`, `parakeet-rnnt-asr:50052`, `tts-service:50051`, `chatterbox-tts-service:50051`, `nvidia-llm:8000`, `nvidia-llm-vllm:8000`, `nvidia-llm-vllm-omni:8002`, `nemotron-speech:50051`, `booking-server:8001`). Host-run backends auto-rewrite to the matching `localhost` ports. - Alternate local ASR/TTS use Compose profiles (`parakeet-ctc-asr`, `parakeet-rnnt-asr`, `chatterbox-tts`) and share ports with the default. Scale the default off (e.g. `--scale tts-service=0` or `--scale nemotron-asr-streaming-multilingual=0`). Stop an opt-in alternate before returning to the default if it still holds the shared ports. - Workstation local Compose runs ASR/TTS and NIM LLM on GPU `0` by default. Single-GPU deployments are supported only when at least 80 GB of VRAM is available. 4. Apply and verify using `references/apply-changes.md`. ## Rules - `.env` changes: compose re-apply. - YAML catalog changes (`prompts.yaml`, `services.*.yaml`, `examples_registry.yaml`): compose restart of the example service. `./src` and `./examples_registry.yaml` are bind-mounted, so no rebuild needed. - Preserve unrelated keys, comments, and entries while editing. - Alternate local ASR/TTS use Compose profiles where needed (`parakeet-ctc-asr`, `parakeet-rnnt-asr`, `chatterbox-tts`). Magpie TTS uses `tts-service`. - Per-example slot defaults live in `examples_registry.yaml` `defaults`. The catalog file ordering only affects UI listings. The actual default is whatever `defaults` declares. ## Examples **Switch the default LLM to a different cloud model:** 1. Open `examples_registry.yaml` and update the relevant `defaults` entry for the active example (e.g. change `llm: [nemotron-lightning]` to `llm: [nemotron-super]`). The catalog key must exist in the active example's `services.cloud.yaml` / `services.local.yaml`. 2. Compose restart of the example service and refresh browser. **Add a multilingual persona prompt:** 1. Add the prompt to the active example's `prompts.yaml`. 2. To make it the per-example default, update `examples_registry.yaml` `defaults.prompt` for that example to the new prompt key. 3. Ensure the active example's catalog has multilingual-capable ASR (`parakeet-rnnt` or `nemotron-asr-streaming-multilingual`) and TTS (`magpie-multilingual-tts` or `chatterbox-multilingual-tts`). 4. Compose restart of the example service and refresh browser. ## Limitations - Does not deploy the stack or change profiles. Use `deploy` (and its per-example references) for that. - Source code or `Dockerfile` changes require an image rebuild (`--build`). Out of scope here. - UI-only ad-hoc service / prompt overrides (saved in `localStorage`) are intentionally not persisted. This skill writes only to repo files. ## Troubleshooting - **YAML catalog change does not appear in the UI** -> compose re-apply and refresh browser. - **`.env` change has no effect on a running container** -> environment is read at container start. Re-apply Compose so the container restarts. - **Local LLM/ASR/TTS missing from the Services tab** -> the corresponding sidecar is not deployed or is unreachable. The catalog filters local entries by TCP reachability. - **Local workstation LLM won't start or OOMs** -> match it to the GPU in `.env`: `NIM_KVCACHE_PERCENT` (**raise** on `No available memory for the cache blocks`, lower on an OOM kill), `NIM_TAGS_SELECTOR` (weight precision and tensor-parallel size), and `LLM_MAX_NUM_SEQS` (lower if CUDA-graph capture fails). On multi-GPU hosts, match the NIM profile `tp` to the exposed GPUs. See "VRAM & hardware support" in `docs/how-to/configure-llm.md`. - **Multilingual responses do not use the right ASR/TTS** -> reorder catalog so a multilingual ASR/TTS sits first, or pick the entry from the UI Services tab. - **ASR/TTS sidecar image fails to pull** -> log in to `nvcr.io` with a `NVIDIA_API_KEY` that has access to the image. The active image is set in `docker/docker-compose.<variant>.yaml`. - **Local LLM 400 (`auto tool choice requires ...`), or reasoning spoken / `<think>` leaks** -> self-hosted Nemotron-3 2.x needs the parsers set (already in `docker/docker-compose.nemotron3-*.yaml`): NIM `NIM_PASSTHROUGH_ARGS=--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser nemotron_v3`, or the same flags on `vllm serve`. See `docs/06-troubleshooting.md`. - **Raw vLLM `nemotron_v3` not found / Super (`MIXED_PRECISION`) won't load** -> image's vLLM too old; use NGC `vllm:26.07-py3` (vLLM ≥ 0.20), not `vllm:25.12.post1-py3` (0.12.0).