nre

name	nre
description	Use to drive NVIDIA Omniverse NuRec / Neural Reconstruction Engine (NRE) via the public NGC containers nvcr.io/nvidia/nre/nre and nvcr.io/nvidia/nre/nre-tools (NGC_API_KEY required) — train 3DGUT Gaussian reconstructions from NCore clips, generate aux data, adapt an existing USDZ to an augmented target-vehicle rig (carline adaptation), render frames or LiDAR sweeps (local or warm `serve-grpc`), export PLY/depth/mesh/USDZ, edit actors, and evaluate metrics. Do NOT use for per-object asset capture (use `asset-harvester`) or sensor-to-NCore conversion (use `ncore`).
version	0.2.2
tools	["Shell","Read","Write"]
license	CC-BY-4.0 AND Apache-2.0
compatibility	Linux x86_64, 1+ NVIDIA GPU (Ampere A100/A10/A40/RTX A6000, Ada L20/L40/L40S, Hopper H100/H20, or Blackwell RTX Pro 6000D) with CUDA 12.8 and >= 24 GB VRAM (48+ GB recommended); driver R570+ recommended (R580+ on Blackwell, R535+ minimum for Fixer-only). Docker >= 23.0.1 + NVIDIA Container Toolkit >= 1.13.5, NGC account with NGC_API_KEY exported.
dependencies	["bash","docker","python3"]
metadata	{"author":"NVIDIA NRS <[e-mail]>","tags":["nurec","autonomous-vehicles","neural-reconstruction","rendering","container"],"product_page":"https://www.nvidia.com/en-us/omniverse/nurec/","ngc_container":"nvcr.io/nvidia/nre/nre:latest","ngc_tools_container":"nvcr.io/nvidia/nre/nre-tools:latest","ngc_fixer_model":"https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nre/models/nurec-fixer","hf_datasets":"https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles-NuRec","hf_fixer_model":"https://huggingface.co/nvidia/Difix3D","release_date":"2026-04-30","time-estimate":"2h"}

NRE — NVIDIA Omniverse NuRec (Neural Reconstruction Engine)

Purpose

Drive the public NVIDIA Omniverse NuRec / Neural Reconstruction Engine containers (nvcr.io/nvidia/nre/nre, nvcr.io/nvidia/nre/nre-tools) to train a 3DGUT/3DGRT Gaussian reconstruction from an NCore V4 camera+LiDAR clip, render novel views (locally or via gRPC), generate aux data, export PLY/depth/mesh/ego-mask/tracks, package Asset Harvester output into a USDZ, and evaluate rendering metrics.

This skill carries the host-side toolkit around the NRE CLI: NGC credential resolution, cached-image notes, local render recipes, MP4 encoding, warm serve-grpc boot/teardown scripts, a thin Python gRPC client for repeated RGB renders, bundled rig JSONs, pre-baked custom-rig trajectories, and bash / Hydra / OSMO workflow templates.

When to Use / When NOT to Use

Use this skill when the user has an NCore V4 clip (or a USDZ + NRE artifact pair) on a Linux x86_64 host with an NVIDIA GPU and an NGC API key, and wants to train, render, generate aux data, export artifacts, insert/remove actors, run the gRPC server, or evaluate metrics. Concrete triggers:

• Train a multi-camera + LiDAR AV clip into a renderable USDZ scene with 3DGUT (or 3DGRT ray-traced) Gaussians.
• Generate NuRec auxiliary data (seg, depth, ego mask, DINOv2, LiDAR-seg visibility) using nre-tools.
• Render frames locally (no server) along the training rig or a custom rig + offsets.
• Render novel views via the sensorsim gRPC API (CARLA, Isaac Sim, AlpaSim, custom simulator), optionally with Difix artifact-removal.
• Render LiDAR sweeps via render-grpc --lidar.
• Export PLY / ego masks / depth / Poisson mesh / ground mesh / point clouds / cuboid tracks / NCore tracks / custom rig trajectories.
• Insert / remove / replace 3D actors with export-external-assets
  • render-grpc --edit-assets.
• Render the gated HF dataset nvidia/PhysicalAI-Autonomous-Vehicles-NuRec.
• Upgrade an old USDZ once (upgrade-artifact).
• Inspect / evaluate (export-parsed-config, gaussian-statistics, eval-rendering-metrics, compute-metrics, eval-ground-mesh).
• Browse a USDZ or PLY in the in-container viewer.

Do NOT use this skill when:

• The user still needs to convert raw sensor data into NCore V4 (use the skill first; NRE consumes NCore-formatted shards).