Shared substrate for mobile robot navigation in Isaac Sim. Covers the primitives both runtime navigation (isaac-sim-robot-navigation) and synthetic data generation (mobility-gen) build on top of: OccupancyMap from USD or ROS YAML, A* path planning + smoothing, collision-derived robot footprint / Z-offset / inscribed-and-circumscribed radii, oriented-footprint PhysX overlap_box validation, differential and holonomic wheel kinematics, look-at chase camera math, and standard navigation gotchas. Use when implementing any mobile-robot navigation in Isaac Sim, computing an occupancy map from a USD stage, planning paths over a grid, sizing buffers for robot footprints, setting up chase cameras, or as the foundation BEFORE choosing between runtime navigation and SDG pipelines. Triggers on: occupancy map, A* path planning, OccupancyMap, generate_paths, differential drive, holonomic, mecanum, robot footprint, chase camera, navigation buffer, overlap_box, oriented footprint, inscribed radius.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Shared substrate for mobile robot navigation in Isaac Sim. Covers the primitives both runtime navigation (isaac-sim-robot-navigation) and synthetic data generation (mobility-gen) build on top of: OccupancyMap from USD or ROS YAML, A* path planning + smoothing, collision-derived robot footprint / Z-offset / inscribed-and-circumscribed radii, oriented-footprint PhysX overlap_box validation, differential and holonomic wheel kinematics, look-at chase camera math, and standard navigation gotchas. Use when implementing any mobile-robot navigation in Isaac Sim, computing an occupancy map from a USD stage, planning paths over a grid, sizing buffers for robot footprints, setting up chase cameras, or as the foundation BEFORE choosing between runtime navigation and SDG pipelines. Triggers on: occupancy map, A* path planning, OccupancyMap, generate_paths, differential drive, holonomic, mecanum, robot footprint, chase camera, navigation buffer, overlap_box, oriented footprint, inscribed radius.
Navigation Primitives — Shared Substrate
Foundation layer for mobile robot navigation. Consumed by:
OccupancyMap.from_ros_yaml(path) loads a YAML+PNG pair (produced by occupancy-map). Both isaac-sim-robot-navigation and mobility-gen consume this same format.
Robot Footprints & Z-Offsets — Derived at Runtime
Do not hardcode footprints. Walk the articulation's collider prims and union their world-space AABBs. This handles every robot (Spot, Carter, VSVXL, Jetbot, Kaya, H1, custom) and stays correct when assets change.
import isaacsim.core.experimental.utils.bounds as bounds_utils
from pxr import Usd, UsdPhysics
import numpy as np
defcompute_robot_footprint(stage: Usd.Stage, robot_root: str) -> dict:
"""Return footprint dims + Z-offset + inscribed/circumscribed radii.
Uses prims tagged with UsdPhysics.CollisionAPI under `robot_root`. Falls back
to UsdGeom.Imageable if no colliders are authored.
"""
collider_paths = []
for prim in Usd.PrimRange(stage.GetPrimAtPath(robot_root)):
if prim.HasAPI(UsdPhysics.CollisionAPI):
collider_paths.append(prim.GetPath())
ifnot collider_paths:
collider_paths = [stage.GetPrimAtPath(robot_root).GetPath()]
aabb = bounds_utils.compute_combined_aabb(collider_paths) # [xmin,ymin,zmin, xmax,ymax,zmax]
mn, mx = aabb[:3], aabb[3:]
size = mx - mn
origin_z = stage.GetPrimAtPath(robot_root).GetAttribute("xformOp:translate").Get()[2]
z_offset = max(0.0, origin_z - mn[2]) # how far origin sits above lowest collider
half_w, half_d = size[0] / 2.0, size[1] / 2.0return {
"size": tuple(size), # full footprint extents (m)"z_offset": float(z_offset), # origin → lowest collider (m)"inscribed_radius": float(min(half_w, half_d)), # safe for ANY yaw"circumscribed_radius": float(np.hypot(half_w, half_d)), # worst-case yaw"aabb_min": tuple(mn),
"aabb_max": tuple(mx),
}
Use inscribed_radius when the robot can rotate freely in place (over-conservative, zero clip). Use circumscribed_radius only when you require zero false negatives. For non-circular robots (Spot, VSVXL), prefer the oriented-footprint check below over a single radius.
Always spawn the robot at z = ground + z_offset. Missing the Z-offset is the #1 cause of "robot falls through the floor" or "feet pop above ground" bugs.
Reference Values (sanity check only)
If your compute_robot_footprint output is far from these, your collider authoring or scene units are wrong:
Robot
Expected size (m)
Expected z_offset
Inscribed r
Spot
~1.08 × 0.44 × 0.55
~0.69
~0.22
Spot + arm
~1.10 × 0.40 × 1.20
~0.69
~0.20
Nova Carter
track_w=0.499, wheel_r=0.14
~0.0
~0.25
VSVXL
~2.52 × 1.72, 6-wheel diff
~0.0
~0.86
Jetbot
wheel_base=0.1125, wheel_r=0.03
~0.02
~0.06
Kaya (holonomic)
wheel_base=0.10, wheel_r=0.04
~0.02
~0.10
H1 (humanoid)
—
~1.05
~0.20
Occupancy Map from USD (Direct Projection)
Use when you need a runtime omap and don't already have a map.yaml. For the canonical map.yaml workflow consumed by MobilityGen, use occupancy-map instead.
A. Collider-driven (preferred when assets have authored colliders). Iterate only prims with UsdPhysics.CollisionAPI. This already excludes visual-only geometry (signage, decals, light cones, debug arrows) without a filter list.
from pxr import UsdPhysics
for prim in Usd.PrimRange(stage.GetPrimAtPath("/World")):
ifnot prim.HasAPI(UsdPhysics.CollisionAPI):
continue
enabled = prim.GetAttribute("physics:collisionEnabled")
if enabled and enabled.Get() isFalse:
continue# rasterize this prim's AABB into the grid
B. Visual-bbox + filter list (fallback for scenes without colliders). Naive bbox projection fills the grid with shell, zones, signage. Filter aggressively:
For Spot (circumscribed_radius ≈ 0.58 m) this yields ~0.88–1.08 m, not the legacy 1.5 m blanket value. The legacy value was conservatively tuned against a circular proxy; with the oriented-footprint check (below) you recover the extra ~0.5 m of navigable space.
A* Path Planning
Erode by inscribed_radius (fast, conservative). Then validate the smoothed path with an oriented-footprint collision check, which recovers the navigable space the inscribed-radius erosion threw away.
from scipy.ndimage import binary_erosion
import numpy as np
fp = compute_robot_footprint(stage, "/World/Robot")
kernel_r = int(fp["inscribed_radius"] / RESOLUTION)
kernel = np.zeros((2*kernel_r+1, 2*kernel_r+1), dtype=bool)
for dy inrange(-kernel_r, kernel_r+1):
for dx inrange(-kernel_r, kernel_r+1):
if dx*dx + dy*dy <= kernel_r*kernel_r:
kernel[dy+kernel_r, dx+kernel_r] = True
navigable = (grid == 0)
eroded = binary_erosion(navigable, structure=kernel)
# Standard A* over `eroded` (heapq-based)
Oriented-Footprint Collision Check (recommended for non-circular robots)
Drives the same PhysX query the simulator uses. Works in two modes:
1. Against PhysX scene (after SimulationManager.initialize()): use get_physx_scene_query_interface().overlap_box. Returns hit count; >0 means clip.
import carb
from omni.physx import get_physx_scene_query_interface
from pxr import Gf
deffootprint_clips(x: float, y: float, yaw: float, fp: dict, z_query: float = 0.2) -> bool:
# half-extents of the robot footprint
half = carb.Float3(fp["size"][0] / 2, fp["size"][1] / 2, fp["size"][2] / 2)
origin = carb.Float3(x, y, z_query + fp["size"][2] / 2)
# quaternion (x, y, z, w) for yaw about Z
rot = Gf.Rotation(Gf.Vec3d(0, 0, 1), np.degrees(yaw)).GetQuat()
quat = carb.Float4(*rot.GetImaginary(), rot.GetReal())
hits = get_physx_scene_query_interface().overlap_box(
half, origin, quat, lambda h: True, anyHit=True, # early-exit on first hit
)
return hits > 0
2. Against the rasterized omap (no PhysX needed): stamp the rotated rectangle onto the obstacle grid and AND with the occupied mask.
compute_robot_footprint(stage, robot_root) — get size, z_offset, radii.
Rasterize obstacles onto grid (0.25 m resolution typical).
Binary erode with circular kernel of radius = inscribed_radius / resolution.
A* pathfind on eroded grid.
Catmull-Rom smooth the raw path; assign yaw = atan2(dy, dx) along the curve.
For every smoothed waypoint, run footprint_clips_grid(px, py, yaw, fp, grid) (or footprint_clips(...) against PhysX). Reject the path on any hit.
If a single waypoint fails, snap to the nearest navigable cell and re-validate. If multiple fail, the inscribed-radius A* path is fundamentally bad — re-plan with a larger erosion kernel (circumscribed_radius).
Skipping steps 6–7 produces paths that look fine on the omap but clip walls in render — especially on rectangular robots (Spot, VSVXL) cornering through aisles.
Use HolonomicController with HolonomicRobotUsdSetup to extract wheel positions, orientations, mecanum angles from the robot USD. Apply via WheeledRobot.apply_wheel_actions.
2D MobilityGen action [lin, ang] → 3D holonomic command [forward, lateral=0, yaw]. See mobility-gen for the WheeledMobilityGenRobot.build() override pattern.
DifferentialController + Articulation (Kit 110)
from isaacsim.robot.experimental.wheeled_robots.controllers import DifferentialController
from isaacsim.core.experimental.prims import Articulation
import numpy as np
robot = Articulation("/World/Robot")
robot.initialize_cpp_data_view()
dc = DifferentialController(wheel_radius=0.15, wheel_base=1.52)
wheel_vels = dc.forward(np.array([linear_speed, angular_speed]))
vel_targets = np.zeros((1, num_dofs))
for i in left_wheel_indices: vel_targets[0, i] = wheel_vels[0]
for i in right_wheel_indices: vel_targets[0, i] = wheel_vels[1]
robot.set_dof_velocity_targets(vel_targets)
World Transform Extraction
BBoxCache.ComputeWorldBound() returns LOCAL bounds for Cube prims — wrong for world position. For actual world position use:
xf_cache = UsdGeom.XformCache(Usd.TimeCode.Default())
world_mat = xf_cache.GetLocalToWorldTransform(prim)
position = world_mat.ExtractTranslation()
For simulated position (Articulation runtime, not authored), use:
Chase: 4m behind robot, 2.5m up, looking at robot center
Overhead: 10m up, looking straight down
POV: at robot front (1.26m forward), 0.8m height
Degenerate Up-Vector
When camera looks straight down (fwd ≈ 0,0,−1), cross(fwd, up=(0,0,1)) is zero → broken matrix → blank render. Always fallback:
up = Gf.Vec3d(0, 0, 1)
ifabs(fwd * up) > 0.99:
up = Gf.Vec3d(0, 1, 0)
Common Gotchas (shared across all navigation skills)
Feet/wheels below origin: many robots (Spot ~0.69 m, H1 ~1.05 m) have their articulation origin above the ground contact. Always call compute_robot_footprint(stage, root) and spawn at z = ground + fp["z_offset"].
Instancing invisible in headless Hydra: instanceable=true prims don't render in arm64 headless. Flatten first.
OmniGraph crashes on out-of-range frames: jumping past endTimeCode crashes PushGraph camera animation nodes.
next_update_async: doesn't exist in Isaac Sim 6 — use omni.kit.app.get_app().next_update_async().
Per-frame yaw smoothing: current_yaw += yaw_diff * 0.15 for smooth turning instead of instant snapping.
Frame numbering for ffmpeg: sequential frame_0000.png, frame_0001.png... NOT sparse — ffmpeg skips gaps.
Routes need lit zones: A* paths through dark corridors render as black frames. Define route waypoints in populated/lit areas, or add SphereLights at Z=3m along the route (intensity 800, radius 0.3).
