| name | isaac-camera |
| description | Camera creation, configuration, calibration, render products, lens distortion, and runtime sensor wrapping in Isaac Sim 6 / Kit 110. Modern path: `isaacsim.sensors.experimental.rtx.RtxCamera` (authoring) + `CameraSensor` / `TiledCameraSensor` / `SingleViewDepthCameraSensor` (runtime) on top of `UsdGeomCamera`. Covers AOVs / annotators, OpenCV / fisheye / LUT lens distortion via `OmniLensDistortion*API` schemas, OpenCV intrinsics conversion, animation, and Replicator randomization. Use when creating/configuring camera prims, attaching annotators, applying distortion models, converting real-world calibration to Omniverse units, animating cameras, or generating synthetic data with Replicator.
|
Camera (Isaac Sim 6 / Kit 110)
Two layers stack:
UsdGeomCamera — the USD camera prim (focal length, apertures, clipping, transform).isaacsim.sensors.experimental.rtx — the runtime sensor wrapper (authoring class + runtime sensor class). Use this for any new code that captures frames, attaches annotators, or applies lens distortion.
The legacy isaacsim.sensors.camera.Camera class still works but the extension itself is a stub (no Python). The implementation moved to isaacsim.sensors.experimental.rtx; new code should use that.
Migration: see the Migration from isaacsim.sensors.camera.Camera table below, plus the official camera migration guide for the full API change set.
When to use
- Create or configure camera prims in USD stages.
- Attach RGB / depth / segmentation / bbox annotators for capture.
- Apply lens distortion (OpenCV pinhole, OpenCV fisheye, LUT).
- Convert real-world calibration (OpenCV / ROS) to Omniverse units.
- Build stereo, multi-sensor, or tiled multi-camera rigs.
- Animate cameras (keyframes, motion paths, orbit, fly-through).
- Generate synthetic data with Replicator randomization.
- Set up Metropolis cameras for MEGA scenarios.
Fundamentals
Unit system
OpenUSD expresses optical properties in tenths of a scene unit.
| Property | Units | Notes |
|---|
| Focal length | tenths of scene unit | 35 = 35 mm when scene units are cm |
| Horizontal aperture | tenths of scene unit | sensor / film width |
| Vertical aperture | tenths of scene unit | sensor / film height |
| Focus distance | scene units | perfectly-sharp distance |
| Clipping range | scene units | near / far clip planes |
Meter stage: a 25 mm focal length is 0.025.
Coordinate system
Camera looks down -Z; +Y is up, +X is right. Same regardless of stage up-axis.
Path A — RtxCamera + CameraSensor (recommended)
RtxCamera wraps a UsdGeom.Camera prim with OmniSensorAPI applied; CameraSensor builds a Replicator render product on top and exposes annotators. Both extend the isaacsim.core.experimental.prims.XformPrim family, so the standard pose / transform methods apply.
from isaacsim.sensors.experimental.rtx import RtxCamera, CameraSensor
import isaacsim.core.experimental.utils.app as app_utils
cam = RtxCamera(
"/World/cam",
tick_rate=30.0,
aux_output_level="NONE",
)
cam.camera.set_focal_lengths(24.0)
cam.camera.set_clipping_ranges(0.01, 1000.0)
sensor = CameraSensor(
cam,
resolution=(480, 640),
annotators=["rgb", "distance_to_image_plane"],
)
app_utils.play(commit=True)
data = sensor.get_data("rgb")
TiledCameraSensor batches many cameras into one render call; useful for multi-view RL data collection. SingleViewDepthCameraSensor simulates stereo depth via OmniSensorDepthSensorSingleViewAPI.
Lens distortion (OpenCV fisheye / pinhole)
Set via API schemas + USD attributes at authoring time. The schema and attribute names are stable; use them through RtxCamera's schemas and attributes constructor args, or Apply them on the camera prim directly.
from isaacsim.sensors.experimental.rtx import RtxCamera
from pxr import Gf
cam = RtxCamera(
"/World/fisheye",
schemas=["OmniLensDistortionOpenCvFisheyeAPI"],
attributes={
"omni:lensdistortion:opencvFisheye:fx": 500.0,
"omni:lensdistortion:opencvFisheye:fy": 500.0,
"omni:lensdistortion:opencvFisheye:cx": 640.0,
"omni:lensdistortion:opencvFisheye:cy": 360.0,
"omni:lensdistortion:opencvFisheye:k1": 0.05,
"omni:lensdistortion:opencvFisheye:imageSize": Gf.Vec2i(1280, 720),
},
)
Equivalent OpenCV-pinhole schema: OmniLensDistortionOpenCvPinholeAPI (attributes omni:lensdistortion:opencvPinhole:{fx,fy,cx,cy,k1,k2,p1,p2,...}). LUT-based distortion uses OmniLensDistortionLutAPI.
Path B — UsdGeomCamera + omni.replicator.core (no RTX wrapper)
When you don't need RTX sensor features (lens distortion, ISP, tick-rate control), straight USD + Replicator works:
from pxr import UsdGeom, Gf
import omni.replicator.core as rep
cam = UsdGeom.Camera.Define(stage, "/World/Cam")
cam.GetFocalLengthAttr().Set(24.0)
cam.GetHorizontalApertureAttr().Set(36.0)
cam.GetVerticalApertureAttr().Set(20.25)
cam.GetClippingRangeAttr().Set(Gf.Vec2f(0.01, 1000.0))
rp = rep.create.render_product(cam.GetPath(), (1920, 1080))
rgb = rep.AnnotatorRegistry.get_annotator("rgb")
depth = rep.AnnotatorRegistry.get_annotator("distance_to_camera")
semseg = rep.AnnotatorRegistry.get_annotator("semantic_segmentation",
init_params={"colorize": True})
normals = rep.AnnotatorRegistry.get_annotator("normals")
for ann in (rgb, depth, semseg, normals):
ann.attach([rp])
For pose authoring use isaacsim.core.experimental.objects.Camera (wraps UsdGeom.Camera with the standard pose / scale API), or the look-at helper:
from isaacsim.core.experimental.objects import Camera
cam = Camera(paths="/World/Cam", positions=[2.0, 2.0, 2.0])
cam.set_world_poses(positions=[[2.0, 2.0, 2.0]],
orientations=[[1.0, 0.0, 0.0, 0.0]])
OpenCV intrinsics -> USD
from pxr import UsdGeom
def opencv_to_usd_camera(stage, path, fx, fy, cx, cy, image_w, image_h,
sensor_w_m=0.036):
"""OpenCV intrinsics -> USD camera on a meter-stage."""
focal_length_m = fx * sensor_w_m / image_w
h_aperture = sensor_w_m
v_aperture = fy * sensor_w_m / fx
cam = UsdGeom.Camera.Define(stage, path)
cam.GetFocalLengthAttr().Set(focal_length_m)
cam.GetHorizontalApertureAttr().Set(h_aperture)
cam.GetVerticalApertureAttr().Set(v_aperture)
cam.GetHorizontalApertureOffsetAttr().Set((cx - image_w / 2) * sensor_w_m / image_w)
cam.GetVerticalApertureOffsetAttr().Set((cy - image_h / 2) * sensor_w_m / image_w)
return cam
Animation
from pxr import Usd, Gf
for frame, pos in enumerate(camera_positions):
cam.GetPrim().GetAttribute("xformOp:translate").Set(
Gf.Vec3d(*pos), Usd.TimeCode(frame)
)
Replicator randomization
import omni.replicator.core as rep
with rep.new_layer():
cameras = rep.get.prims(path_pattern="/World/Camera.*")
with rep.trigger.on_frame(num_frames=200):
with cameras:
rep.modify.pose(
position=rep.distribution.uniform((-5, -5, 2), (5, 5, 8)),
look_at="/World/Target",
)
Sensor checker / supported configs
For asset-driven lidar / camera configs and validation helpers, see isaacsim.sensors.experimental.rtx.SUPPORTED_LIDAR_CONFIGS and the sensor_checker module bundled with the extension. The Replicator AOV annotator names match the standard registry: rgb, distance_to_image_plane, distance_to_camera, normals, semantic_segmentation, instance_segmentation, bounding_box_2d_tight, bounding_box_2d_loose, bounding_box_3d, camera_params, pointcloud, occlusion.
Migration from isaacsim.sensors.camera.Camera
Full guide: isaacsim.sensors.camera → isaacsim.sensors.experimental.rtx.
| Old | New |
|---|
isaacsim.sensors.camera.Camera("/World/cam", resolution=(1280, 720)) | RtxCamera("/World/cam") + CameraSensor(cam, resolution=(720, 1280), annotators=["rgb"]) |
camera.set_focal_length(24) | cam.camera.set_focal_lengths(24.0) |
camera.set_clipping_range(0.01, 1000) | cam.camera.set_clipping_ranges(0.01, 1000) |
camera.initialize() then camera.get_current_frame() | app_utils.play(commit=True) then sensor.get_data(annotator) |
Carb-settings distortion (/<path>/distortionModel) | OmniLensDistortion*API schemas on the camera prim |
