isaac-sim

Connect to a running Isaac Sim via the `isaacsim.code_editor.python_server` TCP socket (port 8226) to execute Python remotely. Launch Isaac Sim, send code, create/modify USD stages, run simulations, take viewport or full-app screenshots, inspect/modify prims, control the camera, step physics, read console logs, execute Kit commands. Works in `--no-window` headless mode.

Profile and optimize Isaac Sim performance using benchmark scripts and Tracy profiling. Use when the user asks to profile, benchmark, measure performance, compare frame times, or optimize Isaac Sim workloads.

Generate pixel-difference GIF animations comparing benchmark validation captures against golden images. Use when the user asks to visualize validation failures, compare captured vs golden images, generate diff GIFs, or debug image tolerance issues.

Build headless static-scene data collection pipelines using Isaac Sim 6 / Kit 110 Replicator. Produces annotated RGB, depth, segmentation, bounding boxes, and pose data via `BasicWriter`, `KittiWriter`, `CocoWriter`, `CosmosWriter`, `FPSWriter` (from `omni.replicator.core`) and `PoseWriter`, `DataVisualizationWriter` (from `isaacsim.replicator.writers`). Uses `isaacsim.core.experimental.utils.stage` to author the scene and either `rep.functional.modify.semantics` or `isaacsim.core.experimental.utils.semantics.add_labels` to label prims for annotators. Use when generating synthetic data, creating training datasets, or building SDG pipelines. For mobile-robot trajectory-driven SDG see `mobility-gen`; for grasp / teleop / episode-record workflows see `isaacsim.replicator.*` pointers below.

Camera creation, configuration, calibration, render products, lens distortion, and runtime sensor wrapping in Isaac Sim 6 / Kit 110. Modern path: `isaacsim.sensors.experimental.rtx.RtxCamera` (authoring) + `CameraSensor` / `TiledCameraSensor` / `SingleViewDepthCameraSensor` (runtime) on top of `UsdGeomCamera`. Covers AOVs / annotators, OpenCV / fisheye / LUT lens distortion via `OmniLensDistortion*API` schemas, OpenCV intrinsics conversion, animation, and Replicator randomization. Use when creating/configuring camera prims, attaching annotators, applying distortion models, converting real-world calibration to Omniverse units, animating cameras, or generating synthetic data with Replicator.

Running Isaac Sim 6.0 headlessly with `--no-window` for batch simulation, automated data generation, and server-side workloads. Covers launch modes (native headless, physics-only, standalone Python), key CLI flags, app configurations, and the canonical `SimulationApp` bootstrap pattern. Use when: launching Isaac Sim without a display, running batch simulations on a server, generating synthetic data headlessly, or wrapping a standalone Python script in the headless `SimulationApp` lifecycle. Triggers on: headless Isaac Sim, --no-window, batch simulation, SimulationApp, physics-only, standalone Python script.

Top-level dispatcher: turns a natural-language Isaac Sim request into a runnable simulation. Owns the env-var contract (`$ISAAC_SIM_DIR`, `$ISAAC_LAB_DIR`, `$WORKSPACE_DIR`, `$CIP_ROOT`), decomposes the task into capabilities, routes to specialist skills (`usd-pipeline`, `isaac-sim-rendering`, `isaac-sim-validator`, `physics-simulation`, `usd-composition-architecture`), and validates output before delivery. Use when (1) creating a sim scene with robots, objects, environments, (2) controlling robots in Isaac Sim, (3) generating renders or camera captures, (4) collecting Physical AI training data, (5) running headless sims on GPU, (6) orchestrating multi-robot fleets in warehouses.

Headless rendering with Isaac Sim 6.0+ / Kit 110 — frame capture, lighting setup, ACES tone mapping calibration, look-at camera math, and validation thresholds for batch simulation and synthetic data generation. Covers SimulationApp + Replicator RGB annotator, RT2 (RayTracedLighting) vs PathTracing tradeoffs, multi-layer warehouse lighting recipes, deep-aisle vs overview exposure tension, and frame quality validation. Use when capturing screenshots or video from Isaac Sim, generating synthetic data, building rendering pipelines, or tuning lighting and tonemapping for photoreal warehouse / robot / factory scenes. Triggers on: Isaac Sim rendering, headless render, RT2, RayTracedLighting, PathTracing, ACES tonemap, filmIso, SimulationApp render, replicator annotator, warehouse lighting, look-at camera, frame validation.

Creates Arena-conformant commits and pull requests. Enforces DCO sign-off (git commit -s), no AI attribution lines (no Co-Authored-By, no Generated-with-Claude footers), <username>/<type>/<short-description> branch naming, separate new commits rather than --amend when iterating on PR feedback, and main as the default base branch. Use when the user asks to commit, stage changes, push the current branch, open a pull request, make a PR, submit changes, or mark work as ready to merge.

Sets up and manages the isaaclab_arena Docker container — the single environment used for Arena's development, testing, training, and evaluation. Use when the user asks to set up the dev environment, bootstrap the project, get started on a fresh clone, start developing, build or rebuild the image, start or attach to the container, or run any command inside it. Also covers ./docker/run_docker.sh flag combinations (-r rebuild, -R rebuild without cache, -d/-m/-e custom dataset/model/eval mounts), docker exec usage, and the /isaac-sim/python.sh aliasing.

Runs the Isaac Lab-Arena pytest suite across its three required phases (no-cameras, with-cameras, with-subprocess) inside the running isaaclab_arena Docker container. Use when the user asks to run the tests, test their changes, verify the test suite passes, check whether a change broke anything (e.g. "did I break anything", "do the tests still pass", "is everything still working"), run only one specific phase (e.g. "run Phase 1", "run only the no-camera tests", "skip the camera and subprocess tests"), or run coverage for a specific module.

Deploy and operate a cloud Isaac Workstation with Isaac Automator: provision a GPU VM running Isaac Sim, Isaac Lab, and/or Isaac Lab Arena on AWS, GCP, Azure, or Alibaba Cloud, connect to it, move data in and out, control cost with stop/start, repair, import existing deployments, and destroy. Use when the user wants a ready-to-use remote Isaac workstation in the cloud via the Isaac Automator CLI. Do NOT use for installing Isaac Sim/Lab on local hardware, for developing Isaac Automator itself, or for writing simulation / robot-learning code.