generate-world

name	generate-world
description	Drive `geniesim_world` to produce a photorealistic, explorable 3D world from a single equirectangular panorama — uses the `geniesim_world create` CLI (Click subcommand), pairs SHARP + DA360 to fuse panorama RGB with metric depth, and optionally upscales with Real-ESRGAN. Trigger: When the user asks to "generate a world", "生成 3D 世界", "pano to 3D", "PanoRecon", "make a scene from a photo", "create a world from a panorama", or references `geniesim_world` / a `.png` panorama input.
license	MPL-2.0
metadata	{"author":"genie-sim","version":"1.0"}
prerequisites	[]
inputs	[{"name":"panorama","desc":"Equirectangular RGB image (2:1 aspect)","required":true},{"name":"work_dir","desc":"Output directory","required":false,"default":"runs/<timestamp>"},{"name":"device","desc":"Torch device","required":false,"default":"cuda:0"},{"name":"depth_max","desc":"Clip predicted depth at N metres","required":false},{"name":"super_sample","desc":"Run Real-ESRGAN before fusing (needs optional binary)","required":false,"default":"false"}]
outputs	[{"desc":"Work dir with `cubes/`, `depth/`, and a fused world export (`.gsp` / `.ply`). RT Engine load path is W.I.P. — for now the output is consumed by `geniesim_world`'s own tooling."}]

When to Use

• User has an equirectangular panorama (2:1 aspect, e.g. 4096×2048) and wants a 3D world (Gaussian splat / depth map / fused cubes) out of it.
• User wants to seed a Genie Sim scene with a generated environment rather than a hand-authored USD.
• Researcher reproducing the PanoRecon paper (arXiv 2604.07105).

Do not use for:

• USD scene authoring without a panorama input → that's the generate-scene skill in geniesim_generator.
• Asset / object library search → search-assets in geniesim_generator.
• Importing an existing USD scene into the RT Engine → just point the scene yaml's scene_usda at the existing USD (see add-robot / launch-scene). This skill is not the path for that — geniesim_world produces .ply / .gsp Gaussians today, and loading them into a scene_*.yaml is 🚧 W.I.P.

Critical Patterns

1. Out-of-band install. geniesim_world is not pulled in by geniesim bootstrap — it lives behind heavy CUDA deps (PyTorch + ml-sharp + DA360 + optional Real-ESRGAN). Install in its own conda env, separate from the rest of the stack.
2. Three external dependencies, not on pip. Need external/ml-sharp/, external/DA360/ (with checkpoint), and optionally external/realesrgan-ncnn-vulkan-…/. The external/ tree sits next to geniesim_world/ under source/.
3. DA360 checkpoint is mandatory. Either drop it at external/DA360/DA360_large.pth, pass --da360-checkpoint, or set GENIESIM_DA360_CHECKPOINT. Without it, depth prediction fails immediately.
4. Tested on RTX 5090 / CUDA 12.8. Other GPUs work but need a matching requirements-cu<XX>.txt. Don't paste the cu128 line verbatim onto a cu118 box.
5. Real-ESRGAN is optional. It improves visual fidelity but introduces synthetic texture — disable for tasks where ground-truth pixel statistics matter.

Workflow

Step 1 — Verify the layout

source/
├── geniesim_world/                              # this project
└── external/
    ├── ml-sharp/                                # git clone https://github.com/apple/ml-sharp.git
    ├── DA360/                                   # git clone https://github.com/DepthAnything/DA360.git
    │   └── DA360_large.pth                      # checkpoint (download separately)
    └── realesrgan-ncnn-vulkan-20220424-ubuntu/  # optional, super-sample binary
        └── realesrgan-ncnn-vulkan