| name | renderer-gpu-interop |
| description | Share renderer output zero-copy across GPU APIs and runtimes (Vulkan↔CUDA, Metal/MPS, DLPack to Warp/PyTorch) and own GPU resource lifetime and teardown across that boundary. Use when Codex wires external-memory/-semaphore interop, hands a render target to a training/compute consumer without a CPU bounce, or debugs interop fd ownership, cross-API sync, double-buffered overlap, ARC/handle lifetime, or teardown-order crashes. |
Renderer GPU Interop
Purpose
The headline value of a headless sensor renderer is often that its pixels never touch the CPU between raygen and the consumer (a Warp/PyTorch training tensor). That payoff lives entirely in the interop seam — and the seam is where the scariest, silent bugs live: leaked kernel fds, use-after-free across the API boundary, desynced double buffers, and teardown-order crashes that only fire at process exit. This skill is the knowledge for crossing that boundary safely. Pairs with renderer-realtime-sensor (the sensor-output side) and renderer-simulation-loop (the per-step update side).
Ownership is a transfer protocol — name the exact transfer point
- An exported memory fd (
VK_KHR_external_memory_fd → cuImportExternalMemory, or the reverse) is consumed by exactly one call on success; on failure the producer still owns it and must close() it. Define and document that single transfer point, or you leak a kernel fd every frame (or double-close → EBADF).
- The consumer's view — a DLPack tensor wrapping interop VRAM — does not own the memory: its deleter must be a no-op. The producing allocation owns the buffer and outlives the per-frame view; a real deleter frees memory the renderer still owns → use-after-free next frame.
- On a unified-memory backend (Apple Silicon), the whole external-memory dance collapses: a buffer the staging buffer and the consumer reads its pointer directly. Do not port the fd/external-memory plumbing onto UMA — stub it and record why. (And a host-sync call like is on Shared storage.)
