iwsdk-dev

# IWSDK Development Guide [iwsdk.dev](https://iwsdk.dev) · [GitHub](https://github.com/facebook/immersive-web-sdk) This guide is for AI agents and developers building WebXR or desktop browser 3D applications with IWSDK in cloud/headless Linux environments (no physical GPU, no display server). It assumes Node.js satisfies IWSDK's package engine range: `>=20.19.0 <21.0.0-0 || >=22.12.0 <23.0.0-0 || >=24.0.0`. The managed dev browser uses Playwright Chromium; the dev plugin installs the matching Chromium binary when it is missing. Explain immersive terms in plain language when they first matter instead of assuming the reader knows them. In particular, introduce XR (an immersive session), IWER (IWSDK's browser-based XR emulator), ECS (entity-component system), and controller target-ray versus grip poses with a link to the applicable IWSDK guide or concept page. ### CLI-First Operating Model This guide is intentionally **CLI-first** for cloud-based harnesses. Treat `npx @iwsdk/cli ...` commands as the primary control surface for setup, runtime inspection, screenshots, ECS debugging, XR emulation, and reference queries. The scoped package name is a security boundary: do not shorten it to the unscoped package name, because npx can download that unrelated package when a local binary is missing. Reason: MCP adapter support in cloud harnesses is often missing, stale, or partially wired. Relying on MCP tool availability can degrade generation quality because the agent may plan around tools that are not actually connected. Use MCP tools only when the harness clearly exposes them and they are already working; otherwise use the CLI commands in this guide. Do not rewrite CLI examples into MCP-only workflows. --- ## 1. Project Scaffolding Always use the official `@iwsdk/create` package. The interactive CLI is a short human-facing flow: project name when absent, starting point, then recommended settings or an advanced setup. It does not ask users to classify every low-level WebXR capability. Recommended settings use TypeScript, initialize Git, and install dependencies. **Agent workflow:** infer the requested experience and features, map that intent to the flags below, and run with `--yes`. The Create CLI deliberately does not infer product intent for an agent. Do not replay or script the interactive questions. When the request is ambiguous in a way that changes the starter or a material feature, ask the user. Cloud coding harnesses often start inside an already-created repository. In that case, scaffold in place with `.` and pass `--force` explicitly: ```bash npx @iwsdk/create@latest . --yes --force --target vr ``` Create refuses every non-empty target without `--force`. Forced scaffolding overwrites conflicting generated files, preserves unrelated files, and reuses the existing Git repository. `--yes` answers setup questions; it does not grant overwrite permission. ### Step 1: Starting Point | Target | Flag | When to use | |--------|------|-------------| | **VR** | `--target vr` | Fully immersive virtual environment. | | **Mixed reality** | `--target ar` | Passthrough experience with virtual content in the physical environment. | | **Desktop 3D** | `--target browser` | Desktop browser 3D app without an immersive session. This dedicated starter has `xr: false`, browser locomotion, canvas pointer input, browser interaction support, and IWER disabled. Camera look is app-owned. | The flag value remains `ar` because it maps to WebXR's `immersive-ar` session mode; the user-facing starting point is mixed reality / passthrough. ### Step 2: Features | Feature | Flag | Enable when… | Disable when… | |---------|------|-------------|---------------| | **Physics** | `--physics` / `--no-physics` | Gravity, collisions, bouncing, throwing. | UI-only, data viz, grid-snapping board games. | | **Locomotion** | `--locomotion` / `--no-locomotion` | VR user moves through a large space (teleport/slide). The Desktop 3D starter already supplies browser controls. | Stationary VR experience. | | **Grabbing** | `--grabbing` / `--no-grabbing` | VR/MR users pick up, move, or manipulate objects. The Desktop 3D starter has browser interaction behavior built in. | Gaze/pointer-only or observational VR/MR experience. | | **Scene Understanding** | `--scene-understanding` / `--no-scene-understanding` | MR: interact with real-world surfaces. | MR without surface interaction. | | **Environment Raycast** | `--environment-raycast` / `--no-environment-raycast` | MR: cast rays against real-world geometry. | No real-world hit testing needed. | Create derives WebXR feature requirements from the target and these high-level features. Raw tri-state WebXR settings are not interactive choices and are not agent-facing scaffolding flags. ### Step 3: Project Setup | Choice | Flag | Default with `--yes` | |--------|------|----------------------| | **Language** | `--language ts` / `--language js` | TypeScript | | **Coding-tool files** | `--ai-tools <tools>` (comma-separated) or `--ai-tools none` | None | | **Git** | `--git` / `--no-git` | Initialize a repository | | **Dependencies** | `--install` / `--no-install` | Install dependencies | | **Existing target** | `--force` | Required only when the target directory contains files | ### Step 4: Source (optional) | Flag | Effect | |------|--------| | *(none)* | Matching SDK packages from npm; starter source is embedded in `@iwsdk/create`. | | `--canary` | Baked-in CloudFront CDN canary bundle. | | `--canary <url>` | Custom bundle URL for internal builds. | ### Examples ```bash # VR game (stationary, physics for gameplay) npx @iwsdk/create@latest space-pong --yes --target vr --physics --no-locomotion --grabbing # Mixed reality object placer npx @iwsdk/create@latest mr-placer --yes --target ar --physics --scene-understanding --environment-raycast # Desktop 3D game (no headset; locomotion and pointer input are scaffolded) npx @iwsdk/create@latest desktop-game --yes --target browser --physics # Existing cloud-harness repository (explicitly confirm generated-file overwrite) npx @iwsdk/create@latest . --yes --force --target vr ``` --- ## 2. Headless Browser & SwiftShader ### Auto-Detection (0.4.x) When `@iwsdk/vite-plugin-dev` launches its managed Playwright browser, it auto-detects GPU availability first: 1. Probes `/dev/dri` for render nodes. 2. Falls back to **SwiftShader** (Playwright Chromium's bundled software renderer) when no GPU is found. 3. Logs the selected backend every launch. **No manual patching required** in 0.4.x. Use the normal generated app workflow: `npm run dev` or `npx @iwsdk/cli dev restart --open --foreground`. ### Environment Variable Override ```bash IWSDK_GPU=auto # Default — auto-detect IWSDK_GPU=gpu # Force hardware GPU IWSDK_GPU=swiftshader # Force SwiftShader (software rendering) ``` ### Legacy Notes Current 0.4.x apps should not patch `node_modules` or Chromium launch args. If an older 0.3.x harness carried a manual SwiftShader patch, treat that as historical migration context and remove it after upgrading to 0.4.x. ### Playwright Browser Revision Recovery The vite plugin declares `playwright: ^1.58.2`; this repo lockfile currently resolves it to Playwright 1.58.2. If an environment has a mismatched Playwright browser cache, launch can fail with: ``` Executable doesn't exist at …/chromium_headless_shell-1208/… ``` **Preferred fixes:** 1. **Install the matching Playwright browser revision:** ```bash npx playwright@1.58.2 install chromium ``` 2. **Align the project Playwright version with the plugin's resolved version:** ```bash npm install playwright@1.58.2 --save-dev npx playwright install chromium ``` Copying `~/.cache/ms-playwright` revision directories is an emergency cache workaround only; prefer reinstalling the matching browser revision. --- ## 3. Three.js Imports Standard Three.js classes are re-exported from `@iwsdk/core`. For app code, import standard Three classes from `@iwsdk/core` so they match IWSDK's Three/super-three build. Keep the template's `three` dependency/override; import `three/examples/jsm/...` only for addons that `@iwsdk/core` does not re-export. ```typescript // ✅ CORRECT import { Mesh, Group, BoxGeometry, SphereGeometry, CylinderGeometry, PlaneGeometry, ConeGeometry, TorusGeometry, MeshStandardMaterial, MeshBasicMaterial, LineBasicMaterial, Color, Vector3, Quaternion, Euler, Matrix4, Fog, AmbientLight, PointLight, DirectionalLight, BufferGeometry, Float32BufferAttribute, EdgesGeometry, LineSegments, AdditiveBlending, // ... every standard Three.js export is available } from "@iwsdk/core"; // Avoid for standard Three classes in app code // import { Mesh } from "three"; ``` --- ## 4. Key API Surface (0.4.x) ### World Creation ```typescript // XR world (default) const world = await World.create(container); // XR world with options const world = await World.create(container, { xr: { offer: 'once' }, // or 'always' / 'none'; omit xr for the default render: { near: 0.01, far: 200 }, features: { grabbing: true, locomotion: true, physics: true, spatialUI: false, }, }); // Browser-first world (no XR) const world = await World.create(container, { xr: false, render: { near: 0.001, far: 200, camera: { position: [0, 1.6, 0], lookAt: [0, 1.55, -1] }, }, input: { canvasPointerEvents: true }, features: { grabbing: true, locomotion: { browserControls: true }, // WASD/arrows + Space + gamepad; app owns mouselook physics: true, }, }); ``` ### Dual-Runtime Pattern (XR + Browser Fallback) For games that should work both in VR and in the browser: ```typescript const world = await World.create(container, { xr: { offer: 'once' }, input: { canvasPointerEvents: true }, features: { grabbing: true, locomotion: { browserControls: true }, physics: true, }, }); ``` ### Input System ```typescript // InputManager facade (0.4.x) world.input.xr // XRInputManager (controllers, hands) world.input.keyboard // StatefulKeyboard world.input.actions // InputActionManager world.input.browserGamepads // Browser gamepad array // Action-backed input (preferred for reusable intent-based systems) world.input.actions.getAxis2D('locomotion.move'); // { x, y } world.input.actions.getButtonDown('locomotion.jump'); // Built-in action names: // locomotion.move, locomotion.turn, locomotion.jump, // locomotion.teleportAim, locomotion.teleportCommit, // interaction.select // // Some names are reserved for app/user bindings and are not default-bound. // Keyboard (browser-first) world.input.keyboard.getKeyPressed('KeyW'); // held this frame world.input.keyboard.getKeyDown('Space'); // pressed this frame world.input.keyboard.getKeyUp('KeyE'); // released this frame ``` ### ECS-Native Player Rig ```typescript world.playerEntity // Entity wrapping XROrigin Group world.playerHeadEntity // Entity for head space; not a default UI parent world.playerSpaceEntities // { head, raySpaces, gripSpaces, indexTipSpaces } ``` `ScreenSpace` is a browser-viewport HUD and returns to world space in XR. For immersive UI, default to an authored world transform or attach contextual UI to the object it controls. Use a thresholded `Follower` targeting `world.player.head` only when compact global UI must remain discoverable. Reserve direct `world.playerHeadEntity` parenting for tiny, transient, non-interactive markers that require exact view alignment; do not head-lock menus or persistent panels. See https://iwsdk.dev/concepts/spatial-ui/hud.html. ### Asset Management IWSDK loads 3D models as GLTF or GLB. For FBX, OBJ, or another source format, use Blender's **File → Import**, then **File → Export → glTF 2.0**, and register the converted output in the asset manifest. ```typescript // getGLTF returns a fresh scene graph clone by default // (geometries, materials, and animations remain shared) const { scene } = AssetManager.getGLTF('myModel')!; world.scene.add(scene); // Opt into shared instance (old behavior) const { scene } = AssetManager.getGLTF('myModel', { shared: true })!; ``` --- ## 5. Dev Server ```bash npm run dev # Generated apps: dev restart --open --foreground npx @iwsdk/cli dev up # Start dev:runtime through the CLI, usually backgrounded npx @iwsdk/cli dev up --foreground # Stay attached to terminal npx @iwsdk/cli dev down # Stop npx @iwsdk/cli dev restart # Restart npx @iwsdk/cli dev status # Check running state npx @iwsdk/cli dev logs # View recorded background server logs npx @iwsdk/cli dev open # Open in browser ``` Generated apps use bare `iwsdkDev()` and the CLI launches the managed Playwright runtime/editor workspace, registers the MCP WebSocket endpoint, and records runtime state. AI, headed/headless, open, and screenshot behavior are launch-time `iwsdk dev up` flags. The developer owns the headed/headless choice; announce any change before restarting rather than silently changing their workspace. Use `npx @iwsdk/cli dev status` for the resolved `runtimeUrls.local` and `runtimeUrls.network` fields in the JSON result; the generated starter template defaults to `https://localhost:8081/`, but examples or existing apps may use another Vite port. ### Physical Headset Smoke Test Run `npx @iwsdk/cli dev status`, then open one of the URLs in `data.runtimeUrls.network` on a headset connected to the same Wi-Fi network as the development computer. Accept the expected warning for IWSDK's untrusted local certificate. For alternate connection methods and complete instructions, see [Testing Your Experience](/guides/02-testing-experience). ### Node.js Requirement IWSDK 0.4.x requires Node.js satisfying: `>=20.19.0 <21.0.0-0 || >=22.12.0 <23.0.0-0 || >=24.0.0`. If your environment ships an unsupported default (for example Node 18), install a supported Node version with your environment's version manager or platform package and prepend it to `PATH`. ```bash node --version ``` --- ## 6. Adapter Management Project creation syncs MCP adapter configs after dependency installation for the selected AI tools. Sync or manage them explicitly with: ```bash