Run any Skill in Manus with one click

xb-sound

Stars437

Forks85

UpdatedJune 5, 2026 at 04:54

Play and capture audio in an XR Blocks app via `xb.sound` — spatial (positional) audio anchored to scene objects, master volume control, microphone recording and playback, plus speech recognition and synthesis. Use to add sound effects, 3D-positioned audio, voice recording, or text-to-speech / speech-to-text. Covers `xb.core.sound` (`startRecording`/`stopRecording`/`playRecordedAudio`, `getAudioListener`, `setMasterVolume`) and the exported `SpeechRecognizer` / `SpeechSynthesizer` / `SoundSynthesizer` classes. See samples/sound.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

google

google/xrblocks

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software DevelopersComputer and Mathematical Occupations·SOC 15-1252

SKILL.md

readonly

name

xb-sound

description

xb-sound: spatial audio & speech

xb.sound (a.k.a. xb.core.sound) manages audio. See samples/sound, demos/sound_detector, and src/sound/ (which has its own README).

Spatial (positional) audio

Attach a THREE.PositionalAudio to any object so it pans/attenuates with the listener:

const listener = xb.core.sound.getAudioListener();
const audio = new THREE.PositionalAudio(listener);
audio.setBuffer(audioBuffer); // a decoded AudioBuffer
audio.setRefDistance(0.5);
audio.setVolume(1.0);
ball.add(audio); // parented to the object → 3D positioned
audio.play();

Microphone recording & playback

await xb.core.sound.startRecording();
// …later…
const pcm = xb.core.sound.stopRecording(); // ArrayBuffer of Int16 PCM
const rate = xb.core.sound.getRecordingSampleRate();
await xb.core.sound.playRecordedAudio(pcm, rate);
xb.core.sound.setMasterVolume(0.8); // 0..1

Speech recognition & synthesis

The SDK exports xb.SpeechRecognizer and xb.SpeechSynthesizer (plus xb.SoundSynthesizer and xb.BackgroundMusic). Use them for speech-to-text and text-to-speech; see src/sound/SpeechRecognizer.ts and src/sound/SpeechSynthesizer.ts for their APIs.

Notes

Microphone access prompts for permission; set options.permissions.microphone = true (or use a feature that requests it) before relying on recording.
Recorded data is Int16 PCM; convert to a THREE AudioBuffer for positional playback (see the conversion in samples/sound/main.js).
Call xb.core.sound.disableAudio() to tear down capture when done.

More from this repository

same repository

xrblocks

google/xrblocks

Guide to building AI + XR applications with the XR Blocks SDK (the `xrblocks` package, source in `src/`). Use when writing, editing, or debugging WebXR / Android XR / VR / AR / mixed-reality experiences on this framework — authoring `xb.Script` classes, configuring `xb.Options`, and wiring spatial UI, hand tracking, gestures, depth & occlusion, plane/object detection, physics, spatial audio, or Gemini/OpenAI integration — and when running them in the desktop simulator or on-device. Covers the canonical app skeleton, the real `enable*` option methods, the `Script` lifecycle hooks, the global aliases (`xb.core`, `xb.user`, `xb.world`, `xb.ai`...), and the most common hallucinated-API mistakes to avoid. Read before generating XR Blocks code.

2026-07-08437

xb-simulator

google/xrblocks

Develop and test XR Blocks apps on the desktop without a headset using the built-in simulator — a simulated user, hands, depth, and planes rendered in a normal browser, with control modes for moving the user, posing hands, or driving controllers. Use when running/iterating locally, reproducing XR interactions on desktop, posing hands for gesture work, or adding the optional 2D simulator settings UI. Covers the `?formFactor=desktop` autostart, `options.simulator.*`, `xb.SimulatorMode`, the `SimulatorAddons` 2D UI import, and the `onSimulatorStarted()` hook.

2026-07-06437

xb-testing

google/xrblocks

Write sequential asynchronous functional, integration, or simulator tests for xrblocks apps using the testing addon. Use this when you need to mock WebGL/WebAudio in headless environments (like JSDOM / Vitest), simulate user locomotion, trigger controller pointing, raycasts, and select/squeeze hand inputs. Covers `TestRunner` and `TestRunnerConfig`.

2026-06-20437

uiblocks

google/xrblocks

Build rich spatial user interfaces in XR Blocks apps with the uiblocks addon — flexbox-laid-out 3D cards and panels with gradients, strokes, rounded corners, drop/inner shadows, MSDF text, material icons, images, and spatial behaviors (head-leash, billboard, grab/manipulate, object-anchor, show/hide animations). Use when authoring or debugging `UICard` / `UIPanel` / `UIText` / `UIImage` / `UIIcon` UI imported from `xrblocks/addons/uiblocks/src` (wrapping `@pmndrs/uikit` + yoga-layout) — for panels, menus, HUDs, dialogs, or any styled, web-like UI in WebXR / Android XR. Includes the bootstrap (`options.uikit.enable(uikit)` + `raycastSortFunction`), styling/layout rules, behavior config, and a troubleshooting playbook for clicks/styling/sizing failures. (For lightweight panels with no extra deps, prefer core `xb.SpatialPanel` instead.) Includes a design guide (§6) for UX designers composing complex, elegant, multi-section spatial UI — tokens, spatial comfort, elevation/shadows, passthrough legibility, motion, a

2026-06-17437

netblocks

google/xrblocks

Add real-time multiplayer to an XR Blocks app with the netblocks addon — presence avatars (remote heads + hands), replicated `NetObject` transforms with cooperative ownership, typed pub/sub RPC events, and opt-in spatial WebRTC voice, over pluggable transports (BroadcastChannel for local dev, WebRTC/PeerJS for serverless P2P, WebSocket relay for scalable rooms). Use when authoring or debugging shared/co-located XR, `enableNet()`, `joinRoom()`, `NetObject`, `session.events`, or `session.voice` imported from `xrblocks/addons/netblocks/src`. For the wire protocol, transport details, and threat model, read the full reference in this folder's README.md.

2026-06-09437

xb-lipsync

google/xrblocks

Add audio-driven avatar mouths to an XR Blocks app with the lipsync addon — heuristic vowel-formant viseme mapping that turns any `MediaStream` (mic or remote peer's voice) into mouth shapes on a `StylizedFace` decal attached to an avatar's head. Zero ML runtime, no model download. Use when you want shared rooms to stop being silent spheres and become faces that visibly speak, or to lip-sync a TTS playback to an NPC. Covers `LipsyncMouth`, `xb.StylizedFace`, the `target`/`audioContext`/`fftSize` constructor options, and the `session.voice.onTrack` netblocks pairing. Lower-level pieces (`FormantVisemeMapper`, `MfccExtractor`, `computeAudioFeatures`) and types (`VisemeWeights`, `VisemeTarget`) are exported for swapping in a model-based mapper. Full reference at src/addons/lipsync/.

2026-06-06437

name

xb-sound

description

xb-sound: spatial audio & speech

xb.sound (a.k.a. xb.core.sound) manages audio. See samples/sound, demos/sound_detector, and src/sound/ (which has its own README).

Spatial (positional) audio

Attach a THREE.PositionalAudio to any object so it pans/attenuates with the listener:

const listener = xb.core.sound.getAudioListener();
const audio = new THREE.PositionalAudio(listener);
audio.setBuffer(audioBuffer); // a decoded AudioBuffer
audio.setRefDistance(0.5);
audio.setVolume(1.0);
ball.add(audio); // parented to the object → 3D positioned
audio.play();

Microphone recording & playback

await xb.core.sound.startRecording();
// …later…
const pcm = xb.core.sound.stopRecording(); // ArrayBuffer of Int16 PCM
const rate = xb.core.sound.getRecordingSampleRate();
await xb.core.sound.playRecordedAudio(pcm, rate);
xb.core.sound.setMasterVolume(0.8); // 0..1

Speech recognition & synthesis

Notes

Microphone access prompts for permission; set options.permissions.microphone = true (or use a feature that requests it) before relying on recording.
Recorded data is Int16 PCM; convert to a THREE AudioBuffer for positional playback (see the conversion in samples/sound/main.js).
Call xb.core.sound.disableAudio() to tear down capture when done.