name	voice-spatial-audio
description	Use when capturing the expert's voice during a masterclass recording, or replaying it as spatialised audio in Ankhora — microphone capture, timeline-aligned storage, and 3D playback via the Meta XR Audio spatialiser. Triggers: voice, microphone, record audio, narration, spatial audio, Meta XR Audio, spatialise, 3D sound, audio playback, mic permission, audio timeline.

Voice & Spatial Audio (Meta XR Audio)

Voice is half of an Ankhora masterclass (voice + ghost hands). This skill covers capturing the expert's narration during recording and replaying it as spatialised audio coming from where the expert "was", in sync with the ghost hands.

Verify the API first. Microphone capture uses Unity's Microphone API; spatialisation uses the Meta XR Audio plugin (MetaXRAudioSource / the Meta XR Audio spatialiser). Confirm the current component names and the spatialiser plugin setting via context7/Meta docs before coding — names differ across SDK versions.

Capture (during recording)

Request the mic permission: android.permission.RECORD_AUDIO (via meta_update_android_manifest).
Record with Unity Microphone.Start; capture a timestamp at record-start so the audio track aligns with the pose/event timeline (see record-replay-contract).
Store the clip alongside the masterclass data (compressed — Vorbis/AAC; raw PCM is heavy). Keep the audio ↔ timeline offset in the contract so replay stays in sync.

Replay (spatialised)

Set the project audio spatialiser to Meta XR Audio (Project Settings → Audio), and the AudioSource spatialBlend to 3D with the Meta spatial component.
Position the audio source at the expert's recorded head/mouth pose each frame (sampled from the timeline) so the narration appears to come from the right place in the room.
Drive playback from the same replay clock as the ghost hands — never two independent clocks.

Keep it testable

The sync logic (audio offset, timeline → playback position) is plain C# math → EditMode-test it (see unity-testability). Only Microphone / AudioSource calls touch Unity.

Verify

On device / simulator: record a short clip, replay, confirm voice is in sync with hands and spatially placed. Mac Editor can play audio but not hand tracking — verify sync on device and say so.
Unity_ReadConsole: no audio/spatialiser errors; confirm the Meta spatialiser is the active plugin.

Out of scope

Noise suppression, voice cloning, real-time TTS, multi-track mixing — V2. MVP = one narration track per masterclass, captured and spatialised.

Voice & Spatial Audio (Meta XR Audio)

Verify the API first. Microphone capture uses Unity's Microphone API; spatialisation uses the Meta XR Audio plugin (MetaXRAudioSource / the Meta XR Audio spatialiser). Confirm the current component names and the spatialiser plugin setting via context7/Meta docs before coding — names differ across SDK versions.

Capture (during recording)

Request the mic permission: android.permission.RECORD_AUDIO (via meta_update_android_manifest).

Record with Unity Microphone.Start; capture a timestamp at record-start so the audio track aligns with the pose/event timeline (see record-replay-contract).

Store the clip alongside the masterclass data (compressed — Vorbis/AAC; raw PCM is heavy). Keep the audio ↔ timeline offset in the contract so replay stays in sync.

Replay (spatialised)

Set the project audio spatialiser to Meta XR Audio (Project Settings → Audio), and the AudioSource spatialBlend to 3D with the Meta spatial component.

Position the audio source at the expert's recorded head/mouth pose each frame (sampled from the timeline) so the narration appears to come from the right place in the room.

Drive playback from the same replay clock as the ghost hands — never two independent clocks.

Keep it testable

The sync logic (audio offset, timeline → playback position) is plain C# math → EditMode-test it (see unity-testability). Only Microphone / AudioSource calls touch Unity.

Verify

On device / simulator: record a short clip, replay, confirm voice is in sync with hands and spatially placed. Mac Editor can play audio but not hand tracking — verify sync on device and say so.

Unity_ReadConsole: no audio/spatialiser errors; confirm the Meta spatialiser is the active plugin.

Out of scope

Noise suppression, voice cloning, real-time TTS, multi-track mixing — V2. MVP = one narration track per masterclass, captured and spatialised.

voice-spatial-audio