Skip to main content Skills Marketplace Discover and explore AI skills built by the community.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Show prompt details A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/BEKO2210/Firstbrain --skill azure-ai-voicelive-tsThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Download Zip Downloading... More from this repository Related occupations SOC
Based on SOC occupation classification
name azure-ai-voicelive-ts description Azure AI Voice Live SDK for JavaScript/TypeScript. Build real-time voice AI applications with bidirectional WebSocket communication. type skill created 2026-02-27T00:00:00.000Z domain cloud-infrastructure category azure risk unknown source community tags ["skill","cloud-infrastructure","azure","voicelive"]
@azure/ai-voicelive (JavaScript/TypeScript)
Real-time voice AI SDK for building bidirectional voice assistants with Azure AI in Node.js and browser environments.
Installation
npm install @azure/ai-voicelive @azure/identity
npm install @types/node
Current Version : 1.0.0-beta.3
Supported Environments :
Node.js LTS versions (20+)
Modern browsers (Chrome, Firefox, Safari, Edge)
Environment Variables
AZURE_VOICELIVE_ENDPOINT=https://<resource>.cognitiveservices.azure.com
AZURE_VOICELIVE_API_KEY=<your-api-key>
AZURE_LOG_LEVEL=info
Authentication
Microsoft Entra ID (Recommended)
import { DefaultAzureCredential } from "@azure/identity" ;
import { VoiceLiveClient } from "@azure/ai-voicelive" ;
const credential = new DefaultAzureCredential ();
const endpoint = "https://your-resource.cognitiveservices.azure.com" ;
const client = new VoiceLiveClient (endpoint, credential);
API Key
import { AzureKeyCredential } from "@azure/core-auth" ;
import { VoiceLiveClient } from "@azure/ai-voicelive" ;
const endpoint = "https://your-resource.cognitiveservices.azure.com" ;
const credential = new AzureKeyCredential
"your-api-key"
const
new
VoiceLiveClient
Client Hierarchy VoiceLiveClient
└── VoiceLiveSession (WebSocket connection)
├── updateSession() → Configure session options
├── subscribe() → Event handlers (Azure SDK pattern)
├── sendAudio() → Stream audio input
├── addConversationItem() → Add messages/function outputs
└── sendEvent() → Send raw protocol events
Quick Start import { DefaultAzureCredential } from "@azure/identity" ;
import { VoiceLiveClient } from "@azure/ai-voicelive" ;
const credential = new DefaultAzureCredential ();
const endpoint = process.env .AZURE_VOICELIVE_ENDPOINT !;
const client = new VoiceLiveClient (endpoint, credential);
const session = await client.startSession ("gpt-4o-mini-realtime-preview" );
await session.updateSession ({
modalities : ["text" , "audio" ],
instructions : "You are a helpful AI assistant. Respond naturally." ,
voice : {
type : "azure-standard" ,
name : "en-US-AvaNeural" ,
},
turnDetection : {
type : "server_vad" ,
threshold : 0.5 ,
prefixPaddingMs : 300 ,
silenceDurationMs : 500 ,
},
inputAudioFormat : "pcm16" ,
outputAudioFormat : "pcm16" ,
});
const subscription = session.subscribe ({
onResponseAudioDelta : async (event, context) => {
const audioData = event.delta ;
playAudioChunk (audioData);
},
onResponseTextDelta : async (event, context) => {
process.stdout .write (event.delta );
},
onInputAudioTranscriptionCompleted : async (event, context) => {
console .log ("User said:" , event.transcript );
},
});
function sendAudioChunk (audioBuffer : ArrayBuffer sendAudio (audioBuffer);
}
Session Configuration await session.updateSession ({
modalities : ["audio" , "text" ],
instructions : "You are a customer service representative." ,
voice : {
type : "azure-standard" ,
name : "en-US-AvaNeural" ,
},
turnDetection : {
type : "server_vad" ,
threshold : 0.5 ,
prefixPaddingMs : 300 ,
silenceDurationMs : 500 ,
},
inputAudioFormat : "pcm16" ,
outputAudioFormat : "pcm16" ,
tools : [
{
type : "function" ,
name : "get_weather" ,
description : "Get current weather" ,
parameters : {
type : "object" ,
properties : {
location : { type : "string" }
},
required : ["location" ]
}
}
],
toolChoice : "auto" ,
});
Event Handling (Azure SDK Pattern) The SDK uses a subscription-based event handling pattern:
const subscription = session.subscribe ({
onConnected : async (args, context) => {
console .log ("Connected:" , args.connectionId );
},
onDisconnected : async (args, context) => {
console .log ("Disconnected:" , args.code , args.reason );
},
onError : async (args, context) => {
console .error ("Error:" , args.error .message );
},
onSessionCreated : async (event, context) => {
console .log ("Session created:" , context.sessionId );
},
onSessionUpdated : async (event, context) => {
console .log ("Session updated" );
},
onInputAudioBufferSpeechStarted : async (event, context) => {
console .log ("Speech started at:" , event.audioStartMs );
},
onInputAudioBufferSpeechStopped : async (event, context) => {
console .log ("Speech stopped at:" , event.audioEndMs );
},
onConversationItemInputAudioTranscriptionCompleted : async (event, context) => {
console .log ("User said:" , event.transcript );
},
onConversationItemInputAudioTranscriptionDelta : async (event, context) => {
process.stdout .write (event.delta );
},
onResponseCreated : async (event, context) => {
console .log ("Response started" );
},
onResponseDone : async (event, context) => {
console .log ("Response complete" );
},
onResponseTextDelta : async (event, context) => {
process.stdout .write (event.delta );
},
onResponseTextDone : async (event, context) => {
console .log ("\n--- Text complete ---" );
},
onResponseAudioDelta : async (event, context) => {
const audioData = event.delta ;
playAudioChunk (audioData);
},
onResponseAudioDone : async (event, context) => {
console .log ("Audio complete" );
},
onResponseAudioTranscriptDelta : async (event, context) => {
process.stdout .write (event.delta );
},
onResponseFunctionCallArgumentsDone : async (event, context) => {
if (event.name === "get_weather" ) {
const args = JSON .parse (event.arguments );
const result = await getWeather (args.location );
await session.addConversationItem ({
type : "function_call_output" ,
callId : event.callId ,
output : JSON .stringify (result),
});
await session.sendEvent ({ type : "response.create" });
}
},
onServerEvent : async (event, context) => {
console .log ("Event:" , event.type );
},
});
await subscription.close ();
Function Calling
await session.updateSession ({
modalities : ["audio" , "text" ],
instructions : "Help users with weather information." ,
tools : [
{
type : "function" ,
name : "get_weather" ,
description : "Get current weather for a location" ,
parameters : {
type : "object" ,
properties : {
location : {
type : "string" ,
description : "City and state or country" ,
},
},
required : ["location" ],
},
},
],
toolChoice : "auto" ,
});
const subscription = session.subscribe ({
onResponseFunctionCallArgumentsDone : async (event, context) => {
if (event.name === "get_weather" ) {
const args = JSON .parse (event.arguments );
const weatherData = await fetchWeather (args.location );
await session.addConversationItem ({
type : "function_call_output" ,
callId : event.callId ,
output : JSON .stringify (weatherData),
});
await session.sendEvent ({ type : "response.create" });
}
},
});
Voice Options Voice Type Config Example Azure Standard { type: "azure-standard", name: "..." }"en-US-AvaNeural"Azure Custom { type: "azure-custom", name: "...", endpointId: "..." }Custom voice endpoint Azure Personal { type: "azure-personal", speakerProfileId: "..." }Personal voice clone OpenAI { type: "openai", name: "..." }"alloy", "echo", "shimmer"
Supported Models Model Description Use Case gpt-4o-realtime-previewGPT-4o with real-time audio High-quality conversational AI gpt-4o-mini-realtime-previewLightweight GPT-4o Fast, efficient interactions phi4-mm-realtimePhi multimodal Cost-effective applications
Turn Detection Options
turnDetection : {
type : "server_vad" ,
threshold : 0.5 ,
prefixPaddingMs : 300 ,
silenceDurationMs : 500 ,
}
turnDetection : {
type : "azure_semantic_vad" ,
}
turnDetection : {
type : "azure_semantic_vad_en" ,
}
turnDetection : {
type : "azure_semantic_vad_multilingual" ,
}
Audio Formats Format Sample Rate Use Case pcm1624kHz Default, high quality pcm16-8000hz8kHz Telephony pcm16-16000hz16kHz Voice assistants g711_ulaw8kHz Telephony (US) g711_alaw8kHz Telephony (EU)
Key Types Reference Type Purpose VoiceLiveClientMain client for creating sessions VoiceLiveSessionActive WebSocket session VoiceLiveSessionHandlersEvent handler interface VoiceLiveSubscriptionActive event subscription ConnectionContextContext for connection events SessionContextContext for session events ServerEventUnionUnion of all server events
Error Handling import {
VoiceLiveError ,
VoiceLiveConnectionError ,
VoiceLiveAuthenticationError ,
VoiceLiveProtocolError ,
} from "@azure/ai-voicelive" ;
const subscription = session.subscribe ({
onError : async (args, context) => {
const { error } = args;
if (error instanceof VoiceLiveConnectionError ) {
console .error ("Connection error:" , error.message );
} else if (error instanceof VoiceLiveAuthenticationError ) {
console .error ("Auth error:" , error.message );
} else if (error instanceof VoiceLiveProtocolError ) {
console .error ("Protocol error:" , error.message );
}
},
onServerError : async (event, context) => {
console .error ("Server error:" , event.error ?.message );
},
});
Logging import { setLogLevel } from "@azure/logger" ;
setLogLevel ("info" );
Browser Usage
import { VoiceLiveClient } from "@azure/ai-voicelive" ;
import { InteractiveBrowserCredential } from "@azure/identity" ;
const credential = new InteractiveBrowserCredential ({
clientId : "your-client-id" ,
tenantId : "your-tenant-id" ,
});
const client = new VoiceLiveClient (endpoint, credential);
const stream = await navigator.mediaDevices .getUserMedia ({ audio : true });
const audioContext = new AudioContext ({ sampleRate : 24000 });
Best Practices
Always use DefaultAzureCredential — Never hardcode API keysSet both modalities — Include ["text", "audio"] for voice assistantsUse Azure Semantic VAD — Better turn detection than basic server VADHandle all error types — Connection, auth, and protocol errorsClean up subscriptions — Call subscription.close() when doneUse appropriate audio format — PCM16 at 24kHz for best quality
Reference Links
When to Use This skill is applicable to execute the workflow or actions described in the overview.
Connections
Domain: [[Cloud & Infrastruktur]]Kategorie: [[Microsoft Azure]]Navigation: [[Skills Uebersicht]], [[Home]] 