azure-speech

name	azure-speech
description	Expert knowledge for Azure AI Speech development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when building STT/TTS, custom speech or voice models, Voice Live/avatars, batch jobs, or telephony voice agents, and other Azure AI Speech related development tasks. Not for Azure Communication Services (use azure-communication-services), Azure AI Bot Service (use azure-bot-service), Azure AI Video Indexer (use azure-video-indexer), Azure Translator (use azure-translator).
compatibility	Requires network access. Uses mcp_microsoftdocs:microsoft_docs_fetch or fetch_webpage to retrieve documentation.
metadata	{"generated_at":"2026-04-19","generator":"docs2skills/1.0.0"}

Azure AI Speech Skill

This skill provides expert guidance for Azure AI Speech. Covers troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. It combines local quick-reference content with remote documentation fetching capabilities.

How to Use This Skill

IMPORTANT for Agent: Use the Category Index below to locate relevant sections. For categories with line ranges (e.g., L35-L120), use read_file with the specified lines. For categories with file links (e.g., [security.md](security.md)), use read_file on the linked reference file

IMPORTANT for Agent: If metadata.generated_at is more than 3 months old, suggest the user pull the latest version from the repository. If mcp_microsoftdocs tools are not available, suggest the user install it: Installation Guide

This skill requires network access to fetch documentation content:

• Preferred: Use mcp_microsoftdocs:microsoft_docs_fetch with query string from=learn-agent-skill. Returns Markdown.
• Fallback: Use fetch_webpage with query string from=learn-agent-skill&accept=text/markdown. Returns Markdown.

Category Index

Category	Lines	Description
Troubleshooting	L37-L47	Diagnosing and fixing common Azure Speech/Text-to-Speech/Voice Live API and SDK errors, container and Foundry issues, CRL/compatibility problems, and retrieving session/transcription IDs for support.
Best Practices	L48-L64	Best practices for audio/video prep, custom voice/avatars, latency and memory tuning, phrase/keyword optimization, and handling real-time Voice Live interactions and interruptions
Decision Making	L65-L82	Guidance on choosing speech features (batch STT, custom/embedded/personal/Whisper), evaluating models/devices, and step‑by‑step migration between Speech API versions and services
Architecture & Design Patterns	L83-L87	Architectural guidance for building call center voice agents using Azure AI Speech with Voice Live and Azure Communication Services, including integration patterns and design best practices.
Limits & Quotas	L88-L96	Quotas, limits, and usage patterns for Azure Speech: batch TTS, custom/pro voice training & deployment, and short audio STT, plus throttling and capacity planning guidance.
Security	L97-L108	Configuring security for Azure AI Speech: auth (Entra, RBAC), network isolation (VNet, Private Link, sovereign clouds), BYOS storage, encryption/keys, and voice talent consent management.
Configuration	L109-L143	Configuring Azure AI Speech/Voice: audio inputs, logging, storage, SSML, languages/voices, custom speech & voice training, batch/real-time settings, and Voice Live/avatars options.
Integrations & Coding Patterns	L144-L168	Patterns and APIs for integrating Azure Speech into apps and voice agents: telephony, SDK/REST, TTS/avatars, translation, OpenAI/Foundry, Voice Live, consent, and automation workflows.
Deployment	L169-L180	Deploying and scaling Azure AI Speech: Docker/Kubernetes containers, on-prem STT/TTS, custom speech models/endpoints, language ID, and batch/long-form synthesis workflows.

Troubleshooting

Topic	URL
Troubleshoot common Azure text to speech issues	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/faq-tts
Retrieve Speech to text session and transcription IDs for support	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-get-speech-session-id
Resolve common Azure Speech in Foundry issues	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/known-issues
Resolve Azure AI Speech SDK CRL compatibility issues	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/migrate-to-sdk-1-48-2
Troubleshoot Speech service container deployments	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-faq
Troubleshoot common Azure Speech SDK issues	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/troubleshooting
Resolve common Voice Live API issues in Speech	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live-faq

Best Practices

Topic	URL
Prepare and locate audio data for batch transcription	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/batch-transcription-audio-data
Create high-quality human-labeled speech transcriptions	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-human-labeled-transcriptions
Prepare training data for professional custom voice	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-voice-training-data
Apply best practices to reduce Speech synthesis latency	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-lower-speech-synthesis-latency
Track and manage Azure Speech SDK memory usage	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-track-speech-sdk-memory-usage
Handle user interruptions and chat truncation in Voice Live	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-voice-live-auto-truncation
Use interim responses in Voice Live to reduce latency gaps	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-voice-live-interim-response
Improve speech recognition with phrase lists	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/improve-accuracy-phrase-list
Apply keyword recognition design and accuracy guidelines	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/keyword-recognition-guidelines
Record high-quality samples for custom voice training

Decision Making

Topic	URL
Plan large-scale transcription with batch processing	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/batch-transcription
Evaluate custom voice lite before professional voice	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/custom-neural-voice-lite
Choose Embedded Speech for offline and hybrid scenarios	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/embedded-speech
Evaluate device suitability for embedded speech models	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/embedded-speech-performance-evaluations
Evaluate and compare custom speech model accuracy	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-inspect-data
Migrate Speech to text REST API from v3.2 to 2024-11-15	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/migrate-2024-11-15
Migrate Speech to text REST API from 2024-11-15 to 2025-10-15	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/migrate-2025-10-15
Migrate from retired Speech intent recognition to Language or OpenAI	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/migrate-intent-recognition
Migrate from Long Audio API to Batch synthesis	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/migrate-to-batch-synthesis
Migrate from v3 text-to-speech to custom voice REST API	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/migrate-to-custom-voice-api

Architecture & Design Patterns

Topic	URL
Design call center voice agents with Voice Live and ACS	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live-telephony

Limits & Quotas

Topic	URL
Manage custom speech model and endpoint lifecycle	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-model-and-endpoint-lifecycle
Deploy professional voice models to custom endpoints	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/professional-voice-deploy-endpoint
Train professional voice models and understand duration	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/professional-voice-train-voice
Use Speech-to-text REST API for short audio	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/rest-speech-to-text-short
Apply Azure Speech quotas, limits, and throttling guidance	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-services-quotas-and-limits

Security

Topic	URL
Configure BYOS storage for Azure Speech resources	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/bring-your-own-storage-speech-resource
Configure Microsoft Entra authentication for Speech SDK	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-configure-azure-ad-auth
Manage voice talent consent for professional voice	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/professional-voice-create-consent
Assign Azure RBAC roles for Speech resources	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/role-based-access-control
Use Azure Speech service in sovereign clouds	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/sovereign-clouds
Manage Speech service data-at-rest encryption and keys	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-encryption-of-data-at-rest
Secure Speech service with Virtual Network service endpoints	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-service-vnet-service-endpoint
Secure Azure AI Speech with Private Link endpoints	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-services-private-link

Configuration

Topic	URL
Configure Batch synthesis properties for text-to-speech	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/batch-synthesis-properties
Check status and retrieve batch transcription results	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/batch-transcription-get
Configure BYOS storage for Speech to text	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/bring-your-own-storage-speech-resource-speech-to-text
Define UPS phonetic pronunciations for Speech to text	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/customize-pronunciation
Configure OpenSSL on Linux for Azure Speech SDK	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-configure-openssl-linux
Control and monitor Speech SDK service connections	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-control-connections
Create and manage custom speech fine-tuning projects	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-create-project
Prepare and upload datasets for custom speech training	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-upload-data
Configure post-processing options for Speech recognition	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-post-processing
Configure real-time speech recognition inputs and options	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-recognize-speech

Integrations & Coding Patterns

Topic	URL
Integrate Speech service with call center telephony	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/call-center-telephony-integration
Call Azure Speech fast transcription API in Foundry	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/fast-transcription-create
Use Speech SDK APIs to handle recognition results	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/get-speech-recognition-results
Integrate custom models with Voice Live BYOM	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-bring-your-own-model
Implement text-to-speech synthesis with Speech SDK	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-speech-synthesis
Implement speech translation with Azure Speech SDK	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-translate-speech
Build real-time voice agents with Voice Live and Foundry Agent Service	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-voice-agent-integration
Connect MCP servers to Azure Voice Live sessions	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-voice-live-mcp-server
Add proactive greetings to Voice Live agents	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-voice-live-proactive-messages
Call the LLM-speech API for transcription and translation	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/llm-speech

Deployment

Topic	URL
Use Batch synthesis API for long-form text-to-speech	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/batch-synthesis
Deploy custom speech models and endpoints	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-deploy-model
Scale Speech containers with batch processing kit	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-batch-processing
Run custom speech to text containers with Docker	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-cstt
Deploy and run Speech containers with Docker	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-howto
Run Speech containers on Kubernetes with Helm	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-howto-on-premises
Deploy language identification containers with Docker	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-lid
Deploy neural text to speech containers with Docker	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-ntts
Deploy speech to text containers for on-premises use	https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-container-stt