zoom-rtms

# Zoom Realtime Media Streams (RTMS) Background reference for live Zoom media pipelines. Prefer `build-zoom-bot` first, then use this skill for stream types, capabilities, and RTMS-specific implementation constraints. # Zoom Realtime Media Streams (RTMS) Expert guidance for accessing live audio, video, transcript, chat, and screen share data from Zoom meetings, webinars, Video SDK sessions, and Zoom Contact Center Voice in real-time. RTMS uses a WebSocket-based protocol with open standards and does not require a meeting bot to capture the media plane. ## Read This First (Critical) RTMS is primarily a **backend media ingestion service**. - Your backend receives and processes live media: **audio, video, screen share, chat, transcript**. - RTMS is not a frontend UI SDK by itself. - Processing is **event-triggered**: backend waits for RTMS start webhook events before stream handling begins. Optional architecture (common): - Add a **Zoom App SDK** frontend for in-client UI/controls. - Stream backend RTMS outputs to frontend via **WebSocket** (or SSE, gRPC, queue workers, etc.). Use RTMS for media/data plane, and use frontend frameworks/Zoom Apps for presentation + user interactions. **Official Documentation**: https://developers.zoom.us/docs/rtms/ **SDK Reference (JS)**: https://zoom.github.io/rtms/js/ **SDK Reference (Python)**: https://zoom.github.io/rtms/py/ **Sample Repository**: https://github.com/zoom/rtms-samples ## Quick Links **New to RTMS? Follow this path:** 1. **[Connection Architecture](concepts/connection-architecture.md)** - Two-phase WebSocket design 2. **[SDK Quickstart](examples/sdk-quickstart.md)** - Fastest way to receive media (recommended) 3. **[Manual WebSocket](examples/manual-websocket.md)** - Full protocol control without SDK 4. **[Media Types](references/media-types.md)** - Audio, video, transcript, chat, screen share **Complete Implementation:** - **[RTMS Bot](examples/rtms-bot.md)** - End-to-end bot implementation guide **Reference:** - **[Lifecycle Flow](concepts/lifecycle-flow.md)** - Complete webhook-to-streaming flow - **[Data Types](references/data-types.md)** - All enums and constants - **[Webhooks](references/webhooks.md)** - Event subscription details - **[Environment Variables](references/environment-variables.md)** - credential modes and runtime knobs - **[Quickstart Notes](references/quickstart.md)** - Secondary quickstart guide - **Integrated Index** - see the section below in this file **Having issues?** - Connection fails -> [Common Issues](troubleshooting/common-issues.md) - Duplicate connections -> [Webhook Gotchas](troubleshooting/common-issues.md#webhook-response-timing) - No audio/video -> [Media Configuration](references/media-types.md) - Start with preflight checks -> [5-Minute Runbook](RUNBOOK.md) ## Supported Products | Product | Webhook Event | Payload ID | App Type | |---------|--------------|------------|----------| | **Meetings** | `meeting.rtms_started` / `meeting.rtms_stopped` | `meeting_uuid` | General App | | **Webinars** | `webinar.rtms_started` / `webinar.rtms_stopped` | `meeting_uuid` (same!) | General App | | **Video SDK** | `session.rtms_started` / `session.rtms_stopped` | `session_id` | Video SDK App | | **Zoom Contact Center Voice** | Product-specific RTMS/ZCC Voice events | Product-specific stream/session identifiers | Contact Center / approved RTMS integration | Once connected, the core signaling/media socket model is shared across products. Meetings, webinars, and Video SDK sessions use the familiar start/stop webhooks. Zoom Contact Center Voice adds its own RTMS/ZCC Voice event family and should be treated as the same transport model with product-specific event payloads. ## RTMS Overview RTMS is a data pipeline that gives your app access to live media from Zoom meetings, webinars, and Video SDK sessions **without participant bots**. Instead of having automated clients join meetings, use RTMS to collect media data directly from Zoom's infrastructure. ### What RTMS Provides | Media Type | Format | Use Cases | |------------|--------|-----------| | **Audio** | PCM (L16), G.711, G.722, Opus | Transcription, voice analysis, recording | | **Video** | H.264, JPG, PNG | Recording, AI vision, thumbnails, active participant selection | | **Screen Share** | H.264, JPG, PNG | Content capture, slide extraction | | **Transcript** | JSON text | Meeting notes, search, compliance | | **Chat** | JSON text | Archive, sentiment analysis | ### March 2026 Protocol Changes - **Zoom Contact Center Voice support**: RTMS now covers Contact Center Voice audio and transcript scenarios. - **Transcript Language Identification control**: transcript media handshakes now support `src_language` and `enable_lid`. Default behavior is LID enabled. Set `enable_lid: false` to force a fixed language. - **Single individual video stream subscription**: RTMS can now stream one participant's camera feed at a time when `data_opt` is set to `VIDEO_SINGLE_INDIVIDUAL_STREAM`. - **Graceful client-initiated shutdown**: backends can send `STREAM_CLOSE_REQ` over the signaling socket and wait for `STREAM_CLOSE_RESP`. - **Media keep-alive tolerance increased**: media socket keep-alive timeout is now **65 seconds**, not 35. ### Two Approaches | Approach | Best For | Complexity | |----------|----------|------------| | **SDK** (`@zoom/rtms`) | Most use cases | Low - handles WebSocket complexity | | **Manual WebSocket** | Custom protocols, other languages | High - full protocol implementation | ## Prerequisites - **Node.js 20.3.0+** (24 LTS recommended) for JavaScript SDK - **Python 3.10+** for Python SDK - Zoom General App (for meetings/webinars) or Video SDK App (for Video SDK) with RTMS feature enabled - Webhook endpoint for RTMS events - Server to receive WebSocket streams > **Need RTMS access?** Post in [Zoom Developer Forum](https://devforum.zoom.us/) requesting RTMS access with your use case. ## Quick Start (SDK - Recommended) ```javascript import rtms from "@zoom/rtms"; // All RTMS start/stop events across products const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"]; // Handle webhook events rtms.onWebhookEvent(({ event, payload }) => { if (!RTMS_EVENTS.includes(event)) return; const client = new rtms.Client(); client.onAudioData((data, timestamp, metadata) => { console.log(`Audio from ${metadata.userName}: ${data.length} bytes`); }); client.onTranscriptData((data, timestamp, metadata) => { const text = data.toString('utf8'); console.log(`${metadata.userName}: ${text}`); }); client.onJoinConfirm((reason) => { console.log(`Joined session: ${reason}`); }); // SDK handles all WebSocket connections automatically // Accepts both meeting_uuid and session_id transparently client.join(payload); }); ``` ## Quick Start (Manual WebSocket) For full control or non-SDK languages, implement the two-phase WebSocket protocol: ```javascript const WebSocket = require('ws'); const crypto = require('crypto'); const RTMS_EVENTS = ['meeting.rtms_started', 'webinar.rtms_started', 'session.rtms_started']; // 1. Generate signature // For meetings/webinars: uses meeting_uuid. For Video SDK: uses session_id. function generateSignature(clientId, idValue, streamId, clientSecret) { const message = `${clientId},${idValue},${streamId}`; return crypto.createHmac('sha256', clientSecret).update(message).digest('hex'); } // 2. Handle webhook app.post('/webhook', (req, res) => { res.status(200).send(); // CRITICAL: Respond immediately! const { event, payload } = req.body; if (RTMS_EVENTS.includes(event)) { connectToRTMS(payload); } }); // 3. Connect to signaling WebSocket function connectToRTMS(payload) { const { server_urls, rtms_stream_id } = payload; // meeting_uuid for meetings/webinars, session_id for Video SDK const idValue = payload.meeting_uuid || payload.session_id; const signature = generateSignature(CLIENT_ID, idValue, rtms_stream_id, CLIENT_SECRET); const signalingWs = new WebSocket(server_urls); signalingWs.on('open', () => { signalingWs.send(JSON.stringify({ msg_type: 1, // Handshake request protocol_version: 1, meeting_uuid: idValue, rtms_stream_id, signature, media_type: 9 // AUDIO(1) | TRANSCRIPT(8) })); }); // ... handle responses, connect to media WebSocket } ``` **See**: [Manual WebSocket Guide](examples/manual-websocket.md) for complete implementation. ## Media Type Bitmask Combine types with bitwise OR: | Type | Value | Description | |------|-------|-------------| | Audio | 1 | PCM audio samples | | Video | 2 | H.264/JPG video frames | | Screen Share | 4 | **Separate from video!** | | Transcript | 8 | Real-time speech-to-text | | Chat | 16 | In-meeting chat messages | | All | 32 | All media types | **Example**: Audio + Transcript = `1 | 8` = `9` ## Critical Gotchas | Issue | Solution | |-------|----------| | **Only 1 connection allowed** | New connections kick out existing ones. Track active sessions! | | **Respond 200 immediately** | If webhook delays, Zoom retries creating duplicate connections | | **Heartbeat mandatory** | Respond to msg_type 12 with msg_type 13, or connection dies | | **Reconnection is YOUR job** | RTMS doesn't auto-reconnect. Media keep-alive tolerance is now about **65s**; signaling remains around **60s** | | **Transcript language drift** | Use `src_language` plus `enable_lid: false` when you want fixed-language transcription instead of automatic language switching | | **Single participant video only** | `VIDEO_SINGLE_INDIVIDUAL_STREAM` supports one participant at a time. A new `VIDEO_SUBSCRIPTION_REQ` overrides the previous selection | | **Graceful close is explicit now** | Use `STREAM_CLOSE_REQ` / `STREAM_CLOSE_RESP` when your backend wants to terminate the stream cleanly | ## Environment Variables ### SDK Environment Variables ```bash # Required - Authentication ZM_RTMS_CLIENT=your_client_id # Zoom OAuth Client ID ZM_RTMS_SECRET=your_client_secret # Zoom OAuth Client Secret # Optional - Webhook server ZM_RTMS_PORT=8080 # Default: 8080 ZM_RTMS_PATH=/webhook # Default: / # Optional - Logging ZM_RTMS_LOG_LEVEL=info # error, warn, info, debug, trace ZM_RTMS_LOG_FORMAT=progressive # progressive or json ZM_RTMS_LOG_ENABLED=true ``` ### Manual Implementation Variables ```bash ZOOM_CLIENT_ID=your_client_id ZOOM_CLIENT_SECRET=your_client_secret ZOOM_SECRET_TOKEN=your_webhook_token # For webhook validation ``` ## Zoom App Setup ### For Meetings and Webinars (General App) 1. Go to [marketplace.zoom.us](https://marketplace.zoom.us) -> Develop -> Build App 2. Choose **General App** -> **User-Managed** 3. Features -> Access -> **Enable Event Subscription** 4. Add Events -> Search "rtms" -> Select: - `meeting.rtms_started` - `meeting.rtms_stopped` - `webinar.rtms_started` (if using webinars) - `webinar.rtms_stopped` (if using webinars) 5. Scopes -> Add Scopes -> Search "rtms" -> Add: - `meeting:read:meeting_audio` - `meeting:read:meeting_video` - `meeting:read:meeting_transcript` - `meeting:read:meeting_chat` - `webinar:read:webinar_audio` (if using webinars) - `webinar:read:webinar_video` (if using webinars) - `webinar:read:webinar_transcript` (if using webinars) - `webinar:read:webinar_chat` (if using webinars) ### For Video SDK (Video SDK App) 1. Go to [marketplace.zoom.us](https://marketplace.zoom.us) -> Develop -> Build App 2. Choose **Video SDK App** 3. Use your SDK Key and SDK Secret (not OAuth Client ID/Secret) 4. Add Events: - `session.rtms_started` - `session.rtms_stopped` ## Sample Repositories ### Official Samples | Repository | Description | |------------|-------------|