microsoft-vibevoice

name	microsoft--vibevoice
description	Microsoft VibeVoice — open-source Voice AI: ASR 7B (60-min audio 1 pass, 50+ ngôn ngữ), TTS 1.5B (90-min, 4 speakers), Realtime 0.5B (300ms latency). HuggingFace.
allowed-tools	Bash, Read, Write
user-invocable	true

VibeVoice: bộ 3 model voice AI open-source của Microsoft Research — ASR dài hạn, TTS đa người nói, Realtime streaming.

3 Models

VibeVoice-ASR (7B)
  - 60-minute long-form audio trong 1 pass
  - Speaker identification + timestamps
  - 50+ languages + custom hotwords

VibeVoice-TTS (1.5B)
  - 90-minute speech synthesis
  - Tối đa 4 distinct speakers
  - English, Chinese, multilingual

VibeVoice-Realtime (0.5B)
  - Streaming với ~300ms latency
  - Robust lên đến 10 phút
  - Lightweight deployment

Architecture

Continuous Speech Tokenizers
  Acoustic tokenizer  — audio quality preservation
  Semantic tokenizer  — contextual understanding
  Frame rate: 7.5 Hz  — ultra-low, efficient cho long sequences

Generation:
  LLM — understand text context + dialogue flow
  Diffusion head — generate high-fidelity acoustic details

Access

# HuggingFace
from transformers import pipeline

# ASR
asr = pipeline("automatic-speech-recognition",
               model="microsoft/VibeVoice-ASR-7B")
result = asr("long_audio.wav")

# TTS
tts = pipeline("text-to-speech",
               model="microsoft/VibeVoice-TTS-1.5B")
audio = tts("Hello, this is VibeVoice.")

Playgrounds

• ASR: aka.ms/vibevoice-asr
• Colab notebooks: HuggingFace model pages
• Interactive demo: HuggingFace Spaces

Khi nào dùng

• Transcribe meeting/podcast dài (>30 min) với speaker diarization
• TTS multi-speaker cho audiobook, dialog synthesis
• Real-time voice interface với latency thấp
• Alternative cho Whisper khi cần long-form hoặc multi-speaker

Source

https://github.com/microsoft/VibeVoice · MIT · +216⭐ today