video-dubbing

name	video-dubbing
description	Translate video/audio content into another language with dubbed voice and synchronized subtitles. Use this skill whenever the user asks to: translate a video, dub a video, generate foreign-language voiceover for a video, add translated subtitles with dubbed audio, or any workflow involving ASR → translate → TTS → video composition.

Video Dubbing & Subtitle Sync

Complete workflow: ASR → translate → TTS → merge audio → burn subtitles

Pipeline

视频 → 抽音轨(16kHz) → ASR(whisper) → AI翻译 → TTS(dub_segments) → 合并音轨 → 烧录硬字幕

Workflow

0. Prerequisite Check

Ensure whisper + moviepy + mlx-audio are installed:

pip install openai-whisper moviepy 2>/dev/null
# mlx_audio for TTS:
pip install mlx-audio 2>/dev/null  # Apple Silicon

1. Extract Audio & Transcribe (ASR)

Extract 16kHz mono audio, then run whisper for accurate per-segment timestamps:

ffmpeg -y -i input.mp4 -ar 16000 -ac 1 -c:a pcm_s16le audio16k.wav

whisper audio16k.wav \
  --model large-v3-turbo \
  --language <source_lang> \
  --output_format srt \
  --output_dir .

Output: audio16k.srt with native timestamps matching the video.

Use large-v3-turbo for best accuracy. If you want even better Chinese recognition, install Qwen3-ASR (pip install qwen-asr) and use scripts/qwen3_asr.py instead.

2. Translate Subtitles (AI does this)

Read the SRT, translate each segment's text into the target language. Output one line per SRT segment, preserving order exactly.

• Count SRT segments first (e.g. 27 segments → 27 translated lines)