video-processor

name	video-processor
version	2.1.0
author	zanearcher
category	media
description	Comprehensive media processing: transcribe, translate, summarize, and dub videos/audio with professional TTS. Supports local files (MP4, MP3, WAV, M4A, etc.) and URLs (YouTube, Twitter, TikTok, 1000+ sites). Use when user says /video-transcribe, /video-translate, /video-dub, /video-summary, or asks to transcribe, translate, dub, or summarize any video/audio.
user_invocable	true

Video/Audio Processor Skill

Comprehensive media processing: transcribe, translate, summarize, and dub videos/audio with professional TTS.

Architecture: Groq Whisper for ASR. Translation and visual analysis use the host LLM by default (higher quality, context-aware). Fallbacks: Groq LLM (llama-3.3-70b) for translation, video_captioner.py (MLX VLM, Qwen2.5-VL-3B) for visual analysis. Condensation, summarization, and filler cleanup are handled by the host agent.

Supports:

• Video files: MP4, MKV, AVI, MOV, WebM, FLV
• Audio files: MP3, M4A, WAV, FLAC, OGG, AAC
• URLs: YouTube, Twitter/X, TikTok, Instagram, and 1000+ sites via yt-dlp

Activation

Use this skill when the user:

• Says /video-transcribe, /video-translate, /video-dub, /video-summary, or /video-caption
• Asks to "transcribe this video"
• Wants to "translate and dub a video"
• Needs "video summary" or "summarize this video"
• Requests "extract transcript from video"
• Asks to "caption this video" or "add word-level captions"
• Asks to "analyze this video", "what's happening in this video", "describe this silent video"
• Asks in ANY language (e.g., "总结这段视频", "résumez cette vidéo", "resume este video")

Language Detection:

• Automatically detects the language of the user's request
• Generates output in the same language as the request
• Example: "总结这段视频" → Summary in Chinese
• Example: "summarize this video" → Summary in English
• Can be overridden with explicit language parameter

Modes

1. Transcription Only

Extract transcript with timestamps from video.

Triggers: /video-transcribe, "transcribe this video"

Pipeline:

1. Run: python3 scripts/transcriber.py <video_file_or_url> [groq_api_key]
2. Output: {name}_original.srt + {name}_transcript.txt

2. Translation

Transcribe + translate to target language (no TTS, subtitles only).

Triggers: /video-translate, "translate this video to {lang}"

Pipeline:

1. Run: python3 scripts/transcriber.py <video_file_or_url> [groq_api_key]
2. Run: python3 scripts/clean_srt.py {name}_original.srt --in-place (removes filler words)
3. Run: python3 scripts/translate_srt.py {name}_original.srt <target_lang>
4. Agent: Show side-by-side review to user

Output:

• {name}_original.srt - Original transcript (cleaned)
• {name}_{target_lang}.srt - Translated subtitles

3. Dubbing (Full Pipeline)

Transcribe, translate, review, TTS, create dubbed video.

Triggers: /video-dub, "dub this video to {lang}"

Audio Mix Modes:

• mix (default): Original audio lowered to ~15% + dubbed voice at full volume. Professional dubbing style — preserves ambiance, emotion, room tone underneath. Like Netflix/Disney+ dubs.
• replace: Only dubbed audio, no original track. Use when user explicitly asks for "clean dub", "no original audio", or "replace audio completely".

Subtitle Burning:

• Default: no burned subtitles. SRT files are output separately.
• Set BURN_SUBS=yes only when user explicitly asks for "burned-in subtitles", "hardcoded subs", or "bake subtitles in".

Agent behavior during dubbing:

• Tell the user: "I'll mix the dubbed voice over the original audio (lowered to ~15%) — this keeps the natural ambiance. No subtitles will be burned in; SRT files are available separately. Let me know if you want clean replacement audio or burned-in subs instead."
• Only switch to MIX_MODE=replace or BURN_SUBS=yes if the user explicitly requests it.

Pipeline:

1. Run: python3 scripts/transcriber.py <video_file_or_url> [groq_api_key] → {name}_original.srt
2. Run: python3 scripts/clean_srt.py {name}_original.srt --in-place (removes filler words)
3. Run: python3 scripts/translate_srt.py {name}_original.srt <target_lang> → {name}_{lang}.srt
4. Agent: Show translation review to user, apply any corrections
5. Run: bash scripts/generate_tts_and_dub.sh <video> <orig.srt> <trans.srt> <lang> [voice] [voice_name]
   • Env vars: MIX_MODE=mix (default), MIX_ORIGINAL_VOLUME=0.15, BURN_SUBS=no (default)
   • For clean replacement: MIX_MODE=replace bash scripts/generate_tts_and_dub.sh ...
   • For burned subs: BURN_SUBS=yes bash scripts/generate_tts_and_dub.sh ...
6. Agent: Read {name}_timing_report.json (see Agent Condensation Protocol)
7. If overlong segments: agent condenses → deletes old raw/adj files → re-runs with WORK_DIR set
8. Output: {name}_dubbed.mp4

When user requests bilingual captions on a dubbed video: Use caption_video.py in plain mode (default). One SRT entry = one caption line. No karaoke, no wrapping. Because the SRTs were sentence-split upstream by clean_srt.py, each caption is already one meaningful sentence. No Groq re-transcription needed.

Example (dubbed Chinese main + English below):

python3 scripts/caption_video.py dubbed.mp4 \
  --srt=video_chinese.srt --translation-srt=video_original.srt \
  --output=video_final.mp4

If the user explicitly asks for karaoke/word-level highlighting, use --style=highlight (or appear/bounce/etc.) which triggers word-level transcription and the fancier animated styles.

4. Summary

Transcribe video and generate comprehensive summary.

Triggers:

• /video-summary, "summarize this video" (English)
• "总结这段视频" (Chinese)
• "résumez cette vidéo" (French)
• "resume este video" (Spanish)
• Any natural language request

Pipeline:

1. Run: python3 scripts/transcriber.py <video_file_or_url> [groq_api_key]
2. Agent: Read transcript, generate summary (see Agent Summary Protocol)
3. Output: {name}_summary.md

5. Captioning

Burn captions into a video. Default is plain sentence captions — one SRT entry per caption line, no karaoke, no wrapping. Reads SRTs directly, no Groq transcription needed. Karaoke/animated styles are opt-in.

Triggers: /video-caption, "caption this video", "add captions"

Pipeline (plain, default):

python3 scripts/caption_video.py <video> --srt=<main.srt> [--translation-srt=<other.srt>]

• Works on any dubbed or original video
• --srt= is required (main SRT — Chinese if dubbed, English if original)
• --translation-srt= is optional (shows below main as secondary)
• No GROQ_API_KEY needed for plain mode
• Output: {name}_captioned.mp4 + {name}_captions.ass

Karaoke / animated styles (opt-in): Use when the user explicitly asks for "word-level captions", "karaoke captions", "highlight each word", "bouncing captions", etc. These styles call Groq Whisper for word-level timestamps.

python3 scripts/caption_video.py video.mp4 --style=highlight
python3 scripts/caption_video.py video.mp4 --style=bounce --position=center

Style	Technique	Description
plain (default)	None	One SRT entry → one caption line. No karaoke, no wrapping. Reads SRTs directly.
highlight	Karaoke Fill	Full line shown; current word sweeps from white → yellow as spoken
appear	Word Reveal	Words appear one by one and accumulate until line ends
underline	Active Underline	Full line visible; current word is yellow + bold + underlined
bounce	Spring Physics	Word-by-word pop — each word bounces in (140%→95%→105%→100%) and exits; 1.8× bigger font
fade	Fade In/Out	Dim context line persists; current word fades in bright yellow, fades out at end
zoom	Zoom In	Word scales from 0% → 115% → 100% with slight overshoot — punchy entrance
slide	Slide Up	Word slides up 50px into position, then gently fades out
wave	Rock/Oscillate	Dim context line persists; current word rocks −8°→+8°→−4°→0° (settling oscillation)
typewriter	Typewriter	Characters appear one by one per word; previous words shown in white, current in yellow

Options:

• --style=<style> — caption style (default: plain). Non-plain styles require Groq API key and re-transcribe the video for word-level timestamps.
• --srt=<file> — main SRT file (REQUIRED for plain mode)
• --bilingual=<lang> — add secondary language translation below main captions (e.g. --bilingual=english)
• --main-lang=<lang> — make translated language the MAIN (top, karaoke) caption; original becomes secondary below
• --position=bottom|top|center — caption position (default: bottom)
• --font-size=<size> — font size (default: auto-calculated from video resolution; bounce auto-scales 1.8×)
• --words-per-line=8 — max words per caption line (default: 8)
• --color=BBGGRR — text color in ASS hex (default: 00FFFFFF = white)
• --highlight=BBGGRR — highlight color in ASS hex (default: 0000FFFF = yellow)
• --output=file.mp4 — custom output filename
• --srt-only — only generate ASS subtitle file, don't burn into video
• --lang=en — source language code (default: auto-detect)
• --words-json=<file> — skip re-transcription; load cached _words.json from a previous run
• --translation-srt=<file> — use a pre-made SRT file for translations instead of auto-translating. Critical for dubbing workflows — ensures captions match the exact text that was TTS'd, preventing audio/caption mismatch

Line Breaking (Sentence-Aware): Caption lines are grouped by meaningful sentence boundaries, NOT arbitrary word counts. The script uses a 3-tier strategy:

1. Whisper segment boundaries (preferred) — ASR segments represent natural sentences/phrases. Each caption line = one complete thought.
2. Punctuation boundaries (fallback) — splits at commas, periods, etc.
3. Time-gap heuristic (last resort) — for unpunctuated content like song lyrics, splits on pauses > 1.5s between words.

This prevents amateur-looking captions like "eyes Edge of the" that break mid-sentence. When using --words-json with corrected lyrics, the segments from the original transcription are still used for line grouping.

Auto-sizing: The script probes video resolution via ffprobe and auto-calculates optimal font size, margins, and outline thickness based on the video's height (scaled from 1080p baseline). Aspect ratio is also considered — ultrawide (>2.0) gets tighter margins, portrait (<1.0) gets compact margins. User CLI flags (--font-size, --position) override auto values.

Fonts: Cinema-style defaults — PingFang SC for CJK, Helvetica Neue for Latin. Regular weight, clean outline. Matches the bilingual subtitle style used in Chinese cinemas for Hollywood films.

Bilingual layout: Main language on top (larger, karaoke highlighting) + secondary language below (smaller, white). Both bottom-aligned with proper spacing.

Example Sessions:

User: "Caption this video with word-level timestamps"
Claude: python3 scripts/caption_video.py video.mp4
→ video_captioned.mp4 (karaoke-style word highlighting)

User: "Caption with bilingual Chinese + English"
Claude: python3 scripts/caption_video.py video.mp4 --bilingual=english
→ Chinese karaoke on top, English translation below

User: "Dub to Chinese with bilingual captions (use same translation)"
Claude: python3 scripts/caption_video.py dubbed.mp4 --main-lang=chinese \
  --translation-srt=video_chinese.srt --words-json=video_words.json
→ Chinese captions match dubbed audio exactly, English below

User: "Add word captions to this video, style: appear, position: top"
Claude: python3 scripts/caption_video.py video.mp4 --style=appear --position=top

User: "Caption https://youtube.com/watch?v=xxx"
Claude: python3 scripts/caption_video.py "https://youtube.com/watch?v=xxx"

6. Visual Analysis (Silent Video)

Analyze video frames visually and generate scene descriptions. Works on silent videos with no audio.

Triggers: "analyze this video", "what's happening in this video", "describe this silent video"

Fallback Sequence:

1. DEFAULT — Host LLM: Extract keyframes with ffmpeg (ffmpeg -i video.mp4 -vf "fps=1/4" frame_%04d.jpg), then read the images directly and describe each scene. Produces significantly better descriptions than the local model.
2. FALLBACK — video_captioner.py (MLX VLM): Use when the host LLM is unavailable or the pipeline must run unattended/offline.

   python3 scripts/video_captioner.py <video_file> [options]
   

video_captioner.py Options:

• --model MODEL — HuggingFace model ID (default: mlx-community/Qwen2.5-VL-3B-Instruct-8bit)
• --interval N — Extract 1 frame every N seconds (default: 3)
• --max-frames N — Maximum frames to process (default: 60)
• --language LANG — Output language: english/chinese (default: english)
• --backend BACKEND — Force backend: enconvo/standalone/auto (default: auto)
• --output FILE — Output SRT file path

Output: {name}_captions.srt — SRT file with visual descriptions synced to video timestamps.

Known limitation: The local 3B model returns !!! garbage on ~50% of frames (dark transitions, fast motion). Filter these out in post-processing. The host LLM does not have this issue.

Agent Protocols

Transcript Cleanup + Sentence Split

After transcription, run the cleanup script. It does TWO things in one pass:

python3 scripts/clean_srt.py {name}_original.srt --in-place

1. Filler removal. Strips:

• Filler words: "you know", "um", "uh", "like" (filler), "I mean", "basically", "actually" (filler)
• Repeated phrases: "you know... you know... you know..." → removed
• Stutters: "I- I was going" → "I was going"
• Excessive "sort of", "kind of", "right" (when repeated)

2. Sentence splitting. Whisper segment mode produces multi-sentence chunks up to 60+ words. This pass splits every segment on sentence punctuation (. ! ? 。 ！ ？) so each SRT entry is one meaningful sentence. Time redistributed proportionally by character count. Fallback tiers: commas → hard word/char cap (~15 English words / ~22 CJK chars) for unpunctuated run-ons.

Why upstream. Splitting BEFORE translation means:

• Translator gets sentence-sized chunks → better quality
• TTS fits each sentence into its own time window → fewer overlong segments to condense
• Captioner renders one sentence per caption line, no wrapping → fixes the crammed-caption bug

Opt out with --no-split if you need raw Whisper segments (rare — only for debugging).

Preserves meaning, tone, and natural phrasing. Same total time, redistributed across sentence-sized entries.

Important: This happens BEFORE translation so filler words don't propagate into the target language (e.g., "you know" → "你知道"). The translation prompt also drops any remaining fillers as a second safety net.

Pre-Translation Review (Agent Gate)

After clean_srt.py and before any translation or TTS work, run the review script and act on its verdict. This catches cases where the splitter couldn't fix everything and saves you from burning TTS time on a bad SRT.

python3 scripts/review_srt.py {name}_original.srt

The script reports total entries, avg/max length, shortest duration, and emits a verdict:

• OK — no anomalies. Proceed with translate/dub.
• WARN — some long entries or some sub-1s fragments. Mention the flags in your next user-facing message, then proceed. No user intervention needed unless they ask.
• CRITICAL — stop. Present the report to the user and propose edits before translating. Two common causes:
  1. Long compound entries (>35 CJK chars or >25 English words) → propose manual split at comma/semicolon, or investigate why clean_srt.py's splitter missed them.
  2. Too many sub-1s fragments (>30%) → when dubbing from a dense language to a wordier one (CJK→English especially), English TTS can't fit in those windows. Propose merging adjacent fragments within the same utterance before running TTS.

Why this exists: even with a correct automatic splitter, Whisper's prosody-based segmentation sometimes produces dozens of micro-fragments or joins entire paragraphs with commas. The review gate gives the agent one cheap inspection step before the expensive translate/TTS round-trip.

Script Translation

Translation uses the host LLM by default (the agent translates directly — higher quality, context-aware). Fallback: scripts/translate_srt.py using Groq LLM (llama-3.3-70b):

python3 scripts/translate_srt.py <srt_file> <target_lang>

The script translates segment-by-segment with quality rules baked in:

• Natural target language phrasing (not word-for-word)
• Preserves tone, style, brand names, and technical terms
• Optimized for listening, not reading
• Outputs {name}_{target_lang}.srt + review preview

Agent Condensation Protocol

After generate_tts_and_dub.sh runs, the agent handles any overlong segments:

1. Read {name}_timing_report.json — contains segments where TTS audio exceeds 1.3x the time window
2. If no overlong segments: done, pipeline complete
3. If overlong segments exist: a. For each segment, condense the translated text to the target_pct specified in the report b. Rules: Keep core message, cut filler/qualifiers/repetition, use shorter words, must sound natural when spoken c. Update the translated SRT file with condensed text d. Delete old raw + adj audio files for condensed segments:

   for idx in <condensed_indices>; do
     rm -f "$WORK_DIR/raw_$(printf '%04d' $idx).mp3" "$WORK_DIR/raw_$(printf '%04d' $idx).wav" "$WORK_DIR/adj_$(printf '%04d' $idx).wav"
   done
   

   e. Re-run with the same work directory:

   WORK_DIR=<work_dir_path> bash scripts/generate_tts_and_dub.sh <video> <orig.srt> <trans.srt> <lang> [voice] [voice_name]
   

4. Repeat if needed (max 2 passes)

Agent Summary Protocol

The agent reads the transcript and generates a structured summary:

1. Read {name}_transcript.txt (or extract text from SRT)
2. Generate summary with this structure:
   • Overview: 2-3 sentence overview
   • Key Points: 5-7 bullet points of most important takeaways
   • Detailed Summary: 2-3 paragraphs with context and details
   • Important Timestamps: Key moments with descriptions
   • Action Items: If applicable
3. Write to {name}_summary.md
4. Output language matches the user's request language

Parameters

• input (required): Path to video/audio file OR URL
  • Local: video.mp4, audio.mp3, podcast.m4a
  • URL: https://youtube.com/watch?v=xxx, https://twitter.com/user/status/xxx
• target_lang (optional): Target language (chinese, spanish, french, etc.)

Features

• Audio + Video support (MP4, MP3, WAV, M4A, and more)
• URL download (YouTube, Twitter, TikTok, 1000+ sites)
• Ultra-fast transcription (Groq Whisper Large V3)
• Natural translation via host LLM (default) or Groq LLM fallback (context-aware, preserves technical terms)
• Transcript cleanup (removes filler words and verbal tics before translation)
• Segment-by-segment TTS (precise timing per subtitle)
• Agent-driven condensation (shortens overlong translations via agent instead of external LLM)
• Perfect audio sync (natural speed with conservative adjustment)
• Voice cloning support (use any voicebox profile)
• Translation review (edit before TTS generation)
• Professional audio mixing (original at 15% + dub overlay at full volume, like Netflix/Disney+)
• Optional burned-in dual subtitles (original top/yellow + translated bottom/white, off by default)
• Multi-language TTS (Kokoro + edge-tts + voicebox)
• Intelligent summaries (with timestamps and key points)
• Language-aware detection (auto-detects request language)
• Word-level captioning (karaoke-style word highlighting with Groq Whisper word timestamps)

Requirements

Required

• ffmpeg (video/audio processing): brew install ffmpeg
• yt-dlp (URL downloads): brew install yt-dlp
• Groq API key (Whisper ASR): Free at console.groq.com
  • Whisper Large V3 for transcription — fast, free, supports SRT output, stable for long videos
  • Set: export GROQ_API_KEY=gsk_xxx or add to .env file in skill root
• EnConvo (translation): Must be running on localhost:54535
• edge-tts (default TTS engine): pip install edge-tts
• Python packages: pip install groq numpy soundfile (groq needed for Whisper ASR only)

Optional TTS Engines

Kokoro (local, offline, no internet needed):

• Fast local TTS, supports English/Chinese/Japanese + more
• Install: conda create -n kokoro python=3.10 && conda activate kokoro && pip install kokoro soundfile numpy
• If not installed, the skill automatically falls back to edge-tts

Voicebox (voice cloning & design):

• Install from: github.com/EnConvo/skill/tree/main/curated/voicebox
• Supports three voice profile types:
  • Qwen-TTS Clone: Cloned from reference audio (e.g., celebrity voice)
  • Descriptional Designed: Designed from text description (e.g., "calm male narrator")
  • Custom_Voice: Preset voice profiles with customizable emotions
• Important: Voicebox is best for short videos (1-5 minutes). For long videos (30+ min), voicebox generates segments sequentially which takes too long — use edge-tts instead (parallel generation, much faster)
• If not installed, the skill automatically falls back to edge-tts with a helpful install guide

Example Sessions

Transcription (Video/Audio/URL):

User: "Transcribe this video"
Claude: Extracts transcript → video_original.srt + video_transcript.txt

User: "Transcribe podcast.mp3"
Claude: Extracts audio transcript → podcast_original.srt + podcast_transcript.txt

User: "Transcribe https://youtube.com/watch?v=xxx"
Claude: Downloads video → Transcribes → transcript files

Summary (Language-Aware):

User: "What's this video about?"
Claude: Transcribes → Generates English summary in video_summary.md

User: "总结这段视频" (video.mp4)
Claude: Transcribes → Generates Chinese summary in video_summary.md

User: "Summarize https://twitter.com/user/status/xxx"
Claude: Downloads → Transcribes → Generates English summary

User: "这个YouTube视频讲什么？https://youtube.com/watch?v=xxx"
Claude: Downloads → Transcribes → Detects Chinese → Generates Chinese summary

Dubbing (Audio Support):

User: "Dub this to Chinese"
Claude: Transcribe → Clean filler → Translate → Review → TTS → Check timing → Dubbed video

User: "把这个音频配音成中文" (podcast.mp3)
Claude: Transcribe audio → Clean filler → Translate to Chinese → Review → TTS

User: "Dub this YouTube video to Spanish: https://youtube.com/watch?v=xxx"
Claude: Downloads → Transcribes → Clean filler → Translates → TTS → Dubbed video

Technical Details

Segment-by-Segment TTS Processing

The dubbing system uses a pipeline that scales to 1500+ segments:

1. TTS Generation - Each subtitle entry becomes a separate TTS audio file
   • edge-tts: Async parallel generation in batches of 10 (fastest for large files)
   • Kokoro: Single-process batch generation via KPipeline
   • voicebox: Sequential generation with voice cloning
2. Timing Analysis - Measures each TTS segment's actual duration against its SRT time window. Segments exceeding 1.3x their target duration are flagged in a timing report for agent-driven condensation. The agent shortens the translated text and re-runs TTS for those segments.
3. Speed Adjustment - Conservative tempo tuning for remaining timing mismatches
   • Never slows down — audio shorter than its window plays at natural speed (silence fills gaps)
   • Mild speedup only — capped at 2.0x via single ffmpeg atempo filter (rare after condensation)
   • Most segments need no adjustment after condensation
4. Numpy Timeline Assembly - Places each adjusted segment at its exact SRT start position in a pre-allocated numpy array. Scales to any number of segments without ffmpeg input limits.
5. Subtitle Burn-In - Burns dual subtitles into video (original top/yellow + translated bottom/white) using ffmpeg subtitles filter. Requires video re-encode (-c:v libx264 -crf 20 -preset fast)

Performance (tested on 2h22m video, 1,554 segments):

• edge-tts TTS generation: ~12 min (parallel batches of 10)
• Timing analysis + agent condensation: ~30-60s
• Speed adjustment: ~48s
• Numpy timeline assembly: ~1.3s
• Video muxing: ~43s
• Total: ~14-15 min for a 2h22m video

Benefits:

• Natural-sounding audio (no slow-motion or chipmunk effect)
• Scales to 1500+ segments (numpy, not ffmpeg amix)
• edge-tts parallel batches for 10x faster generation
• Burned-in dual subtitles (always visible, no player support needed)
• Resume support (skips already-generated segments)

TTS Engine Selection

Default: edge-tts — Used automatically unless the user explicitly requests otherwise.

• Male default: en-US-BrianMultilingualNeural (Brian Multilingual) — handles all languages natively
• Female default: en-US-EmmaMultilingualNeural (Emma Multilingual) — handles all languages natively
• Chinese male override: zh-CN-YunxiNeural (dedicated Chinese voice, better for Chinese dubbing)
• If user doesn't specify gender, default to Brian Multilingual (Male)
• Full voice list: edge-tts --list-voices

Kokoro (local, no internet) — Only used when the user explicitly asks for Kokoro.

• Trigger phrases: "use Kokoro", "use local TTS", "offline TTS"
• English: am_michael, am_adam, af_heart
• Chinese: zf_001 (and 100+ Chinese voices)

Voicebox (voice cloning/design) — Used when the user requests a specific voice persona, cloned voice, or voice description.

Install: github.com/EnConvo/skill/tree/main/curated/voicebox If not installed, the skill prints the install URL and falls back to edge-tts automatically.

Duration caveat: Voicebox generates segments sequentially (voice cloning is compute-intensive). Best for short videos (1-5 minutes). For long videos (30+ min), it takes too long and makes no practical sense — use edge-tts instead.

Three voice profile types:

Type	Description	Example
Qwen-TTS Clone	Cloned from reference audio	Celebrity voice, your own voice
Descriptional Designed	Designed from text description	"calm male narrator", "energetic female host"
Custom_Voice	Preset profiles with customizable emotions	Adjust pitch, speed, emotion per profile

Profile lookup: Voicebox stores all profiles in ~/.claude/skills/voicebox/data/profiles.json. Each profile has a name, type ("cloned" or "designed"), and language.

• List all profiles: uv run ~/.claude/skills/voicebox/scripts/voicebox.py list
• Lookup is case-insensitive with substring fallback (e.g., "trump" matches "Trump", "panic" matches "Panic Granny")

Scenario A: Cloned voice — User asks for a known cloned voice (e.g., "use Trump's voice")

• Run voicebox.py list to find the existing clone profile by name
• If found (type: "cloned") → use it directly for TTS

Scenario B: Named designed voice — User asks for a named voice persona (e.g., "use Panic Granny voice")

• Run voicebox.py list to search for an existing designed profile by name
• If found → use it for TTS
• If not found → invoke voicebox skill to design the voice profile first, then use it for TTS

Scenario C: Voice description — User describes voice characteristics (e.g., "a male mid-aged calm narrator")

• Invoke voicebox skill to design a new voice profile matching the description
• Then use the newly created profile for TTS

# Dub with a cloned/designed voice profile
generate_tts_and_dub.sh video.mp4 original.srt translated.srt chinese "Trump"

Selection logic summary:

1. User names a cloned voice → voicebox (find clone profile → TTS)
2. User names a voice persona → voicebox (find or design profile → TTS)
3. User describes voice traits → voicebox (design profile → TTS)
4. User explicitly asks for Kokoro → Kokoro (falls back to edge-tts if not installed)
5. Everything else → edge-tts (Brian Multilingual male / Emma Multilingual female)

Same-Language Re-voicing

The dubbing pipeline supports re-voicing in the same language (no translation needed). Use the original SRT as both the original and translated SRT:

generate_tts_and_dub.sh video.mp4 transcript.srt transcript.srt english none en-US-BrianMultilingualNeural

Notes

• All modes start with transcription (Groq Whisper ASR)
• Translation via host LLM (default) or Groq LLM fallback (natural, context-aware phrasing)
• Transcript cleanup removes filler words before translation
• Dubbing includes perfect audio-subtitle sync (segment-by-segment)
• Burns in dual subtitles (original top/yellow + translated bottom/white, always visible)
• Summaries are comprehensive but concise
• Long videos (1000+ segments) handled efficiently via numpy timeline