| name | super-transcribe |
| description | Unified speech-to-text skill. Use when the user asks to transcribe audio or video, generate subtitles, identify speakers, translate speech, search transcripts, diarize meetings, or perform any speech-to-text task. Also use when a voice message or audio file appears in chat and the user's intent to transcribe it is very clear. |
| version | 1.0.4 |
| author | Sarah Mak |
| tags | ["audio","transcription","whisper","parakeet","speech-to-text","ml","cuda","gpu","subtitles","diarization","podcast","unified"] |
| platforms | ["linux","wsl2"] |
| metadata | {"openclaw":{"emoji":"🎙️","requires":{"bins":["python3"],"optionalBins":["ffmpeg","yt-dlp"],"optionalPaths":["~/.cache/huggingface/token"]}}} |
🎙️ Super-Transcribe — Unified Speech-to-Text
A self-contained transcription skill with two bundled backends that intelligently routes based on task requirements:
- 🦜 Parakeet (NVIDIA NeMo) — best accuracy (6.34% WER), ~3380× realtime, auto-punctuation, 25 European languages, NeMo diarization
- 🗣️ faster-whisper (CTranslate2) — translation, 99+ languages, initial prompting, advanced inference tuning
Lazy loading: each backend sets up its own venv on first use. No pre-configuration needed — just transcribe and the right backend installs itself.
⚡ First-Time Setup
Follow these 3 steps on first use. Goal: install only what this user's hardware needs — nothing extra.
Step 1 — Check readiness (instant, no downloads)
./scripts/transcribe --check --json
action result | What to do |
|---|
"ready" | Skip setup — already installed. Go straight to usage. |
"install_python" | Tell user to install Python 3.10+ first, then re-run |
"run_quickstart" | Continue to Step 2 |
Key fields: gpu (name + VRAM), ffmpeg, backends status, estimated_install (download size for recommended backend), missing_optional (non-blocking items — do NOT eagerly install these).
Step 2 — Preview the install (no downloads)
./scripts/transcribe --quickstart --dry-run --json
Tell the user what will be downloaded before proceeding. Relay from the estimated field:
- Which backend and why it was chosen
- Total download size (setup + model on first use)
Example message: "Your system has an NVIDIA GPU, so I'd install the Parakeet backend for best accuracy. That's ~5GB for setup plus ~1.2GB for the model on first use (~6.2GB total). OK to proceed?"
For GPU systems where the user wants a smaller install, see Lean Install Option below.
Step 3 — Install (after user confirms)
./scripts/transcribe --quickstart --json
ok: true → backend installed. First transcription downloads model weights (~1-2 GB, cached permanently).
ok: false → check errors array for what failed.
What Gets Installed
Quickstart installs exactly ONE backend — the best for detected hardware:
| Hardware | Backend installed | Setup download | Model (first use) | Total |
|---|
| NVIDIA GPU | Parakeet (NeMo) | ~5 GB (PyTorch + CUDA + NeMo) | ~1.2 GB | ~6.2 GB |
| CPU / macOS | faster-whisper | ~300 MB (CTranslate2, no PyTorch) | ~756 MB | ~1 GB |
NOT installed by quickstart (deferred until actually needed):
- Second backend — auto-installs only when the user triggers a feature that needs it (
--translate or non-EU language → faster-whisper; --fast/--multitalker → Parakeet)
- System deps — install separately only if the user needs them:
ffmpeg — only for non-WAV input (mp3/m4a/mp4/ogg). Install: sudo apt install ffmpegyt-dlp — only for YouTube/URL downloads. Install: pipx install yt-dlp- HuggingFace token — only for
--diarize with faster-whisper. Setup: huggingface-cli login
- PyTorch for faster-whisper — deferred until
--diarize is first used (saves ~2.8 GB on initial install)
Do NOT pre-install optional deps. Wait until the user actually needs a feature — the script auto-installs or clearly reports what's missing at that point.
Lean Install Option
If a GPU user wants a smaller download (~1 GB instead of ~6.2 GB):
./scripts/transcribe --setup faster-whisper
Trade-off: ~20× realtime speed (vs ~3380× with Parakeet), no built-in punctuation. Only offer this if the user explicitly asks for a minimal install or has limited bandwidth/disk.
Supplementary info: see Setup Details further down.
✅ Setup done. Everything below is usage reference — read on demand.
When to Use
Use this skill for any speech-to-text task. It replaces both the faster-whisper and parakeet skills as your single entry point.
Trigger phrases:
"transcribe this", "speech to text", "what did they say", "make a transcript",
"subtitle this video", "who's speaking", "translate this audio", "transcribe this podcast",
"transcribe with parakeet", "transcribe with whisper", "best accuracy transcription",
"transcribe in French", "diarize this meeting", "find where X is mentioned",
"audio to text", "translate to English", "search transcript for", "when did they say",
"at what timestamp", "add chapters", "detect chapters", "find breaks in the audio",
"table of contents for this recording", "TTML subtitles", "DFXP subtitles",
"broadcast format subtitles", "Netflix format", "ASS subtitles", "aegisub format",
"LRC subtitles", "timed lyrics", "karaoke subtitles", "HTML transcript",
"confidence-colored transcript", "separate audio per speaker", "export speaker audio",
"transcript as CSV", "spreadsheet output", "podcast RSS feed", "per-file language",
"remove filler words", "clean up ums and uhs", "transcribe left channel",
"wrap subtitle lines", "character limit per line", "detect paragraphs"
Auto-Routing Logic
The router picks the backend automatically:
┌─────────────────────────────────────────────┐
│ --backend specified? │
│ YES → use that backend │
│ NO ↓ │
├─────────────────────────────────────────────┤
│ --fast or --multitalker? │
│ YES → parakeet │
│ NO ↓ │
├─────────────────────────────────────────────┤
│ Needs faster-whisper-only feature? │
│ (translate, initial-prompt, hotwords, │
│ multilingual, non-EU language, advanced │
│ inference tuning params) │
│ YES → faster-whisper │
│ NO ↓ │
├─────────────────────────────────────────────┤
│ Needs parakeet-only feature? │
│ (--long-form, --streaming, --no-align) │
│ YES → parakeet │
│ NO ↓ │
├─────────────────────────────────────────────┤
│ Default: prefer Parakeet (best accuracy │
│ + speed + auto-punctuation) │
│ Fall back to faster-whisper if Parakeet │
│ not installed │
└─────────────────────────────────────────────┘
Quick Reference
Basic (Auto-Routes to Best Available)
| Task | Command |
|---|
| Basic transcription | ./scripts/transcribe audio.mp3 |
| SRT subtitles | ./scripts/transcribe audio.mp3 --format srt -o subs.srt |
| VTT subtitles | ./scripts/transcribe audio.mp3 --format vtt -o subs.vtt |
| ASS subtitles | ./scripts/transcribe audio.mp3 --format ass -o subs.ass |
| LRC lyrics | ./scripts/transcribe audio.mp3 --format lrc -o lyrics.lrc |
| TTML broadcast | ./scripts/transcribe audio.mp3 --format ttml -o subs.ttml |
| CSV spreadsheet | ./scripts/transcribe audio.mp3 --format csv -o out.csv |
| JSON output | ./scripts/transcribe audio.mp3 --format json -o out.json |
| YouTube/URL | ./scripts/transcribe https://youtube.com/watch?v=... |
| Batch process | ./scripts/transcribe *.mp3 -o ./transcripts/ |
| Force backend | ./scripts/transcribe --backend parakeet audio.mp3 |
| List backends | ./scripts/transcribe --backends |
| Show version | ./scripts/transcribe --version |
| Probe metadata | ./scripts/transcribe --probe audio.mp3 |
| Fast mode (110M) | ./scripts/transcribe --fast audio.mp3 |
| Multitalker | ./scripts/transcribe --multitalker meeting.wav |
| Resume batch | ./scripts/transcribe *.mp3 --resume progress.json -o ./out/ |
| Exact model | ./scripts/transcribe -m nvidia/parakeet-tdt-1.1b audio.wav |
| Model alias | ./scripts/transcribe --backend pk -m 1.1b audio.wav |
| Speaker diarization | ./scripts/transcribe meeting.wav --diarize |
| Search transcript | ./scripts/transcribe audio.mp3 --search "keyword" |
| Detect chapters | ./scripts/transcribe audio.mp3 --detect-chapters |
| Clean filler words | ./scripts/transcribe audio.mp3 --clean-filler |
| Denoise audio | ./scripts/transcribe audio.mp3 --denoise |
| Podcast RSS feed | ./scripts/transcribe --rss https://feed.url |
| Burn subtitles | ./scripts/transcribe video.mp4 --burn-in out.mp4 |
| Name speakers | ./scripts/transcribe audio.mp3 --diarize --speaker-names "Alice,Bob" |
| Export speaker audio | ./scripts/transcribe audio.mp3 --diarize --export-speakers ./spk/ |
Routes to Faster-Whisper Automatically
| Task | Command | Why |
|---|
| Translate → English | ./scripts/transcribe audio.mp3 --translate | Whisper-specific feature |
| Canary translation | ./scripts/transcribe audio.mp3 --backend parakeet --translate --source-lang fr --target-lang de | NeMo Canary (EN/FR/DE/ES bidirectional) |
| Domain jargon | ./scripts/transcribe audio.mp3 --initial-prompt "Kubernetes" | Whisper-specific prompting |
| Non-European language | ./scripts/transcribe audio.mp3 -l ja | Parakeet: 25 EU langs only |
| Multilingual mode | ./scripts/transcribe audio.mp3 --multilingual | Whisper-specific feature |
| Hotwords boost | ./scripts/transcribe audio.mp3 --hotwords 'JIRA Kubernetes' | Whisper-specific feature |
| Prefix conditioning | ./scripts/transcribe audio.mp3 --prefix 'Good morning,' | Whisper-specific feature |
| Clip time range | ./scripts/transcribe audio.mp3 --clip-timestamps "30,60" | Whisper-specific feature |
| Detect language only | ./scripts/transcribe audio.mp3 --detect-language-only | Routes to fw by default; use --backend parakeet for Parakeet detection |
| Parallel batch | ./scripts/transcribe *.mp3 --parallel 4 -o ./out/ | Whisper-specific feature |
Routes to Parakeet Automatically
| Task | Command | Why |
|---|
| Long audio (>24 min) | ./scripts/transcribe lecture.wav --long-form | Parakeet-only: local attention mode |
| Streaming output | ./scripts/transcribe audio.wav --streaming | Parakeet-only: chunked inference |
| Fast / low VRAM (110M) | ./scripts/transcribe --fast audio.mp3 | Parakeet 110M model (~2GB VRAM) |
| Multitalker / overlap | ./scripts/transcribe --multitalker meeting.wav | Sortformer + multitalker pipeline |
| Skip alignment | ./scripts/transcribe audio.wav --no-align | Parakeet-only: faster, less precise |
| Canary translation | ./scripts/transcribe --backend parakeet --translate --source-lang fr --target-lang de audio.wav | Parakeet-only: Canary bidirectional EU translation |
| Detect language (Parakeet) | ./scripts/transcribe --backend parakeet --detect-language-only audio.wav | Parakeet language detection (requires --backend parakeet) |
| Danish transcription | ./scripts/transcribe -l da audio.wav | Auto-selects dedicated Danish model |
⚠️ Agent Guidance — Keep Invocations Minimal
CORE RULE: The default command (./scripts/transcribe audio.mp3) is the fastest path. Add flags only when the user explicitly asks for that capability. The router handles backend selection automatically.
⚠️ Model preference: When the router selects faster-whisper, it uses distil-large-v3.5 by default — this is the preferred model (fastest, better accuracy than large-v3-turbo). Don't override unless the user asks.
Probe Mode (--probe)
Use --probe to check audio metadata before deciding whether to transcribe:
./scripts/transcribe --probe audio.mp3
- Returns compact JSON with duration, format, channels, sample rate, bitrate, and file size
- No transcription — runs in milliseconds using ffprobe
- Use to implement the "ask before transcribing long files" flow:
- Probe → duration > 10 min → ask user "This is 45 minutes, want me to transcribe?"
- Probe → duration < 2 min → transcribe immediately
- Works with multiple files:
./scripts/transcribe --probe *.mp3
Agent Output Mode (--agent)
Use --agent for structured output the agent can parse directly:
./scripts/transcribe --agent audio.ogg
- Emits a single compact JSON line to stdout — no stderr noise (implies
--quiet)
- Core fields:
text, duration, language, language_probability, processing_time, backend, segments (count), speakers (list or null), word_count
- Agent-specific fields:
avg_confidence (0.0–1.0) — average word/segment confidence; use to gauge transcript reliability. Present for faster-whisper (from avg_logprob); absent for Parakeet (NeMo doesn't expose confidence scores)file_path — absolute path of input file (for multi-file tracking)output_path — absolute path of written output file (when -o is used)summary_hint — {"first": "...", "last": "..."} for long transcripts (>400 chars); gives the agent a quick preview without reading the full text
- Combine with
-o to also save full transcript to file: ./scripts/transcribe --agent -o /tmp/transcript.txt audio.mp3
- Use for quick voice message transcription where the agent just needs the text and metadata
Exit Codes
Standardized exit codes for agent error handling:
| Code | Meaning | Agent response |
|---|
| 0 | Success | Normal reply |
| 1 | General error | "Transcription failed" |
| 2 | Missing dependency | "Backend not set up yet — need to run setup" |
| 3 | Bad input (file not found, invalid args) | "Couldn't process that audio file" |
| 4 | GPU/VRAM error (OOM) | "GPU ran out of memory — try a smaller model or CPU" |
Chat Media Handling (OpenClaw Agents)
OpenClaw downloads media and provides local file paths in the message context (e.g. <media:voice path="/path/to/audio.ogg"> or <media:audio path="/path/to/file.mp3">).
⚠️ Do NOT auto-transcribe every voice message. Determine intent first:
When to Transcribe Immediately (No Confirmation Needed)
- User explicitly asks for transcription: "transcribe this", "what does this say", "make subtitles for this"
- User sends an audio/video file (not a voice note) with a transcription request
- User sends media in an ongoing transcription conversation (context makes intent clear)
When to Ask First
- User sends a voice message with no context — they're probably talking to you, not asking for a transcript. Respond to the content of their message instead, or ask: "Want me to transcribe this, or were you telling me something?"
- User sends a long audio/video file (>10 min) without explicit instructions — confirm before committing to a long transcription
- User forwards audio from someone else without comment — unclear if they want transcription or something else
Technical Notes
- Voice messages (.ogg opus) are auto-converted by ffmpeg — no special handling needed
- Video files work too — ffmpeg extracts the audio track automatically
- Use
-q (quiet) for short voice messages to suppress stderr progress noise
Example Flows
User wants transcription:
User: Can you transcribe this? [sends audio file]
Agent: Transcribing now...
Agent: [runs ./scripts/transcribe -q /path/to/audio.mp3]
Agent: Here's the transcript: "..."
User is talking to the agent via voice:
User: [sends voice message saying "Hey, what's on my calendar today?"]
Agent: [transcribes silently to understand the request]
Agent: [responds to the calendar question, does NOT parrot back the transcript]
Ambiguous intent (use --probe to check duration first):
User: [sends 45-minute meeting recording, no comment]
Agent: [runs ./scripts/transcribe --probe /path/to/recording.mp3]
Agent: Got a 45-minute recording. Want me to transcribe it? I can also do speaker identification or subtitles if you need those.
Output Length Management for Chat
Transcripts can be very long. Follow these rules to avoid flooding the chat:
- Short audio (<2 min, <~2000 chars): Reply directly with the full transcript text
- Medium audio (2-10 min): Reply with full text; if it exceeds ~3000 chars, consider saving to file and offering a summary
- Long audio (>10 min): Always save to a file (
-o /tmp/transcript.txt), share the file path, and provide a brief summary of the content
- Subtitle/data formats (SRT, VTT, ASS, CSV, JSON, etc.): Always save to file with
-o, tell the user the path
- Search results (
--search): Show directly — they're already concise
- Language detection (
--detect-language-only): Show directly — single line output
When in doubt, save to file and summarize. Users can always ask for the full text.
Do NOT add flags unless explicitly requested:
--backend — unless the user specifically requests a backend--diarize — unless the user asks "who said what" / "identify speakers"--translate — unless the user wants audio translated to English--format srt/vtt/ass/etc. — unless the user asks for subtitles in that format--long-form — unless the audio is confirmed >24 minutes--denoise/--normalize — unless the user mentions bad audio quality--initial-prompt — unless there's domain-specific jargon to prime--search — unless the user asks to find/locate a word in audio--detect-chapters — unless the user asks for chapters/sections--clean-filler — unless the user asks to remove filler words--word-timestamps — unless the user needs word-level timing--stream / --streaming — unless the user wants live/progressive output--clip-timestamps — unless the user wants a specific time range--temperature 0.0 — unless the model is hallucinating on music/silence--vad-threshold — unless VAD is aggressively cutting speech or including noise--min-speakers/--max-speakers — unless you know the speaker count--hf-token — unless the token is not cached at ~/.cache/huggingface/token--max-words-per-line / --max-chars-per-line — unless the user asks for subtitle wrapping--filter-hallucinations — unless the transcript contains obvious artifacts--merge-sentences — unless the user asks for sentence-level subtitle cues--channel left|right — unless the user mentions stereo tracks--detect-paragraphs — unless the user asks for paragraph breaks--speaker-names — unless the user provides real names (always requires --diarize)--hotwords — unless the user names specific rare terms--prefix — unless the user knows the exact words the audio starts with--detect-language-only — unless the user only wants language ID--stats-file — unless the user asks for performance stats--parallel N — only for large CPU batch jobs--retries N — only for unreliable inputs (URLs, network files)--burn-in — only when user explicitly asks to embed subtitles into video--keep-temp — only when the user may re-process the same URL--output-template — only when user specifies a custom naming pattern--timestamps (parakeet) — unless the user asks for word-level timestamps--fast — unless the user explicitly wants speed over accuracy or mentions low VRAM--multitalker — unless the user mentions overlapping speech or asks for better multi-speaker handling--resume <path> — unless the user is resuming a batch job that crashed/was interrupted--no-align — unless the user wants faster processing and doesn't need precise word timestamps--multitalker-diar-model / --multitalker-asr-model — only when user provides specific model names- Multi-format (
--format srt,text) — only when user explicitly wants multiple formats; always pair with -o <dir>
--agent — use when you need structured JSON for parsing (e.g. silent transcription of voice messages to understand user intent); don't use when the user explicitly asked for a transcript (just show them the text directly)--probe — use to check duration before transcribing ambiguous long files; don't probe short voice messages (just transcribe them directly)
Overhead notes:
- Any word-level feature (faster-whisper) auto-runs wav2vec2 alignment (~5-10s overhead)
--diarize adds ~20-30s on top
Search guidance:
--search replaces the normal transcript output — it prints only matching segments with timestamps- Add
--search-fuzzy only when the user mentions approximate/partial matching
- To save search results to a file, use
-o results.txt
Chapter detection guidance:
- Default
--chapter-gap 8 (8-second silence = new chapter) works for most content; tune down for dense content
--chapter-format youtube (default) outputs YouTube-ready timestamps; use json for programmatic use- Always use
--chapters-file PATH when combining chapters with a transcript output
- Batch mode limitation:
--chapters-file takes a single path — in batch mode, each file's chapters overwrite the previous
Speaker audio export guidance:
- Always pair
--export-speakers with --diarize
- Requires ffmpeg; outputs
SPEAKER_1.wav, SPEAKER_2.wav, etc. (or real names if --speaker-names is set)
Language map guidance:
- Only use
--language-map in batch mode when the user has confirmed different languages across files
- Inline format:
"interview*.mp3=en,lecture*.mp3=fr" — fnmatch globs on filename
- JSON file format:
@/path/to/map.json
RSS / Podcast guidance:
- Default fetches 5 newest episodes;
--rss-latest 0 for all; --skip-existing to resume safely
- Always use
-o <dir> with --rss — without it, all episode transcripts print to stdout concatenated
Output format for agent relay:
- Text transcript → show directly to user (summarise long ones)
- Subtitle formats (SRT, VTT, ASS, LRC, TTML) → write to
-o file, tell user the path
- Data formats (CSV, TSV, JSON, HTML) → write to
-o file, tell user the path
- Search results → show directly (human-readable)
- Chapter output → show directly or write to
--chapters-file
- Stats output (
--stats-file) → summarise key fields for the user
- Language detection (
--detect-language-only) → print directly; it's a single line
- Multi-format (
--format srt,text) → requires -o <dir>; tell user all paths written
Don't use this skill when:
- User wants text-to-speech (use the
tts tool instead)
- User wants audio editing or music generation
- User wants to summarize a YouTube video without needing a raw transcript (use
summarize skill)
Backend Comparison
| Feature | 🦜 Parakeet | 🗣️ faster-whisper |
|---|
| Accuracy | ✅ Best (6.34% avg WER) | Good (distil: 7.08% WER) |
| Speed | ✅ ~3380× realtime | ~20× realtime |
| Auto punctuation | ✅ Built-in | ❌ Requires post-processing |
| Languages | 25 European | ✅ 99+ worldwide |
| Diarization | ✅ NeMo ClusteringDiarizer | ✅ pyannote speaker ID |
| Translation | ✅ Canary: EN/FR/DE/ES | ✅ Any → English |
| Language detection | ✅ Auto-detect + output | ✅ --detect-language-only |
| Chapters/search | ✅ Shared | ✅ Shared |
| Output formats | ✅ All 10 formats | ✅ All 10 formats |
| Audio preprocessing | ✅ --denoise, --normalize | ✅ --denoise, --normalize |
| Filler removal | ✅ --clean-filler | ✅ --clean-filler |
| Long audio | ✅ Up to 3 hours | Limited by VRAM |
| Streaming | ✅ Chunked inference | ✅ Segment streaming |
| RSS/podcast | ✅ --rss | ✅ --rss |
| VRAM usage | ~2GB | ~1.5GB (distil) |
| Burn-in subtitles | ✅ --burn-in | ✅ --burn-in |
| Initial prompt | ❌ | ✅ Domain jargon priming |
| Word timestamps | ✅ NeMo + wav2vec2 aligned | ✅ wav2vec2 aligned (~10ms) |
| Custom models | ✅ Any NeMo / HuggingFace | ✅ CTranslate2 / HuggingFace |
| Multitalker | ✅ Sortformer + per-speaker | ❌ |
| Fast/small model | ✅ 110M (--fast) | ✅ distil-small.en (166M) |
| Batch resume | ✅ --resume | ✅ --resume |
| Noise handling | Good (built-in robustness) | ✅ --denoise, --normalize |
Setup Details
For first-time setup, see ⚡ First-Time Setup at the top. This section is supplementary reference.
What Gets Installed (and What Doesn't)
Quickstart installs one backend — the best for the user's hardware. The second backend auto-installs later only if a feature needs it. Heavy optional deps are deferred until first use.
| Hardware | Backend | Setup download | Model (first use) | What triggers more downloads |
|---|
| NVIDIA GPU | Parakeet | ~5 GB (PyTorch+CUDA+NeMo) | ~1.2 GB | --translate or non-EU language → faster-whisper |
| CPU / macOS | faster-whisper | ~300 MB (CTranslate2) | ~756 MB | --diarize → PyTorch (~2.8 GB); --fast/--multitalker → Parakeet |
Why the size difference? Parakeet needs PyTorch+CUDA upfront (~5 GB, NeMo depends on it). faster-whisper uses CTranslate2 directly — PyTorch is only pulled in for diarization/alignment features.
Parakeet uses nemo_toolkit[asr-only] — this skips ~19 training-only packages (wandb, transformers, datasets, lightning, pandas, peft, etc.) saving ~400 MB vs [asr].
Optional Extras (Not Needed for Basic Transcription)
| Feature | Install | Auto-installs? |
|---|
| Non-WAV audio (mp3/m4a/mp4) | sudo apt install ffmpeg (Linux) · brew install ffmpeg (macOS) | No — quickstart reports if missing |
| YouTube/URL downloads | pipx install yt-dlp | No |
| Speaker diarization (faster-whisper) | huggingface-cli login + accept model | Partial (PyTorch auto-installs; token + model agreement manual) |
| Both backends | --quickstart --all --json | Yes (second backend auto-installs on first use of a feature that needs it) |
Manual Backend Setup
./scripts/transcribe --check
./scripts/transcribe --check --json
./scripts/transcribe --setup parakeet
./scripts/transcribe --setup faster-whisper
./scripts/transcribe --setup all --diarize
Hard Requirements & Install Strategy
Hard requirements: Python 3.10+ only. Everything else is installed by --quickstart or auto-installs on first use.
Lean install strategy:
- faster-whisper: ~300 MB base (CTranslate2, no PyTorch). PyTorch (~2.8 GB) deferred until
--diarize.
- Parakeet: ~5 GB base (NeMo + PyTorch CUDA). Uses
nemo_toolkit[asr-only] — skips training-only packages.
- System deps:
--quickstart does NOT auto-install system packages (ffmpeg, yt-dlp). It reports them as missing; install them separately if needed.
Auto-install behavior: Backends lazy-load on first use. Optional deps (like pyannote for diarization) auto-install when you use the feature that needs them. In --agent mode, setup emits structured JSON status to stderr.
Platform Support
| Platform | Acceleration | Speed |
|---|
| Linux + NVIDIA GPU | CUDA | Parakeet ~3380× RT, Whisper ~20× RT |
| WSL2 + NVIDIA GPU | CUDA | Parakeet ~3380× RT, Whisper ~20× RT |
| macOS Apple Silicon | CPU | Whisper ~3-5× RT (Parakeet limited) |
| macOS Intel | CPU | Whisper ~1-2× RT |
| Linux (no GPU) | CPU | ~1× RT |
GPU Troubleshooting
If setup didn't detect your GPU, manually install PyTorch with CUDA:
uv pip install --python .venv/bin/python torch --index-url https://download.pytorch.org/whl/cu121
WSL2 users: Ensure you have the NVIDIA CUDA drivers for WSL installed on the Windows side.
Options
Super-Transcribe Routing Options
--quickstart One-command setup: install best backend for hardware (lean)
--quickstart --all Install both backends (Parakeet + faster-whisper)
--quickstart --json Same, structured JSON output for agent parsing
--quickstart --dry-run --json Preview what would be installed with estimated sizes
--check Check prerequisites + backend status (exit 0=ready, 2=not ready)
--check --json Same check, compact JSON with estimated install sizes
--backend <name> Force a backend: faster-whisper | parakeet | fw | pk
--backends List available/installed backends and exit
--probe Quick audio metadata (duration, format, channels) — no transcription
--setup <backend> Pre-install a backend: faster-whisper | parakeet | all
--help-routing Show detailed auto-routing logic
--fast Use small 110M Parakeet model (quick, lower accuracy)
--multitalker Multi-speaker pipeline with overlapped speech handling
--resume <path> Resume batch processing from checkpoint file
--version Show super-transcribe + backend versions
Shared Options (Work with Both Backends)
AUDIO Audio file(s), directory, glob, or URL
-f, --format FMT text | json | srt | vtt | ass | lrc | ttml | csv | tsv | html
-o, --output PATH Output file or directory
-m, --model NAME Model name (backend-specific)
-l, --language CODE Language code
--max-words-per-line Subtitle word wrapping
--max-chars-per-line Subtitle character wrapping
--batch-size N Inference batch size
--skip-existing Skip already-transcribed files
--resume PATH Resume batch from checkpoint file (skips completed files)
--device DEV auto | cpu | cuda
-q, --quiet Suppress progress messages
--version Print version info
--diarize Speaker diarization
--min-speakers N Min speakers hint for diarization
--max-speakers N Max speakers hint for diarization
--speaker-names LIST Replace SPEAKER_1, SPEAKER_2 with names
--export-speakers DIR Export each speaker's audio to WAV files
--search TERM Search transcript for TERM
--search-fuzzy Fuzzy/approximate search matching
--detect-chapters Detect chapter breaks from silence gaps
--chapter-gap SEC Min silence gap for chapter break (default: 8.0)
--chapters-file PATH Write chapter markers to file
--chapter-format FMT youtube | text | json (default: youtube)
--clean-filler Remove hesitation fillers (um, uh, etc.)
--filter-hallucinations Filter common hallucination patterns
--merge-sentences Merge segments into sentence chunks
--detect-paragraphs Insert paragraph breaks based on gaps
--paragraph-gap SEC Min gap for paragraph break (default: 3.0)
--normalize EBU R128 volume normalization
--denoise High-pass + FFT noise reduction
--channel CH left | right | mix (default: mix)
--burn-in OUTPUT Burn subtitles into video file
--rss URL Podcast RSS feed to transcribe
--rss-latest N Latest N episodes from RSS (default: 5)
--stats-file PATH Write performance stats JSON sidecar
Faster-Whisper Only Options
Model & Language:
--revision REV Model revision (git branch/tag/commit) to pin a specific version
--initial-prompt TEXT Prompt to condition the model (terminology, formatting style)
--prefix TEXT Prefix to condition the first segment (e.g. known starting words)
--hotwords WORDS Space-separated hotwords to boost recognition
--translate Translate any language to English (instead of transcribing)
--multilingual Enable multilingual/code-switching mode
--hf-token TOKEN HuggingFace token for private/gated models and diarization
--model-dir PATH Custom model cache directory (default: ~/.cache/huggingface/)
Output:
--word-timestamps Include word-level timestamps (wav2vec2 aligned automatically)
--stream Output segments as they are transcribed (disables diarize/alignment)
--output-template TPL Batch output filename template ({stem}, {lang}, {ext}, {model})
Inference Tuning:
--beam-size N Beam search size; higher = more accurate but slower (default: 5)
--temperature T Sampling temperature or comma-separated fallback list
--no-speech-threshold PROB Mark segments as silence (default: 0.6)
--no-vad Disable voice activity detection
--vad-threshold T VAD speech probability threshold (default: 0.5)
--vad-neg-threshold T VAD negative threshold for ending speech
--min-speech-duration MS Minimum speech segment duration in ms
--max-speech-duration SEC Maximum speech segment duration
--min-silence-duration MS Minimum silence before splitting (default: 2000)
--speech-pad MS Padding around speech segments (default: 400)
--no-batch Disable batched inference
--hallucination-silence-threshold SEC Skip silent sections where model hallucinates
--no-condition-on-previous-text Don't condition on previous text (auto-enabled for distil models)
--condition-on-previous-text Override auto-disable for distil models
--compression-ratio-threshold RATIO Filter segments above this ratio (default: 2.4)
--log-prob-threshold PROB Filter segments below avg log probability (default: -1.0)
--max-new-tokens N Maximum tokens per segment
--clip-timestamps RANGE Transcribe specific time ranges: '30,60' or '0,30;60,90'
--progress Show transcription progress bar
--best-of N Candidates when sampling with non-zero temperature (default: 5)
--patience F Beam search patience factor (default: 1.0)
--repetition-penalty F Penalty for repeated tokens (default: 1.0)
--no-repeat-ngram-size N Prevent n-gram repetitions (default: 0 = off)
Advanced Inference:
--no-timestamps Output text without timing info
--chunk-length N Audio chunk length for batched inference
--language-detection-threshold T Confidence threshold for language detection (default: 0.5)
--language-detection-segments N Segments to sample for detection (default: 1)
--length-penalty F Beam search length penalty (default: 1.0)
--prompt-reset-on-temperature T Reset prompt on temperature fallback (default: 0.5)
--no-suppress-blank Disable blank token suppression
--suppress-tokens IDS Token IDs to suppress
--max-initial-timestamp T Maximum timestamp for first segment (default: 1.0)
--prepend-punctuations CHARS Punctuation merged into preceding word
--append-punctuations CHARS Punctuation merged into following word
Advanced:
--min-confidence PROB Filter segments below avg word confidence (0.0–1.0)
--detect-language-only Detect language and exit (no transcription)
--keep-temp Keep temp files from URL downloads
--parallel N Parallel workers for batch processing
--retries N Retry failed files with exponential backoff
--language-map MAP Per-file language overrides for batch mode
Device:
--compute-type TYPE auto | int8 | int8_float16 | float16 | float32
--threads N CPU thread count for CTranslate2
--log-level LEVEL debug | info | warning | error
Parakeet Only Options
--long-form Local attention for audio >24 min (up to ~3 hours)
--streaming Print segments as they're transcribed (chunked inference)
--timestamps Enable word/segment/char timestamps (auto-enabled for timed formats)
--fast Use small 110M model for quick transcription (~2GB VRAM)
--no-align Skip wav2vec2 alignment (faster, less precise timestamps)
--multitalker Multi-speaker pipeline (Sortformer diar + per-speaker ASR)
--multitalker-diar-model MODEL Override diarizer model for multitalker
--multitalker-asr-model MODEL Override ASR model for multitalker
Parakeet Model Aliases
Short names for -m (Parakeet backend only):
| Alias | Resolves to | Notes |
|---|
v3, tdt-v3 | nvidia/parakeet-tdt-0.6b-v3 | Default multilingual |
v2, tdt-v2 | nvidia/parakeet-tdt-0.6b-v2 | English only |
1.1b, tdt-1.1b | nvidia/parakeet-tdt-1.1b | 1.1B English |
110m, fast, small | nvidia/parakeet-tdt_ctc-110m | Tiny/fast model |
ja, japanese | nvidia/parakeet-tdt_ctc-0.6b-ja | Japanese |
vi, vietnamese | nvidia/parakeet-ctc-0.6b-Vietnamese | Vietnamese |
da, danish | nvidia/parakeet-rnnt-110m-da-dk | Danish |
canary, canary-v2 | nvidia/canary-1b-v2 | Translation (25 EU) |
canary-flash | nvidia/canary-1b-flash | Fast translation (4 langs) |
multitalker | nvidia/multitalker-parakeet-streaming-0.6b-v1 | Multi-speaker |
Full HuggingFace model paths always work too (e.g. -m nvidia/parakeet-tdt-1.1b).
Both backends accept any valid model name via -m.
🗣️ faster-whisper Backend
Local speech-to-text using faster-whisper — a CTranslate2 reimplementation of OpenAI's Whisper that runs 4-6x faster with identical accuracy. With GPU acceleration, expect ~20x realtime transcription (a 10-minute audio file in ~30 seconds).
Faster-Whisper Model Selection
Choose the right model for your needs:
digraph model_selection {
rankdir=LR;
node [shape=box, style=rounded];
start [label="Start", shape=doublecircle];
need_accuracy [label="Need maximum\naccuracy?", shape=diamond];
multilingual [label="Multilingual\ncontent?", shape=diamond];
resource_constrained [label="Resource\nconstraints?", shape=diamond];
large_v3 [label="large-v3\nor\nlarge-v3-turbo", style="rounded,filled", fillcolor=lightblue];
large_turbo [label="large-v3-turbo", style="rounded,filled", fillcolor=lightblue];
distil_large [label="distil-large-v3.5\n(default)", style="rounded,filled", fillcolor=lightgreen];
distil_medium [label="distil-medium.en", style="rounded,filled", fillcolor=lightyellow];
distil_small [label="distil-small.en", style="rounded,filled", fillcolor=lightyellow];
start -> need_accuracy;
need_accuracy -> large_v3 [label="yes"];
need_accuracy -> multilingual [label="no"];
multilingual -> large_turbo [label="yes"];
multilingual -> resource_constrained [label="no (English)"];
resource_constrained -> distil_small [label="mobile/edge"];
resource_constrained -> distil_medium [label="some limits"];
resource_constrained -> distil_large [label="no"];
}
Standard Models (Full Whisper)
| Model | Size | Speed | Accuracy | Use Case |
|---|
tiny / tiny.en | 39M | Fastest | Basic | Quick drafts |
base / base.en | 74M | Very fast | Good | General use |
small / small.en | 244M | Fast | Better | Most tasks |
medium / medium.en | 769M | Moderate | High | Quality transcription |
large-v1/v2/v3 | 1.5GB | Slower | Best | Maximum accuracy |
large-v3-turbo | 809M | Fast | Excellent | High accuracy (slower than distil) |
Distilled Models (~6x Faster, ~1% WER Difference)
| Model | Size | Speed vs Standard | Accuracy | Use Case |
|---|
distil-large-v3.5 | 756M | ~6.3x faster | 7.08% WER | Default, best balance |
distil-large-v3 | 756M | ~6.3x faster | 7.53% WER | Previous default |
distil-large-v2 | 756M | ~5.8x faster | 10.1% WER | Fallback |
distil-medium.en | 394M | ~6.8x faster | 11.1% WER | English-only, resource-constrained |
distil-small.en | 166M | ~5.6x faster | 12.1% WER | Mobile/edge devices |
.en models are English-only and slightly faster/better for English content.
Note for distil models: HuggingFace recommends disabling condition_on_previous_text for all distil models to prevent repetition loops. The script auto-applies --no-condition-on-previous-text whenever a distil-* model is detected. Pass --condition-on-previous-text to override if needed.
Custom & Fine-Tuned Models
WhisperModel accepts local CTranslate2 model directories and HuggingFace repo names — no code changes needed.
./scripts/transcribe audio.mp3 --model /path/to/my-model-ct2
./scripts/transcribe audio.mp3 --model username/whisper-large-v3-ct2
./scripts/transcribe audio.mp3 --model-dir ~/my-models
pip install ctranslate2
ct2-transformers-converter \
--model openai/whisper-large-v3 \
--output_dir whisper-large-v3-ct2 \
--copy_files tokenizer.json preprocessor_config.json \
--quantization float16
Faster-Whisper Quick Reference
Essential commands (faster-whisper-specific features only — shared features like diarization, search, chapters, formats are in the shared options above):
| Task | Command | Notes |
|---|
| Translate → English | ./scripts/transcribe audio.mp3 --translate | Any language → English |
| Domain terms | ./scripts/transcribe audio.mp3 --initial-prompt 'Kubernetes gRPC' | Boost rare terminology |
| Hotwords boost | ./scripts/transcribe audio.mp3 --hotwords 'JIRA Kubernetes' | Bias decoder toward specific words |
| Clip time range | ./scripts/transcribe audio.mp3 --clip-timestamps "30,60" | Only 30s–60s |
| Stream output | ./scripts/transcribe audio.mp3 --stream | Live segments as transcribed |
| Fix hallucinations | ./scripts/transcribe audio.mp3 --temperature 0.0 --no-speech-threshold 0.8 | Lock temperature + skip silence |
| Tune VAD sensitivity | ./scripts/transcribe audio.mp3 --vad-threshold 0.6 --min-silence-duration 500 | Tighter speech detection |
| Hybrid quantization | ./scripts/transcribe audio.mp3 --compute-type int8_float16 | Save ~1GB VRAM, minimal quality loss |
| Parallel batch | ./scripts/transcribe *.mp3 --parallel 4 -o ./out/ | CPU multi-file processing |
| Language map (batch) | ./scripts/transcribe *.mp3 --language-map "int*.mp3=en,lec*.mp3=fr" | Per-file language in batch |
| Detect language only | ./scripts/transcribe audio.mp3 --detect-language-only | Fast language ID, no transcription |
| Custom model | ./scripts/transcribe audio.mp3 -m ./my-model-ct2 | Local CTranslate2 model dir |
| Output filename template | ./scripts/transcribe *.mp3 -o ./out/ --output-template "{stem}_{lang}.{ext}" | Custom batch output naming |
Output Formats (Both Backends)
Both backends support all 10 output formats via --format:
| Format | Extension | Use Case |
|---|
text | .txt | Default. Plain transcript |
json | .json | Full metadata: segments, timestamps, language, stats |
srt | .srt | Standard subtitle format (video players) |
vtt | .vtt | WebVTT (web video players) |
ass | .ass | Advanced SubStation Alpha (Aegisub, mpv, VLC) |
lrc | .lrc | Timed lyrics (music players) |
ttml | .ttml | Broadcast standard (Netflix, Amazon, BBC) |
csv | .csv | Spreadsheet-ready (with speaker column if diarized) |
tsv | .tsv | OpenAI Whisper–compatible TSV |
html | .html | Confidence-colored transcript |
All formats support diarization labels. Multi-format output: --format srt,text -o ./out/
Faster-Whisper Speaker Diarization
Identifies who spoke when using pyannote.audio.
Setup:
./setup.sh --diarize
Requirements:
- HuggingFace token at
~/.cache/huggingface/token (huggingface-cli login)
- Accepted model agreements:
Usage:
./scripts/transcribe meeting.wav --diarize
./scripts/transcribe meeting.wav --diarize --format srt -o meeting.srt
./scripts/transcribe meeting.wav --diarize --format json
Speakers are labeled SPEAKER_1, SPEAKER_2, etc. in order of first appearance. Runs on GPU automatically if CUDA is available.
Faster-Whisper Precise Word Timestamps
Whenever word-level timestamps are computed (--word-timestamps, --diarize, or --min-confidence), a wav2vec2 forced alignment pass automatically refines them from Whisper's ~100-200ms accuracy to ~10ms. Uses the MMS (Massively Multilingual Speech) model — supports 1000+ languages, cached after first load.
Faster-Whisper Performance Notes
- First run: Downloads model to
~/.cache/huggingface/ (one-time)
- Batched inference: Enabled by default — ~3x faster than standard mode; VAD on by default
- GPU: Automatically uses CUDA if available
- Quantization: INT8 used on CPU for ~4x speedup
- Benchmark (RTX 3070, 21-min file): ~24s with batched inference vs ~69s without
- --precise overhead: ~5-10s for wav2vec2 model load + alignment (cached for batch)
- Diarization overhead: ~10-30s depending on audio length
- Memory:
distil-large-v3: ~2GB RAM / ~1GB VRAM; large-v3-turbo: ~4GB RAM / ~2GB VRAM; Diarization: +~1-2GB VRAM
- OOM: Lower
--batch-size (try 4) if you hit out-of-memory errors
- Silero VAD V6: faster-whisper 1.2.1+ has improved speech detection
Faster-Whisper Common Mistakes
| Mistake | Problem | Solution |
|---|
| Using CPU when GPU available | 10-20x slower | Check nvidia-smi; verify CUDA |
| Not specifying language | Wastes time auto-detecting | Use --language en when known |
| Using wrong model | Unnecessary slowness | Default distil-large-v3.5 is excellent |
| Ignoring distilled models | Missing 6x speedup | Try distil-large-v3.5 first |
| Out of memory | Model too large | Use smaller model, --compute-type int8, or --batch-size 4 |
| --diarize without pyannote | Import error | Run setup.sh --diarize first |
| --diarize without HF token | Download fails | Run huggingface-cli login |
| URL without yt-dlp | Download fails | Install: pipx install yt-dlp |
| --min-confidence too high | Drops good segments | Start at 0.5, adjust up |
| Batch without -o directory | All output mixed in stdout | Use -o ./transcripts/ |
| Hallucinations on silence/music | Bad output | Try --temperature 0.0 --no-speech-threshold 0.8 |
Faster-Whisper Troubleshooting
- "CUDA not available — using CPU": Install PyTorch with CUDA (see GPU Support above)
- Setup fails: Make sure Python 3.10+ is installed
- Out of memory: Use smaller model,
--compute-type int8, or --batch-size 4
- Model download fails: Check
~/.cache/huggingface/ permissions
- Diarization model fails: Ensure HuggingFace token exists and model agreements accepted
- URL download fails: Check yt-dlp is installed (
pipx install yt-dlp)
- Check installed version:
./scripts/transcribe --version
- Upgrade:
./scripts/backends/faster-whisper/setup.sh --update
- VAD splits speech incorrectly: Tune with
--vad-threshold 0.3 or --min-silence-duration 300
Faster-Whisper References
🦜 Parakeet Backend
Local speech-to-text using NVIDIA's Parakeet TDT models via the NeMo toolkit. The default parakeet-tdt-0.6b-v3 is a 600-million-parameter multilingual ASR model that delivers state-of-the-art accuracy with automatic punctuation and capitalization, word-level timestamps, and an insane ~3380× realtime inference speed on GPU.
Only needs ~2GB VRAM to run — plenty of room on an RTX 3070 (8GB).
Parakeet Model Selection
Available Models
| Model | Alias | Params | Languages | VRAM | Speed | Use Case |
|---|
nvidia/parakeet-tdt-0.6b-v3 | v3 | 600M | 25 EU languages | ~2GB | ~3380× RT | Default, best multilingual |
nvidia/parakeet-tdt-0.6b-v2 | v2 | 600M | English only | ~2GB | ~3380× RT | English-only, slightly better English WER |
nvidia/parakeet-tdt-1.1b | 1.1b | 1.1B | English only | ~3GB | Slower | Maximum English accuracy |
nvidia/parakeet-tdt_ctc-1.1b | tdt-ctc-1.1b | 1.1B | English only | ~3GB | Fast | Hybrid TDT+CTC; 11hrs in one pass |
nvidia/parakeet-tdt_ctc-110m | 110m/fast | 110M | English only | ~1GB | Fastest | Low-VRAM / quick drafts |
nvidia/parakeet-tdt_ctc-0.6b-ja | ja | 600M | Japanese | ~2GB | ~3380× RT | Japanese (auto-selected with -l ja) |
nvidia/parakeet-ctc-0.6b-Vietnamese | vi | 600M | Vietnamese | ~2GB | ~3380× RT | Vietnamese (auto-selected with -l vi) |
nvidia/parakeet-rnnt-110m-da-dk | da | 110M | Danish | ~1GB | Fast | Danish (auto-selected with -l da) |
nvidia/canary-1b-v2 | canary | 1B | 25 EU languages | ~3GB | ~200× RT | ASR + translation (bidirectional) |
nvidia/canary-1b-flash | canary-flash | 883M | EN/DE/FR/ES | ~2.5GB | >1000× RT | Fast Canary variant (4 languages only) |
nvidia/multitalker-parakeet-streaming-0.6b-v1 | — | 600M | English | ~4GB* | ~3380× RT | Multi-speaker with overlapped speech |
*Multitalker loads diarizer (~2GB) + ASR model (~2GB). Both are lazy-loaded and cached.
Language-Specific Model Auto-Selection
When using the default model (parakeet-tdt-0.6b-v3) and specifying a language with -l, dedicated models are auto-selected:
-l da → nvidia/parakeet-rnnt-110m-da-dk-l ja → nvidia/parakeet-tdt_ctc-0.6b-ja-l vi → nvidia/parakeet-ctc-0.6b-Vietnamese
All models are lazy-loaded — downloaded from HuggingFace on first use and cached locally.
Supported Languages (v3)
Bulgarian (bg), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), German (de), Greek (el), Hungarian (hu), Italian (it), Latvian (lv), Lithuanian (lt), Maltese (mt), Polish (pl), Portuguese (pt), Romanian (ro), Slovak (sk), Slovenian (sl), Spanish (es), Swedish (sv), Russian (ru), Ukrainian (uk)
Language is auto-detected — no prompting or configuration needed.
Accuracy Benchmarks (v3)
| Benchmark | WER |
|---|
| Open ASR Leaderboard (avg) | 6.34% |
| LibriSpeech test-clean | 1.93% |
| LibriSpeech test-other | 3.59% |
| AMI | 11.31% |
| GigaSpeech | 9.59% |
| SPGI Speech | 3.97% |
| TED-LIUM v3 | 2.75% |
| VoxPopuli | 6.14% |
Parakeet Quick Reference
Essential commands (Parakeet-specific features only — shared features are in the shared options above):
| Task | Command | Notes |
|---|
| Long audio (>24 min) | ./scripts/transcribe lecture.wav --long-form | Local attention, up to ~3 hours |
| Streaming output | ./scripts/transcribe audio.wav --streaming | Print segments as transcribed |
| Fast / 110M model | ./scripts/transcribe --fast audio.wav | Quick, lower accuracy (~1GB VRAM) |
| Skip alignment | ./scripts/transcribe audio.wav --no-align | Faster, less precise timestamps |
| Multitalker mode | ./scripts/transcribe --multitalker meeting.wav | Overlapped speech, per-speaker ASR |
| Japanese | ./scripts/transcribe audio.wav -l ja | Auto-selects Japanese model |
| Vietnamese | ./scripts/transcribe audio.wav -l vi | Auto-selects Vietnamese model |
| Danish | ./scripts/transcribe audio.wav -l da | Auto-selects Danish model |
| Canary translation | ./scripts/transcribe audio.wav --backend parakeet --translate --source-lang fr --target-lang de | Bidirectional EU translation |
| Fast Canary | ./scripts/transcribe audio.wav -m canary-flash --translate | >1000× RT, EN/DE/FR/ES only |
| Upgrade NeMo | ./scripts/backends/parakeet/setup.sh --update | Upgrade + verify torch >= 2.6.0 |
Parakeet Output
Same 10 formats as listed in "Output Formats" above. Parakeet text output is automatically punctuated and capitalized. JSON output includes word/segment/char timestamps and performance stats.
Parakeet Long-Form Audio
Parakeet v3 supports two attention modes:
| Mode | Max Duration | Flag | Notes |
|---|
| Full attention (default) | ~24 min | (none) | Best accuracy, requires more VRAM |
| Local attention | ~3 hours | --long-form | Slightly reduced accuracy, much lower VRAM |
For audio over 24 minutes:
./scripts/transcribe long-lecture.wav --long-form
This changes the attention model to rel_pos_local_attn with a context window of [256, 256].
Parakeet Audio Format Support
| Format | Support | Notes |
|---|
.wav | ✅ Native | Preferred; 16kHz mono |
.flac | ✅ Native | Lossless; works directly |
.mp3 | 🔄 Converts | Requires ffmpeg |
.m4a | 🔄 Converts | Requires ffmpeg |
.mp4 | 🔄 Converts | Requires ffmpeg |
.ogg | 🔄 Converts | Requires ffmpeg |
.webm | 🔄 Converts | Requires ffmpeg |
.aac | 🔄 Converts | Requires ffmpeg |
.wma | 🔄 Converts | Requires ffmpeg |
.opus | 🔄 Converts | Requires ffmpeg |
.mkv | 🔄 Converts | Requires ffmpeg |
Non-native formats are automatically converted to 16kHz mono WAV using ffmpeg before transcription.
Parakeet Multitalker Mode
For audio with overlapping speakers (meetings, debates, panel discussions), the --multitalker mode uses a dedicated pipeline:
- Sortformer diarizer (
nvidia/diar_streaming_sortformer_4spk-v2.1) identifies speakers and their active regions
- Per-speaker audio is extracted using ffmpeg
- Multitalker ASR model (
nvidia/multitalker-parakeet-streaming-0.6b-v1) transcribes each speaker separately
- Results are merged and sorted by timestamp
./scripts/transcribe --multitalker meeting.wav
./scripts/transcribe --multitalker meeting.wav --speaker-names "Alice,Bob,Charlie"
./scripts/transcribe --multitalker \
--multitalker-diar-model nvidia/diar_streaming_sortformer_4spk-v2.1 \
--multitalker-asr-model nvidia/multitalker-parakeet-streaming-0.6b-v1 \
meeting.wav
Requirements: CUDA GPU with ~6GB VRAM (both models loaded simultaneously). English only. Both models are lazy-loaded on first use.
vs standard --diarize: Standard diarization transcribes the full audio first, then assigns speakers post-hoc — it can't handle overlapping speech. Multitalker transcribes each speaker independently, so overlapping regions are transcribed correctly per speaker.
Parakeet Timestamp Alignment
Parakeet includes automatic wav2vec2 forced alignment that refines word-level timestamps from NeMo's native output (~100ms) to ~10ms accuracy using the MMS model. This runs automatically after transcription.
- To skip alignment (faster processing): use
--no-align
- Alignment is automatically skipped in
--streaming mode
- The MMS model is cached after first load for batch efficiency
Parakeet Troubleshooting
| Problem | Solution |
|---|
| CUDA not available | Install PyTorch with CUDA: pip install torch --index-url https://download.pytorch.org/whl/cu121 |
| NeMo 2.6+ crashes | Requires torch >= 2.6.0. Run ./scripts/backends/parakeet/setup.sh --update to upgrade both torch and NeMo |
| NeMo import error | Run ./setup.sh or ./scripts/backends/parakeet/setup.sh |
| Out of memory | Use --fast (110M model, ~1GB VRAM) or --device cpu |
| Model download fails | Check HuggingFace connectivity and ~/.cache/huggingface/ permissions |
| ffmpeg not found | Install ffmpeg: apt install ffmpeg (needed for non-WAV input) |
| Multitalker fails | Requires NeMo 2.4+ and ~6GB VRAM. Check nvidia-smi for available memory |
| Diarization fails (NeMo) | Falls back to pyannote automatically. Install: pip install pyannote.audio |
| pyannote auth error | Run huggingface-cli login and accept model agreements |
| Streaming mode not real-time | --streaming uses chunked attention inference, not live audio streaming |
| Japanese/Vietnamese not detected | Explicitly pass -l ja or -l vi for auto-selection of dedicated models |
| Batch crashed mid-way | Use --resume progress.json to continue from where it stopped |
| Wrong model loaded | Check --model value. Use exact HuggingFace paths or aliases (see table above) |
Architecture
Both backends share a common library to avoid code duplication:
scripts/
├── transcribe # Unified entry point (router)
└── backends/
├── lib/ # Shared code (imported by both backends)
│ ├── formatters.py # All 10 output formats
│ ├── postprocess.py # Search, chapters, filler, paragraphs, merge, hallucination filter
│ ├── audio.py # Preprocessing, conversion, download, burn-in, resolve_inputs
│ ├── alignment.py # wav2vec2 forced alignment (MMS model, shared by both backends)
│ ├── speakers.py # Speaker name mapping, audio export
│ └── rss.py # RSS feed parsing
├── faster-whisper/ # CTranslate2 backend (imports from lib/)
└── parakeet/ # NeMo backend (imports from lib/)
