| name | ai-voice-production |
| description | AI voice production judgment for coding agents — TTS tool selection, voice cloning, audiobook/podcast/dubbing pipelines, Apple Silicon optimization, licensing safety |
| version | 0.1.0 |
| type | reference-based |
| keywords | ["TTS","text-to-speech","语音合成","voice cloning","声音克隆","voice design","音色设计","audiobook","有声书","podcast","播客","dubbing","配音","narration","旁白","audio production","音频制作","voice acting","语音","朗读","prosody"] |
AI Voice Production Capability Pack
Cross-agent portable judgment for AI-assisted voice production. Covers TTS tool selection, voice cloning, long-form audiobook pipelines, narration/dubbing workflows, Apple Silicon optimization, and licensing safety.
CONSUMES: Text manuscripts, reference audio samples (optional), brand voice guidelines (optional).
PRODUCES: Production-ready audio files (WAV 48kHz preferred, 44.1kHz for ACX). Naming: {project}/{chapter|segment}-{NNN}.wav.
INTERFACE: video-creation pack defers to this pack for voice/TTS tool selection and audio quality thresholds. This pack defers to video-creation for video-specific timing (audio-to-video sync, pacing). If both packs load, this pack takes precedence for tool selection; video-creation takes precedence for visual pacing. ml-training pack provides platform selection and cost estimation for cloud GPU training. This pack defers cloud platform details to ml-training's platform-selection.md. ml-training defers voice-specific tool selection to this pack.
When both packs load for voice training: ml-training takes precedence for platform/cost decisions; ai-voice-production takes precedence for tool selection and audio quality thresholds.
Step 0: Pack Prerequisites
- Python 3.10+ — most TTS tools require it
- FFmpeg — audio post-processing and mastering
- pip or uv — package installation in virtual environment
- VRAM-dependent: check
references/apple-silicon.md for Mac users
Verify: python3 --version && ffmpeg -version
Step 1: Context Detection
| User Signal | Load Reference |
|---|
| tool comparison, which TTS, choose engine, benchmark | references/tool-landscape.md |
| Mac, Apple Silicon, MPS, M-series, VRAM, 16GB, 32GB | references/apple-silicon.md |
| clone voice, reference audio, sound like, voice design | references/voice-cloning.md |
| audiobook, long-form, chapter, ACX, Audible, 有声书 | references/audiobook-pipeline.md |
| narration, dubbing, podcast, blog voice, 配音, short-form | references/narration-dubbing.md |
| ChatTTS, 自然语音, emotion control, 情绪, 对话, dialogue, speaker | references/chattts-workflow.md |
| license, commercial, legal, watermark, open source | references/licensing-safety.md |
Multi-signal: Load all matched references. Cross-reference links are provided within files.
Step 2: Decision Entry Point
Q1 — What is the use case?
- Audiobook (100K+ words, multi-chapter) → load
audiobook-pipeline.md + voice-cloning.md
- Blog/video narration (single voice, <30 min) → load
narration-dubbing.md
- Podcast (multi-episode, consistent voice) → load
narration-dubbing.md
- Video dubbing (emotion matching, multilingual) → load
narration-dubbing.md
- Chinese narration with emotion control or dialogue → load
chattts-workflow.md
- Quick TTS (one-off, any tool) → load
tool-landscape.md
Q2 — What hardware?
- Apple Silicon Mac → ALSO load
apple-silicon.md
- NVIDIA GPU → proceed with tool default configs
- CPU only → check MeloTTS or Piper in
tool-landscape.md
- No local GPU / insufficient VRAM → load ml-training pack's
references/platform-selection.md for cloud GPU selection. Primary use case: training and fine-tuning; inference can often stay local.
Q3 — Commercial use?
- Yes → ALSO load
licensing-safety.md
Step 3: Apply Rules
Read matched reference(s) and apply rules directly. Rules are concrete parameters — not guidelines.
Quick Rule Index
Tool Landscape (references/tool-landscape.md)
- Tier A/B Split: 9 researched tools with benchmarks vs 4 notable tools with key strengths → §Tier A / §Tier B
- 4 Selection Rules: by language, hardware, use case, commercial license → §Quick Selection Rules
Apple Silicon (references/apple-silicon.md)
- 16GB Budget Table: 6 tools with VRAM data, MPS configs → §16GB Memory Budget
- MPS Workarounds: float32, s3tokenizer patch, PYTORCH_ENABLE_MPS_FALLBACK → §MPS Configuration
Voice Cloning (references/voice-cloning.md)
- 3 Methods: zero-shot, fine-tuned, voice design → §Cloning Methods
- Minimum Reference Duration: 3s to 15s per tool → §Reference Duration Table
- Failure Modes: noise leakage, accent bleeding, emotional flatness → §Failure Modes
Audiobook Pipeline (references/audiobook-pipeline.md)
- 4 Non-Negotiables: consistency, emotion, chapter control, multi-character → §Non-Negotiable Requirements
- 5-Step Pipeline: prep → voice setup → generation → QC → post-processing → §Production Pipeline
- ACX Specs: MP3 192kbps, 44.1kHz, RMS -23 to -18 dB → §ACX/Audible Specifications
Narration & Dubbing (references/narration-dubbing.md)
- Blog Narration: single voice, quick turnaround → §Blog Narration
- Video Dubbing: emotion matching, multilingual → §Video Dubbing
- Mixed Language: Chinese/English strategy → §Mixed Language Strategy
ChatTTS Workflow (references/chattts-workflow.md)
- Voice Persistence: save/load
.pt files for cross-session consistency → §创建并保存声音身份
- Emotion Control: oral/laugh/break parameters + scene presets → §情绪参数体系
- Dual Speaker: two
.pt files + shared seed for dialogue → §双人对话工作流
Licensing & Safety (references/licensing-safety.md)
- GREEN/YELLOW/RED: per-tool commercial safety classification → §License Tiers
- Watermarking Traps: hidden markers in generated audio → §Watermarking
- Quality Sabotage: intentional degradation patterns → §Anti-Patterns
Anti-Skip Table
| Shortcut Attempt | Required Action |
|---|
| "I'll pick an appropriate TTS tool" | MUST use tool-landscape.md decision rules — tool choice depends on use case + hardware + license |
| "Short reference audio is fine" | MUST check minimum duration per tool in voice-cloning.md — ranges from 3s to 15s |
| "This tool is open source" | MUST check license tier in licensing-safety.md — open weights ≠ commercial use |
| "I'll master the audio later" | MUST apply ACX/podcast specs DURING pipeline in audiobook-pipeline.md, not after |
| "Any voice will work for now" | MUST set up voice identity BEFORE generation in voice-cloning.md — retrofitting means re-generating all audio |
