ai-voice-production

name	ai-voice-production
description	AI voice production judgment for coding agents. Covers TTS tool selection, voice cloning, audiobook, podcast, and dubbing pipelines, Apple Silicon optimization, and licensing safety. Use for any AI voice synthesis, voice cloning, audiobook production, or TTS pipeline task.
version	0.1.0
type	reference-based
keywords	["TTS","text-to-speech","语音合成","voice cloning","声音克隆","voice design","音色设计","audiobook","有声书","podcast","播客","dubbing","配音","narration","旁白","audio production","音频制作","voice acting","语音","朗读","prosody"]

AI Voice Production Capability Pack

Cross-agent portable judgment for AI-assisted voice production. Covers TTS tool selection, voice cloning, long-form audiobook pipelines, narration/dubbing workflows, Apple Silicon optimization, and licensing safety. CONSUMES: Text manuscripts, reference audio samples (optional), brand voice guidelines (optional). PRODUCES: Production-ready audio files (WAV 48kHz preferred, 44.1kHz for ACX). Naming: {project}/{chapter|segment}-{NNN}.wav. INTERFACE: video-creation pack defers to this pack for voice/TTS tool selection and audio quality thresholds. This pack defers to video-creation for video-specific timing (audio-to-video sync, pacing). If both packs load, this pack takes precedence for tool selection; video-creation takes precedence for visual pacing. ml-training pack provides platform selection and cost estimation for cloud GPU training. This pack defers cloud platform details to ml-training's platform-selection.md. ml-training defers voice-specific tool selection to this pack. When both packs load for voice training: ml-training takes precedence for platform/cost decisions; ai-voice-production takes precedence for tool selection and audio quality thresholds.

---

Step 0: Pack Prerequisites

• Python 3.10+ — most TTS tools require it
• FFmpeg — audio post-processing and mastering
• pip or uv — package installation in virtual environment
• Memory-dependent: check references/apple-silicon.md for Mac users (Apple Silicon uses unified memory, not discrete VRAM)

Verify: python3 --version && ffmpeg -version

---

Step 1: Context Detection

User Signal	Load Reference
tool comparison, which TTS, choose engine, benchmark	references/tool-landscape.md
Mac, Apple Silicon, MPS, M-series, unified memory, VRAM, 16GB, 32GB	references/apple-silicon.md
clone voice, reference audio, sound like, voice design	references/voice-cloning.md
audiobook, long-form, chapter, ACX, Audible, 有声书	references/audiobook-pipeline.md
narration, dubbing, podcast, blog voice, 配音, short-form	references/narration-dubbing.md
ChatTTS, 自然语音, emotion control, 情绪, 对话, dialogue, speaker	references/chattts-workflow.md
license, commercial, legal, watermark, open source	references/licensing-safety.md

Multi-signal: Load all matched references. Cross-reference links are provided within files.

---

Step 2: Decision Entry Point

Q1 — What is the use case?

• Audiobook (100K+ words, multi-chapter) → load audiobook-pipeline.md + voice-cloning.md
• Blog/video narration (single voice, <30 min) → load narration-dubbing.md
• Podcast (multi-episode, consistent voice) → load narration-dubbing.md
• Video dubbing (emotion matching, multilingual) → load narration-dubbing.md
• Chinese narration with emotion control or dialogue → load chattts-workflow.md
• Quick TTS (one-off, any tool) → load tool-landscape.md

Q2 — What hardware?

• Apple Silicon Mac → ALSO load apple-silicon.md
• NVIDIA GPU → proceed with tool default configs
• CPU only → check MeloTTS or Piper in tool-landscape.md
• No local GPU / insufficient VRAM → load ml-training pack's references/platform-selection.md for cloud GPU selection. Primary use case: training and fine-tuning; inference can often stay local.

Q3 — Commercial use?

• Yes → ALSO load licensing-safety.md

---

Step 3: Apply Rules

Read matched reference(s) and apply rules directly. Rules are concrete parameters — not guidelines.

---

Quick Rule Index

Tool Landscape (references/tool-landscape.md)

• Tier A/B Split: 11 researched tools with benchmarks (incl. IndexTTS2 emotion control, CosyVoice2-0.5B ~150ms streaming) vs 4 notable tools → §Tier A / §Tier B
• 4 Selection Rules: by language, hardware, use case, commercial license → §Quick Selection Rules

Validation Scripts (scripts/)

• scripts/acx-check.sh: asserts the deterministically-measurable ACX specs — MP3 192kbps, 44.1kHz, channel layout (all-mono OR all-stereo), RMS -23..-18, sample peak < -3 dBFS, noise floor < -60, head 0.5-1.0s / tail 1.0-5.0s room-tone duration — exit code drives the gate. Note: "peak" here is SAMPLE peak (dBFS, ACX's spec), NOT dBTP; the script measures silence DURATION but cannot verify the silence is room tone vs digital zero (manual check).
• scripts/lufs-check.sh <platform>: asserts integrated LUFS in band + genuine true peak (input_tp dBTP) <= -1 dBTP (apple/apple-mono/spotify/youtube/ebu)

Apple Silicon (references/apple-silicon.md)

• 16GB Budget Table: 6 tools with VRAM data, MPS configs → §16GB Memory Budget
• MPS Workarounds: float32, s3tokenizer patch, PYTORCH_ENABLE_MPS_FALLBACK → §MPS Configuration

Voice Cloning (references/voice-cloning.md)

• 3 Methods: zero-shot, fine-tuned, voice design → §Cloning Methods
• Minimum Reference Duration: 3s to 15s per tool → §Reference Duration Table
• Failure Modes: noise leakage, accent bleeding, emotional flatness → §Failure Modes

Audiobook Pipeline (references/audiobook-pipeline.md)

• 4 Non-Negotiables: consistency, emotion, chapter control, multi-character → §Non-Negotiable Requirements
• 5-Step Pipeline: prep → voice setup → generation → QC → post-processing → §Production Pipeline
• ACX 8 Hard Specs: MP3 192kbps CBR, 44.1kHz, all-mono OR all-stereo (mixed layout rejected), RMS -23 to -18 dB, sample peak < -3 dBFS, noise floor < -60, head 0.5-1.0s / tail 1.0-5.0s room tone → §ACX/Audible Specifications (validate with scripts/acx-check.sh)
• Two-pass loudnorm: measure (print_format=json) → apply measured values; I=-16 podcast / I=-23 EBU → §FFmpeg Mastering Commands

Narration & Dubbing (references/narration-dubbing.md)

• Blog Narration: single voice, quick turnaround → §Blog Narration
• Video Dubbing: emotion matching, multilingual → §Video Dubbing
• Mixed Language: Chinese/English strategy → §Mixed Language Strategy

ChatTTS Workflow (references/chattts-workflow.md)

• Voice Persistence: save/load .pt files for cross-session consistency → §创建并保存声音身份
• Emotion Control: oral/laugh/break parameters + scene presets → §情绪参数体系
• Dual Speaker: two .pt files + shared seed for dialogue → §双人对话工作流

Licensing & Safety (references/licensing-safety.md)

• GREEN/YELLOW/RED: per-tool commercial safety classification → §License Tiers
• Watermarking Traps: hidden markers in generated audio → §Watermarking
• Quality Sabotage: intentional degradation patterns → §Anti-Patterns

---

Anti-Skip Table

Shortcut Attempt	Required Action
"I'll pick an appropriate TTS tool"	MUST use tool-landscape.md decision rules — tool choice depends on use case + hardware + license
"Short reference audio is fine"	MUST check minimum duration per tool in voice-cloning.md — ranges from 3s to 15s
"This tool is open source"	MUST check license tier in licensing-safety.md — open weights ≠ commercial use
"I'll master the audio later"	MUST apply ACX/podcast specs DURING pipeline in audiobook-pipeline.md, not after
"The loudness looks about right"	MUST run scripts/acx-check.sh / scripts/lufs-check.sh — ACX auto-rejects on ANY of 8 specs; eyeballing is not a gate
"I'll use IndexTTS2 duration control"	MUST NOT — token-count duration control is documented but NOT yet enabled in the current release (voice-cloning.md §4); the API will fail
"Any voice will work for now"	MUST set up voice identity BEFORE generation in voice-cloning.md — retrofitting means re-generating all audio