| name | voice |
| description | VoiceCLI assistant — author TTS scripts, generate speech, clone voices, transcribe audio, manage samples. Knows engine capabilities, markdown format, and all CLI commands. Triggers: "voice" | "voicecli" | "speech" | "generate speech" | "clone voice" | "transcribe" | "TTS" | "text to speech" | "voice script". |
| version | 0.6.0 |
| allowed-tools | Read, Edit, Write, Bash, Glob, Grep |
VoiceCLI — Unified Voice Assistant
Purpose: Author TTS markdown scripts, run voicecli CLI commands, manage the full voice pipeline.
VoiceCLI is a Python CLI for local voice generation (TTS) and transcription (STT) with multiple AI backends.
Auto-Discovery
Before the first command in a session, determine how to invoke voicecli:
if command -v voicecli &>/dev/null; then
echo "voicecli is in PATH — use it directly"
else
if [ -n "$VOICECLI_DIR" ] && [ -f "$VOICECLI_DIR/src/voicecli/cli.py" ]; then
true
else
for d in . .. ../voiceCLI ~/projects/voiceCLI; do
test -f "$d/src/voicecli/cli.py" && VOICECLI_DIR="$(cd "$d" && pwd)" && break
done
fi
[ -z "$VOICECLI_DIR" ] && echo "voiceCLI not found — run 'voicecli init' or set VOICECLI_DIR" && exit 1
echo "Using fallback: cd $VOICECLI_DIR && uv run voicecli"
fi
Cache the result for the session:
- If in PATH: use
voicecli <args> directly (works from any directory)
- Fallback: use
cd "$VOICECLI_DIR" && uv run voicecli <args>
All examples below use voicecli — substitute with the fallback form if needed.
First Use
On the first invocation of any voice/TTS command in a session, proceed directly with the request.
voicecli doctor is a diagnostic/installation tool — only run it when the user explicitly asks for a system check or when troubleshooting an error. Never run it automatically.
Engine Capability Matrix
Five engines with different strengths. Pick based on user intent:
| Capability | Qwen | Qwen-Fast | Chatterbox Multilingual | Chatterbox Turbo | Voxtral |
|---|
| instruct (free-form emotion) | yes | yes | no | no | no |
| segments (per-section overrides) | yes | yes | yes | yes | yes |
| paralinguistic tags | to_instruct | to_instruct | strip | native | strip |
| exaggeration (0.25–2.0) | no | no | yes (per-segment) | yes (per-segment) | no |
| cfg_weight (0.0–1.0) | no | no | yes (per-segment) | yes (per-segment) | no |
| flow_steps (3=fast, 8=quality) | no | no | no | no | yes (per-segment) |
| cfg_alpha (1.0–1.2) | no | no | no | no | yes (per-segment) |
| temperature (0.0–2.0) | yes | yes | yes | yes | no |
| top_p (0.0–1.0) | yes | yes | yes | yes | no |
| min_p (0.0–1.0) | no | no | yes | yes | no |
| repetition_penalty | yes | yes | yes (def 2.0) | yes (def 1.2) | no |
| language (multi) | yes (per-segment) | yes (per-segment) | yes (per-segment) | no (EN only) | yes (9 langs) |
| built-in voices | yes (9, per-segment) | yes (9, per-segment) | no | no | yes (20 presets) |
| voice cloning | yes | yes | yes | yes | no |
| CUDA graph acceleration | no | yes (5-9x speedup) | no | no | no |
| VRAM usage | ~3.5 GB | ~3.5 GB | ~1.8 GB | ~1.8 GB | ~3.7 GB (int4) |
Engine Selection Guide
| User wants… | Engine |
|---|
| Multi-language + voice cloning | chatterbox |
| English + emotion tags ([laugh], [sigh]) | chatterbox-turbo |
| Free-form emotion instructions | qwen or qwen-fast |
| Named built-in voices (9 Qwen) | qwen or qwen-fast |
| Many voice presets (20, 9 languages) | voxtral |
| Per-section emotion changes | qwen or qwen-fast |
| Per-section expressiveness | chatterbox or chatterbox-turbo |
| Bilingual content (language switching) | qwen, qwen-fast, chatterbox, or voxtral |
| Fastest generation (CUDA graph accel.) | qwen-fast |
| High-quality multilingual (no cloning needed) | voxtral |
Tag Handling
Tags: [laugh] [chuckle] [cough] [sigh] [gasp] [groan] [sniff] [shush] [clear throat]
- Turbo: tags kept in text, engine processes them natively
- Qwen: tags split into segments with mapped instruct (e.g.
[laugh] → instruct: "Laughing")
- Multilingual: tags stripped (use exaggeration/cfg_weight instead)
The code translator handles this automatically — write tags in the universal format.
Qwen Voices
Vivian, Serena, Uncle_Fu, Dylan, Eric, Ryan (default), Aiden, Ono_Anna, Sohee
Voxtral Voices
20 presets across 9 languages with auto-selection by language:
neutral_female (default), neutral_male, cheerful_female, casual_female, casual_male, fr_male, fr_female, de_male, de_female, es_male, es_female, it_male, it_female, pt_male, pt_female, nl_male, nl_female, ar_male, hi_male, hi_female
When language is set (e.g. language: French), Voxtral auto-resolves to the matching voice (e.g. fr_female). Explicit voice overrides this.
User Config (voicecli.toml)
Optional file for default settings. Discovery: voicecli walks up from the CWD to $HOME looking for voicecli.toml — place it in a parent directory (e.g. ~/projects/voicecli.toml) to share it across multiple projects. A warning is printed to stderr if no file is found anywhere.
[defaults]
language = "French"
engine = "qwen"
accent = "Léger accent du sud provençal"
personality = "Voix calme, douce et flamboyante"
exaggeration = 0.7
cfg_weight = 0.3
segment_gap = 200
crossfade = 50
Structured instruct parts (accent, personality, speed, emotion) auto-compose into instruct.
Raw instruct bypasses composition. Write instruct parts in the target language.
Segment propagation: toml structured parts are backfilled into .md segments where frontmatter didn't set them, so a script with no frontmatter still inherits instruct from voicecli.toml.
Priority: CLI flag > markdown frontmatter > voicecli.toml > hardcoded default
Unified Markdown Format
One .md file works across all engines — the code translator adapts per engine.
Frontmatter (all optional)
---
language: French
voice: Ryan
engine: qwen
accent: "Provençal"
personality: "Calme"
speed: "Rythme posé"
emotion: "Chaleureuse"
instruct: "Parle avec colère"
exaggeration: 0.75
cfg_weight: 0.3
flow_steps: 8
cfg_alpha: 1.2
temperature: 0.8
top_p: 1.0
min_p: 0.05
repetition_penalty: 2.0
segment_gap: 200
crossfade: 50
---
Per-Section Directives
All frontmatter fields can be overridden per-section using <!-- key: value --> comments.
Multiple keys can be combined in a single comment: <!-- emotion: "Passionnée", speed: "Rapide" -->.
Directives accumulate before a text block and apply to the text that follows.
Each section inherits frontmatter defaults, overridden by its inline directives.
Commas inside quoted values are safe: <!-- emotion: "Passionnée, mais contenue" -->.
Available: accent, personality, speed, emotion, instruct, exaggeration, cfg_weight, flow_steps, cfg_alpha, temperature, top_p, min_p, repetition_penalty, language, voice, segment_gap, crossfade
Segment Transitions
| gap | crossfade | Result |
|---|
| 0 | 0 | Direct concat (default) |
| >0 | 0 | Hard cut, silence, hard cut |
| 0 | >0 | Fade-out then fade-in (no silence) |
| >0 | >0 | Fade-out, silence, fade-in |
Body Features
- Standard markdown (auto-stripped to plain text)
<!-- emotion: "Passionnée", speed: "Rapide" --> — multi-key on one line (preferred)<!-- accent: "Parisien" --> — single-key still works<!-- instruct: "Speak seriously" --> — raw instruct bypass (Qwen)<!-- exaggeration: 0.8, cfg_weight: 0.3 --> — per-section expressiveness (Chatterbox)<!-- flow_steps: 8, cfg_alpha: 1.2 --> — per-section Voxtral quality tuning<!-- temperature: 0.6, repetition_penalty: 1.5 --> — per-section sampling (Qwen/Chatterbox)<!-- language: Japanese, voice: Ono_Anna --> — per-section language + voice switch<!-- segment_gap: 500, crossfade: 100 --> — per-section transition control[laugh] [sigh] etc. — paralinguistic tags (see tag handling above)
Example Script
---
language: French
accent: "Léger accent provençal"
personality: "Calme et douce"
emotion: "Chaleureuse"
exaggeration: 0.7
cfg_weight: 0.3
segment_gap: 200
---
Welcome everyone. [laugh] This is going to be fun!
<!-- emotion: "Passionnée", segment_gap: 500 -->
Now let me tell you something important. [sigh] It has been a long road.
<!-- language: Japanese, voice: Ono_Anna, crossfade: 100, segment_gap: 0 -->
A section in Japanese with a different voice, crossfaded in.
Recommended Settings
- Passionate speech: exaggeration 0.7–0.8, cfg_weight 0.3
- Cross-language cloning: cfg_weight 0.0 (reduces accent bleed)
- Default: exaggeration 0.5, cfg_weight 0.5
- Natural pauses between sections: segment_gap 200–300
CLI Commands Reference
Generate (built-in voices)
voicecli generate "Hello world"
voicecli generate "Bonjour" -e chatterbox --lang French
voicecli generate "text" -e chatterbox-turbo
voicecli generate "text" -e qwen-fast
voicecli generate "Bonjour" -e voxtral
voicecli generate script.md
voicecli generate article.txt
voicecli generate script.md --mp3
voicecli generate "text" -v Ono_Anna --lang Japanese
voicecli generate script.md --segment-gap 300
voicecli generate script.md --crossfade 50
voicecli generate script.md --plain
voicecli generate "text" --fast
voicecli generate "Long text" --chunked
voicecli generate "Long text" --chunked --chunk-size 300
voicecli generate article.txt --chunked
voicecli generate "Bonjour" --via-nats
voicecli generate script.md --via-nats --timeout 90
--via-nats (or VOICECLI_VIA_NATS=1) sends the request to the hub satellite over NATS and writes the returned WAV locally — useful from hosts without GPU/model. Requires NATS_URL + an NKey seed. Full guide: docs/NATS-SERVE.md#client-side.
Clone (voice cloning)
voicecli clone "text" --ref voice.wav
voicecli clone "text"
voicecli clone script.md --mp3
voicecli clone article.txt --chunked
voicecli clone "text" -e chatterbox --lang French
voicecli clone "text" -e qwen-fast
voicecli clone script.md --segment-gap 200
voicecli clone script.md --plain
voicecli clone "text" --fast
voicecli clone "Long text" --chunked
Transcribe (speech-to-text)
voicecli transcribe audio.wav
voicecli transcribe audio.wav --json
voicecli transcribe audio.wav -m large-v3
voicecli transcribe audio.wav -l fr
Models: tiny, base, small, medium, large-v3, large-v3-turbo (default)
Listen (real-time mic STT)
voicecli listen
voicecli listen -m 2.6b
Samples
voicecli samples list
voicecli samples add file.wav
voicecli samples record name -d 30
voicecli samples use name.wav
voicecli samples active
voicecli samples remove name.wav
Daemon (warm model — fast generation)
voicecli serve
voicecli serve --engine qwen
voicecli serve --engine qwen --fast
generate and clone automatically route to the daemon when it's running — no extra flags. Falls back silently to standalone if the socket isn't present.
NATS Serve (distributed queue-group satellite)
voicecli nats-serve {tts,stt} runs voicecli as a long-lived NATS queue-group subscriber instead of a local Unix-socket daemon — the hub and voicecli talk over NATS subjects rather than a local socket, so they can live on different hosts and be restarted independently.
voicecli nats-serve tts
voicecli nats-serve stt
voicecli nats-serve tts --allow-coexist
voicecli nats-serve stt --model large-v3-turbo
When to prefer NATS vs Unix socket:
| Scenario | Use |
|---|
| Hub and voicecli on the same host, no network boundary | voicecli serve / stt-serve (Unix socket) |
| Hub and voicecli on different hosts | voicecli nats-serve tts / nats-serve stt |
| Add satellite capacity without changing hub config | nats-serve (add queue-group members) |
| Production with hub managed separately from GPU worker | nats-serve |
Environment variables (read at startup — restart to apply):
| Variable | Default | Purpose |
|---|
NATS_URL | — (required) | NATS server URL, e.g. nats://lyra-nats:4222 |
NATS_NKEY_SEED_PATH | — (required for nkey auth) | Path to NKey seed file — must be 0600 |
NATS_CA_CERT | — (optional) | PEM CA certificate for TLS verification |
VOICECLI_ENGINE / LYRA_TTS_ENGINE | from voicecli.toml | TTS engine override (qwen, qwen-fast, chatterbox, …) |
VOICECLI_MODEL | large-v3-turbo | STT model name (STT-only) |
VOICECLI_MAX_CONCURRENT | TTS 1 / STT 2 | Parallel request slots — keep at 1 on single-GPU hosts |
VOICECLI_HEARTBEAT_INTERVAL | 5.0 | Seconds between heartbeats (must stay ≤ 5) |
VOICECLI_DRAIN_TIMEOUT | 30 | Graceful-shutdown drain window (exit 3 if exceeded) |
VOICECLI_REJECT_WHEN_FULL | unset | 1 → reject new requests when all slots busy |
VOICECLI_ALLOW_COEXIST | unset | 1 → bypass VRAM-sequencing guard (dev/large-VRAM hosts) |
Full operator reference: docs/NATS-SERVE.md.
Dictate (STT daemon + overlay)
voicecli dictate
voicecli dictate cancel
voicecli dictate next-mode
voicecli dictate status
WSL2 shortcuts (via AHK script): Alt+Shift+Space = toggle, Alt+Shift+Tab = next-mode, Alt+Shift+Esc = cancel.
Auto-paste: enabled by auto_paste = true in [stt] section of voicecli.toml.
Dictate NATS (cross-host toggle)
Routes mic audio to a remote STT satellite. Toggle: first call starts, second call stops + transcribes + clipboard.
voicecli dictate nats
voicecli dictate nats --lang fr
voicecli dictate nats --model large-v3
voicecli dictate nats --paste
Bind ~/.local/bin/voicecli-dictate-nats (wrapper with env vars) to a global shortcut.
On COSMIC (Pop!_OS): System Settings → Keyboard → Custom Shortcuts.
Full setup: docs/STT-SETUP.md#nats-dictation.
Utilities
voicecli mp3 TTS/voices_out/file.wav
voicecli mp3 TTS/voices_out/file.wav -b 320
voicecli voices
voicecli voices -e chatterbox
voicecli engines
voicecli emotions
voicecli doctor
voicecli init
Project Layout
TTS/
texts_in/ — authored .md scripts (git-tracked)
voices_out/ — generated WAV/MP3 (gitignored)
samples/ — voice samples for cloning (gitignored)
STT/
audio_in/ — audio files to transcribe (gitignored)
texts_out/ — transcription results (gitignored)
Chunked Output
Use --chunked for long texts to enable progressive sending via Telegram. Each chunk is saved as a separate numbered file (prefix_001.wav, prefix_002.wav...) and a .done sentinel is written when complete.
- For plain text: splits at paragraph/sentence boundaries (~500 chars per chunk by default)
- For .md scripts with segments: each segment becomes a chunk
--chunk-size N adjusts target chunk size in characters (~15 chars/sec of speech)
Always use --chunked when generating from Telegram so the bot can send audio progressively.
Output format is WAV by default. Do NOT add --mp3 unless the user explicitly asks for it.
Workflow
Intent → Action Mapping
- User wants to create a speech → Author a
.md script in TTS/texts_in/, then run generate (default) or clone (only if explicitly requested)
- User wants quick speech → Run
generate with inline text
- User explicitly wants voice cloning → Ensure sample exists (
samples list/samples add), then clone. Never default to clone — the presence of an active sample does NOT imply the user wants cloning.
- User wants to transcribe → Run
transcribe on audio file
- User wants to edit a script → Read existing
.md, apply changes respecting the format
- User asks about capabilities → Consult engine matrix, recommend engine
Script Authoring Rules
Before writing any script, read voicecli.toml to discover the user's configured defaults (engine, language, accent, personality, etc.). Only add frontmatter fields that override those defaults — never duplicate what toml already sets.
When writing .md scripts:
- Use all features freely — the translator adapts per engine (unsupported fields are nulled per-segment)
- Use
<!-- key: value, key2: value2 --> directives for per-section overrides — combine multiple keys on one line (accent, personality, speed, emotion, instruct, exaggeration, language, voice, segment_gap, crossfade)
- Place
[tags] naturally in the text flow — before or after the relevant phrase
- Keep text segments under ~250 chars for Chatterbox engines (they chunk at sentence boundaries, but shorter is safer)
- For multilingual content, set
language: in frontmatter and override per-section with <!-- language: ... -->
- Scripts go in
TTS/texts_in/ by convention
- Use
segment_gap and crossfade in frontmatter for global defaults, override per-section for fine control
- Write all instruct parts (accent, personality, speed, emotion, instruct) in the target language
Telegram Integration
When running inside the 2ndBrain Telegram bot, voice files auto-send to the user's chat after generation. Always use --chunked so the bot can send audio progressively. Skip send_voice_telegram.py — just generate with --chunked and report what you created.
--plain Flag
Use --plain on generate or clone to strip all [tags] and ignore <!-- directives -->:
- On a
.md file: frontmatter settings (language, voice, instruct) are still applied; only per-section directives and tags are ignored
- On raw text or
.txt file: tags are stripped from the text
- Can be set permanently in
voicecli.toml as plain = true
Key Constraints
- Qwen clone does NOT support instruct — only generate does
- Chatterbox Turbo is English-only
- Chatterbox Multilingual strips all paralinguistic tags
- Voxtral does NOT support voice cloning (codec encoder not released by Mistral)
- Voxtral does NOT support instruct, exaggeration, or cfg_weight — voice character comes from preset selection
- Voxtral (3.7 GB VRAM) cannot coexist with the Qwen daemon (8.5 GB) on 16 GB GPU — stop daemon first
- Both Chatterbox engines have a ~40s generation cutoff (handled by auto-chunking per segment)
- Clone falls back to active sample when
--ref is omitted
qwen-fast has same capabilities as qwen but uses CUDA graph acceleration (5-9x speedup after warmup)--fast flag uses the smaller 0.6B model (Qwen/Qwen-fast only) — faster but lower quality- Base instruct is preserved in tag-split segments (e.g.
[laugh] on Qwen keeps the original instruct alongside the tag instruct)
- Daemon (
voicecli serve) only accelerates Qwen engines — Chatterbox and Voxtral always run standalone
- Daemon socket:
~/.local/share/voicecli/daemon.sock; if absent, generate/clone fall back silently
$ARGUMENTS
