Transcription beyond the GuideAnts wrapper contract: transcribe workspace files by path (no upload, no 50 MB gateway cap), pass language hints, and sideload other qwen3-family ASR snapshots from Hugging Face through the wrapper. Use when a transcription needs a language hint, the file is large, or the user wants to try an ASR model the catalog doesn't list.
Run TTS model families audio.cpp supports but GuideAnts does not ship — Qwen3 CustomVoice builtin speakers, VibeVoice multi-speaker dialogue, MioTTS, VoxCPM2, PocketTTS, Vevo2 voice conversion — by downloading the model and spawning a private audiocpp_server in the sandbox. Use when the user names one of these models or asks for a TTS capability the loaded catalog model rejects.
Speaker diarization: figure out who spoke when in a recording and produce a speaker-labeled transcript, by spawning a private audio.cpp engine with the sortformer diarization model and labeling each turn with the local ASR engine. Use when the user asks to diarize a meeting or call recording, separate or identify speakers, or wants a transcript broken down by speaker.
Synthesize speech against the user's own host-native audiocpp_server build (outside the GuideAnts container) — including model families the container binary lacks, like Kokoro or Parakeet forks. TTS only. Use when the user says they run their own audio.cpp server or asks for a family the container build cannot load.
Experimental: use audio.cpp scenarios and models that GuideAnts does not ship — speaker diarization (who spoke when, speaker-labeled transcripts), voice cloning from user-supplied audio, deterministic seeds, builtin speakers, language forcing, and deferred TTS families (Qwen3 CustomVoice, VibeVoice, MioTTS, VoxCPM2, PocketTTS) — by calling the raw audiocpp_server engines directly or spawning a private engine in the sandbox. Use when the user asks for a voice, model, or audio scenario the built-in audio tools reject or cannot express, including diarizing a meeting recording or labeling speakers in a transcript.
Advanced synthesis controls on the loaded audio.cpp TTS model: deterministic output via seed, forcing the spoken language, voice-design from a text description (instructions), and enumerating builtin speaker ids — none of which the built-in GuideAnts audio tools expose. Use when the user wants reproducible audio, a specific language, a described voice, or a list of available speakers.
Clone a voice from a user-supplied reference clip and synthesize speech with it, by calling the raw audio.cpp TTS engine with voice_ref/reference_text — fields the built-in GuideAnts audio tools do not expose. Use when the user wants generated speech to sound like their own voice or another consenting speaker's recording.
Write ML papers for NeurIPS/ICML/ICLR: design→submit.