hyperframes-media

# HyperFrames Media CLI commands that create assets (`tts`, `bgm`, `transcribe`, `remove-background`), plus everything needed to consume and animate transcript data in HTML. For placing assets into compositions, see `hyperframes-core`. ## Provider chains (auto-detected from env) **TTS** — `npx hyperframes tts "..."` picks the first available provider: | Order | Provider | Detected when | Word timestamps | | ----- | ----------------------------- | -------------------------------------------- | ---------------------------------------------------------------- | | 1 | HeyGen (Starfish) | `$HEYGEN_API_KEY` / `hyperframes auth login` | **Yes, native** — pass `--words narration.words.json` to capture | | 2 | ElevenLabs | `$ELEVENLABS_API_KEY` set | No — chain `transcribe` after | | 3 | Kokoro-82M (local, 54 voices) | always (no key required) | No — chain `transcribe` after | > If the installed `hyperframes tts` is the local-only build (its `--help` says "Kokoro-82M" and has no `--provider`/`--words` flags), it silently falls back to Kokoro even with `$HEYGEN_API_KEY` set. To force HeyGen regardless of CLI version, use the self-contained `scripts/heygen-tts.mjs` (see `references/tts.md`). **BGM** — `npx hyperframes bgm --duration N`: | Order | Provider | Detected when | | ----- | ------------------------------------------- | --------------------------------------------------- | | 1 | Google Lyria (RealTime) | `$GEMINI_API_KEY` or `$GOOGLE_API_KEY` set | | 2 | MusicGen (`facebook/musicgen-small`, local) | Python `transformers + torch + soundfile` installed | Override either with `--provider <name>`. ## Routing | Task | Read | | ----------------------------------------------------------------- | -------------------------------------------------- | | `npx hyperframes tts` — provider chain, voice IDs, words.json | `references/tts.md` | | HeyGen without the CLI — self-contained REST script (wav + words) | `scripts/heygen-tts.mjs` (see `references/tts.md`) | | `npx hyperframes bgm` — Lyria vs MusicGen, mood prompts, tuning | `references/bgm.md` | | `npx hyperframes transcribe` — Whisper, model rules, output shape | `references/transcribe.md` | | `npx hyperframes remove-background` — transparent cutouts | `references/remove-background.md` | | TTS → transcription → captions (no recorded voiceover) | `references/tts-to-captions.md` | | Caption authoring — style detection, layout, word grouping, exit | `references/captions/authoring.md` | | Transcript handling — input formats, quality gates, cleanup, APIs | `references/captions/transcript-handling.md` | | Caption motion — karaoke, marker effects, audio-reactive | `references/captions/motion.md` | | Model caches, system dependencies, troubleshooting | `references/requirements.md` | ## Non-negotiable rules - **Voice IDs are provider-specific.** `am_michael` is Kokoro-only; HeyGen UUIDs don't work on Kokoro. If you pass `--voice`, also pin `--provider` to avoid silent provider drift when the user's env changes. - **Always pass `--model` to `transcribe`.** The CLI default `small.en` silently translates non-English audio. See `references/transcribe.md` → "Language Rule". - **HeyGen returns word timestamps; ElevenLabs / Kokoro do not.** When you want captions, either pass `--words` to HeyGen and use that JSON directly, or run `transcribe` against the audio file. Don't assume word data is always there. - **Captions consume the flat word-array format** with `{ id, text, start, end }`. See `references/transcribe.md` → "Output Shape". - **`remove-background --background-output` is hole-cut, not inpainted.** For "scene without the person", a different tool is needed. See `references/remove-background.md` → "When NOT the right tool".