Nuva-Lab

Align text to audio timestamps using Qwen3-ForcedAligner (~30ms precision)

Full pipeline from manga panels to animated video. Two audio modes: dialogue (Qwen3-TTS + karaoke captions) or music (ElevenLabs + rolling lyrics).

Burn karaoke captions into video using FFmpeg ASS subtitles (~20s for 16s video)

Generate multi-panel manga from character reference and story beats

Generates original background music using Google's Music Generation API. Creates soundtracks matched to scene mood and timing.

Concatenates video clips and optionally adds background music using FFmpeg.

Generates anime character images from photo analysis using Nano Banana Pro (gemini-3-pro-image-preview). Creates character sheets with full body and portrait views at 2K resolution.

Generates speech audio from dialogue using Gemini TTS. Supports multi-speaker conversations with consistent voices per character.

Generate karaoke-style word-level timestamps by aligning script text to audio using Qwen3-ForcedAligner + jieba for Chinese word segmentation. Use when the user says 'align captions', 'karaoke timestamps', 'word timestamps', 'caption alignment', 'sync text to audio'.

Analyze raw video content using Gemini to identify speakers, topics, key moments, and potential clip opportunities

Audio processing utilities - noise reduction, normalization, enhancement

Extract a video segment using FFmpeg with precise start/end times

Find naturally clean, coherent video segments worth keeping (selection over repair)

Text-guided audio source separation using SAM-Audio via mlx-audio

ASR with ~30ms timestamp precision using Qwen3-ASR + ForcedAligner

Transcribe a video clip using Gemini to get timestamped segments for captions