video-generate

name	video_generate
description	Generate narrated videos from a structured video script JSON, slide images, MiniMax TTS audio, computed timelines, subtitles, and FFmpeg rendering.

video_generate

Skill Purpose

This skill consumes a structured video_script.json plus slide images (typically from ppt_generate output: 1.jpg, 2.jpg, ...) and produces a narrated .mp4 with:

• Line-level MiniMax TTS audio
• Program-computed timeline from real audio durations
• SRT and ASS subtitles
• Scene videos with optional motion and wipe transitions

Python scripts handle TTS, timeline, subtitles, and FFmpeg rendering. This skill does not generate PPT images, write scripts from natural language, or perform VLM review.

Inputs

Input	Description
video_script.json	Structured script with scenes and lines
Slide images	Local paths, usually 1.jpg, 2.jpg, ...
.env	MiniMax API configuration

Outputs

work/
├── audio/s01_l001.mp3
├── scenes/scene_001.mp4
├── subtitles/subtitles.srt
├── subtitles/subtitles.ass
├── timeline.json
├── generation_state.json
└── final.mp4

Workflow

1. Agent writes video_script.json (no timestamps)
2. Ensure slide images exist at image_path values
3. Configure .env with MiniMax API key
4. Run scripts/render_video.py
5. Review output video; fix script and re-render failed lines if needed

video_script.json Schema

See references/video_script_schema.md.

Key rules:

• One scene per slide image
• One or more lines per scene — each line gets its own TTS call
• pause_after uses fixed enum: none, short, normal, emphasis, long
• Optional subtitle for display text different from TTS text

Timing Rules

• Do not write explicit timestamps in the script
• Do not estimate scene duration from character count
• Program computes timeline from:
  • Real TTS audio duration (extra_info.audio_length or ffprobe)
  • pause_after gaps
  • timing.scene_intro_pause / scene_outro_pause
  • Transition duration between scenes

TTS Rules

• Line-level TTS only — never send a whole paragraph in one call
• Model default: speech-2.8-hd
• Read API config from .env only — never hardcode keys
• Prefer emotion: calm for formal narration
• text_normalization: true recommended for Chinese/English mixed text

Pause Rules

pause_after	Duration
none	0.0s
short	0.2s
normal	timing.line_gap (default 0.35s)
emphasis	timing.emphasis_gap (default 0.6s)
long	1.0s

MiniMax inline pause <#0.35#>: use only inside a line for sentence pauses. Do not use for line-to-line or scene gaps.

MiniMax Interjection Tags

Only for speech-2.8-hd / speech-2.8-turbo. Official tags only:

(laughs), (chuckle), (coughs), (clear-throat), (groans), (breath), (pant), (inhale), (exhale), (gasps), (sniffs), (sighs), (snorts), (burps), (lip-smacking), (humming), (hissing), (emm), (sneezes)

Use sparingly in formal presentations. Subtitles automatically strip these tags unless explicit subtitle is provided.

MiniMax Emotion Rules

Allowed: happy, sad, angry, fearful, disgusted, surprised, calm, fluent, whisper

Prefer calm for narration. Do not invent free-form emotion labels.

Subtitle Rules

• Optional subtitle field per line
• If missing, display text = cleaned text (strip interjection tags and <#x#> markers)
• One subtitle cue per line, aligned to TTS audio — no subtitle during pause gaps

Visual Rules

Motion	Effect
still	Static slide image
slow_zoom_in	Subtle Ken Burns zoom (1.0 → 1.03)

Transition	Effect
none	Hard cut
wipe_left_to_right	New slide wipes in from left

Cache Behavior

TTS audio is cached per line. Re-run skips unchanged lines unless --force-tts.

Hash includes: text, voice, model, speed, volume, pitch, emotion, normalization settings.

Prohibited Actions

• Do not hardcode API keys
• Do not use character count to determine scene duration
• Do not use whole-paragraph TTS
• Do not let the LLM write timeline timestamps
• Do not delete cached audio unless forced
• Do not implement unrelated skills (template_ingest, ppt_generate)

Render Command

python scripts/render_video.py \
  --script data/video_scripts/sp500_ai.json \
  --output data/videos/sp500_ai/final.mp4 \
  --work-dir data/videos/sp500_ai/work

Final Summary Format

Video render completed.

Script: data/video_scripts/sp500_ai.json
Output: data/videos/sp500_ai/final.mp4
Work dir: data/videos/sp500_ai/work

Scenes: 12
Lines: 36
TTS rendered: 8
TTS skipped: 28
TTS failed: 0
Final duration: 04:32

Generated:
- timeline.json
- subtitles/subtitles.srt
- subtitles/subtitles.ass
- final.mp4