cmm-remotion-video

name	cmm-remotion-video
description	The render/execution engine of the CMM video stage — turns an article, idea, topic, research brief, or an upstream cmm-video-production video_package into a Xiaohongshu/Douyin-ready vertical short video via a structured AI-to-Remotion workflow (TTS narration, synced captions, scene visuals, cover, publish copy). Use it to actually render/produce/package the video. When the format is already fixed (e.g. a vertical card/typography video) it can be called directly and self-derives its script in standalone mode; when cmm-video-production has produced a video_package, it consumes that instead of re-scripting. For choosing format/engine first, use cmm-video-production. Especially for 30-130s Chinese social videos.
version	0.1.0
metadata	{"tags":"remotion, short-video, xiaohongshu, douyin, script, storyboard, tts, captions"}

Remotion Short Video Skill

Use this skill to convert user-provided articles, ideas, or topics into a publish-ready vertical short video package. The skill should keep creative reasoning and deterministic rendering separate:

• The active agent handles understanding, research, scripting, storyboarding, and quality judgment.
• The bundled workflow and Remotion project handle structured files, media assets, timing, rendering, cover export, and packaging.

This separation lets the skill run inside Claude Code, Codex, OpenClaw, or another agent without requiring an internal LLM API. If an LLM API is available, it can be used as an optional backend for standalone or batch runs.

Default Target

Unless the user specifies otherwise, use these defaults:

• Platform: Xiaohongshu and Douyin compatible
• Format: vertical video, 3:4 (1080×1440) by default — 账号选定的默认竖版比例。 (注:抖音/小红书原生全屏是 9:16;本账号默认用 3:4,需全屏再显式指定 9:16。)
• Duration: 90-130 seconds
• FPS: 30
• Language: Chinese
• Style: knowledge explainer or opinion analysis
• Visual approach: AI images or sourced stills, animated typography, simple data graphics, keyword-highlight captions
• Outputs: video.mp4, cover.png, script.md, captions.srt, publish.md, metadata.json

Core Workflow

Follow this sequence:

1. Parse the input.

   • Identify whether the user gave an article, a rough idea, a topic, or a partial script.
   • If an upstream cmm-video-production video_package is provided (script, storyboard, captions, asset specs), consume it — adopt its scenes/voiceover/ visual direction and skip re-deriving the script (steps 2-4); jump to producing video-plan.json. Only fill gaps the package leaves open. When no video_package is supplied (standalone use), derive everything yourself via steps 2-4 below.
   • Extract topic, audience, platform, tone, expected duration, and constraints.
   • Ask only if the missing information materially changes the output; otherwise use the defaults.

2. Research and analyze.

   • For current, factual, financial, legal, medical, technical, or news-like topics, verify with reliable sources before writing the script.
   • Capture the angle, audience pain point, core claim, supporting evidence, risk notes, and recommended narrative structure.
   • Save or produce an analysis.json-compatible structure.

3. Write the short-video script.

   • Build for a 90-130 second spoken video, not an article summary.
   • Use a strong hook in the first 3-6 seconds.
   • Keep one central thesis and 2-4 supporting points.
   • Include voiceover, on-screen caption text, visual direction, emotional tone, and estimated duration per scene.

4. Convert the script into a storyboard.

   • Split the video into scenes.
   • Each scene should include narration, caption text, visual prompt or asset direction, layout type, motion style, and transition intent.
   • Prefer 5-9 scenes for a 2 minute video.

5. Generate audio and timing.

   • Treat narration audio as the master timeline.
   • Generate TTS audio or instruct the user/agent to generate it using the configured TTS provider.
   • Default to Volcengine 火山 TTS(账号默认,需 VOLCENGINE_TTS_* 凭证)。火山不可用或失败时回退 Edge TTS(免费、无需密钥)。火山已知坑(角色音色可能未开通 / HTTP 无词时间戳→字幕对轴差)一旦出现,即回退 Edge。HTTP adapter 可选。
   • Use transcription or forced alignment to produce word-level or sentence-level timestamps.
   • Use those timestamps to build captions and scene boundaries.

6. Prepare visuals.

   • Use AI-generated still images, sourced images/video, or Remotion-native graphics.
   • For MVP work, prefer still images plus motion, typography, charts, and transitions over AI-generated video clips.
   • Check licensing and factual fidelity when using sourced assets.
   • AI 配图：node scripts/generate-images.mjs <plan.json> [--cover] [--scenes] [--force] 用 GPT-Image-2 兼容 API（凭证在 ~/.cmm/.env 的 GPT_IMAGE2_*）生成封面背景 （写入 cover.backgroundAsset）和场景图（apple-text-video 用 16:9、image-overlay 用 9:16， 写入各 scene.visual.asset）。dark-card 不需要场景图，只生成封面。

7. Build video-plan.json.

   • Convert analysis, script, captions, audio, visual assets, style, and cover plan into a single Remotion input file.
   • Use the schema in references/video-plan-schema.md.
   • Choose the template and per-scene effects using references/templates.md: pick style.template from the content type, then fill the fields that template needs (e.g. scene.data for data-punch, scene.visual.asset for image templates) and set each scene's transition. Missing fields silently disable the matching animation, so fill them deliberately.

8. Render and package.

   • Generate a first-frame preview first when the user has not explicitly confirmed full rendering.
   • Render the video through Remotion.
   • Render the cover still.
   • Export the publish package with script, captions, metadata, and platform copy.

9. Quality check.

   • Verify duration, audio presence, caption timing, missing assets, unreadable text, aspect ratio, and cover readability.
   • If issues are detected, fix the plan or assets and render again.

Recommended File Layout

For each video job, create a job folder:

jobs/<slug>/
  input.md
  analysis.json
  script.json
  storyboard.json
  audio/
    voiceover.mp3
    bgm.mp3
  captions/
    captions.json
    captions.srt
  assets/
    scene-01.png
    scene-02.png
  video-plan.json
  output/
    video.mp4
    cover.png
    script.md
    publish.md
    metadata.json

Dependency Policy

Required:

• Node.js
• Remotion
• FFmpeg and ffprobe
• TTS provider: Volcengine 火山 is the account default (creds required); Edge TTS is the fallback when 火山 is unavailable or fails. HTTP adapter optional.
• Caption alignment or transcription capability

Recommended:

• Image generation or image sourcing capability
• Search API or browser research capability

Optional:

• LLM API for standalone or batch execution
• AI video generation API
• Stock media API
• Automatic publishing API

Agent vs API Responsibility

When running inside Claude Code, Codex, or OpenClaw:

• Use the active agent model for content understanding, research synthesis, script writing, and storyboard planning.
• Do not require an internal LLM API unless the user asks for standalone/batch automation.
• Prefer deterministic scripts for validation, media copying, Remotion rendering, and packaging.

When running standalone:

• Read provider keys from environment variables.
• Keep provider choice configurable.
• Never hardcode API keys.

Quality Bar

The result is acceptable only if:

• Video is vertical 3:4 (1080×1440) by default, or the requested aspect ratio.
• Duration is close to target, normally 90-130 seconds.
• Voiceover, captions, and scene timing are aligned.
• Captions are readable on mobile.
• Visuals support the narration instead of being generic decoration.
• Cover is readable at small thumbnail size.
• publish.md includes platform-ready title, body copy, tags, and optional comment prompt.

References

• Read references/mvp-spec.md before designing or implementing the first version.
• Read references/video-plan-schema.md before generating Remotion input data.
• Read references/templates.md to choose a template and activate its effects.
• Use examples/input.md as the first smoke-test input.