remotion-video-development

name	remotion-video-development
description	Use when implementing Remotion video scenes from an approved design spec. Executes scene-by-scene with flex-first layout, subtitle-based timing, multi-segment audio playback, and per-scene verification. Requires completed design from remotion-video-design. Also use when regenerating voiceover segments, fixing pronunciation, or re-aligning timeline.

Remotion Video Development

Overview

Implement a Remotion video scene-by-scene based on an approved design spec. Each scene follows: static layout → animations → verify. Flex flow by default, absolute positioning only for overlays.

Core principle: Code each scene to match the timeline.md exactly. Verify per scene, not at the end.

File-first workflow: After implementing each scene, the file is on disk. For quick tweaks (timing constants, colors, spacing), tell the user which file and which constants to edit — faster than another conversation round.

Announce at start: "Using remotion-video-development to implement the video."

CWD discipline: All shell commands MUST use absolute paths or be prefixed with cd to the project root. The Remotion CLI outputs paths relative to CWD, and npx remotion render --scale=2 will place output in $CWD/output/ — not necessarily the project's output/ directory.

# ALWAYS start every Bash call with:
cd /path/to/project-root && <actual command>

# Or use absolute paths everywhere:
npx remotion render CompositionID /absolute/path/to/output.mp4

4K rendering: Use --scale=2 flag — keeps internal 1920x1080 resolution, outputs 3840x2160. No code changes needed.

Pipeline position: Second step. Requires completed design from remotion-video-design. After implementation, transition to remotion-video-review.

CWD discipline — ALL Bash commands in this skill MUST:

• Use absolute paths for all file operations (output paths, file lists)
• Start with cd /absolute/path/to/project-root && when the context might have drifted
• Never rely on relative CWD for npx remotion render output paths — CWD can change between commands

Rendering resolution:--scale=2 to the npx remotion render command to produce 4K output

• No changes needed to WIDTH/HEIGHT, font sizes, or layout — Remotion handles scaling internally

Prerequisites

Before starting, confirm these exist:

• docs/video-design.md — approved design spec
• docs/timeline-estimated.md — estimated timeline (Phase 2 of design)
• src/styles/theme.ts — design system constants (with estimated durations)
• public/images/ — all image assets with ASCII names
• voiceover-text.json — finalized voiceover script

Audio files are NOT required at start. They are generated in Phase D, after all scenes are coded.

Process Flow

digraph dev {
  rankdir=TB;
  "Scaffold project" [shape=box];
  "Implement scene N" [shape=box];
  "N: Static layout" [shape=box];
  "N: Add animations" [shape=box];
  "N: tsc --noEmit" [shape=box];
  "N: Render 1 frame" [shape=box];
  "Layout correct?" [shape=diamond];
  "More scenes?" [shape=diamond];
  "Generate voiceover" [shape=box];
  "Replace precise timeline" [shape=box];
  "Add Audio components" [shape=box];
  "Compose TransitionSeries" [shape=box];
  "Full preview" [shape=box];
  "Done" [shape=doublecircle];

  "Scaffold project" -> "Implement scene N";
  "Implement scene N" -> "N: Static layout";
  "N: Static layout" -> "N: Add animations";
  "N: Add animations" -> "N: tsc --noEmit";
  "N: tsc --noEmit" -> "N: Render 1 frame";
  "N: Render 1 frame" -> "Layout correct?";
  "Layout correct?" -> "N: Static layout" [label="no"];
  "Layout correct?" -> "More scenes?" [label="yes"];
  "More scenes?" -> "Implement scene N" [label="yes"];
  "More scenes?" -> "Generate voiceover" [label="no"];
  "Generate voiceover" -> "Replace precise timeline";
  "Replace precise timeline" -> "Add Audio components";
  "Add Audio components" -> "Compose TransitionSeries";
  "Compose TransitionSeries" -> "Full preview";
  "Full preview" -> "Done";
}

Key change from previous workflow: Voiceover generation and precise timeline replacement happen AFTER all scenes are coded, not before. This allows:

1. Coding starts immediately with estimated subtitles
2. Layout/animation iteration doesn't depend on audio
3. Audio is generated once, after text is stable

Per-Scene Implementation

Phase A-0: Layout Pre-Confirmation

Before writing ANY code, present a text-based layout diagram to the user. This prevents the most common cause of rework: the user's mental picture not matching the implementation.

Steps:

1. Read the scene's design spec and voiceover text
2. Identify all visual elements and their spatial relationships
3. Create an ASCII layout diagram showing:
   • Container boundaries (CONTAINER_W × CONTAINER_H)
   • Relative positioning (title at top, left/right split, etc.)
   • How elements connect (arrows, lines, direction indicators)
   • Approximate proportions (60/40, centered column, etc.)
4. Present diagram to user and confirm before proceeding
5. Do NOT write code until layout is confirmed

Example format:

┌──────────────────────────────────────────────┐
│                  标题                         │
├─────────────────────┬────────────────────────┤
│    Element A         │   Element B            │
│    (60%)             │   (40%)                │
│                      │   - sub-element        │
│                      │   - sub-element        │
└─────────────────────┴────────────────────────┘

If the scene involves multiple cards/panels with connecting arrows, the diagram MUST show which card connects to which and in what direction (→, ↓, ↑, etc.).

Phase A: Static Layout

Build the scene with all elements visible (no animations, no FadeIn). Verify element sizes, positions, and text content match the CONFIRMED layout diagram.

Layout rules:

DEFAULT:  flex flow for ordered content
EXCEPTION: absolute positioning ONLY for overlay elements (floating badges, decorations)
ALIGNMENT: alignItems: "flex-end" for bottom-align, NOT hardcoded heights
CENTERING: margin: "0 auto" + fixed width, NOT percentage
WINDOWS:  macOS title bar (3 dots) for screenshot containers

Layout hard rules (from layout-templates.md in design skill):

完整规范见 ~/.claude/skills/remotion-video-design/layout-templates.md。核心规则摘要：

1. 80% content area: 1536×864 + margin auto。lint 检查。
2. 间距只用 SP 变量: SP.xs(8)/sm(16)/md(24)/lg(40)/xl(64)。lint 检查。
3. 标题区 80px: FS.title + 居中 + 下划线装饰。lint 检查。
4. 字号底线: body≥24px, title≥40px, 最小18px。lint 检查。
5. 卡片规范: 白底+2px border+12px圆角+padding ${SP.lg}px ${SP.md}px。
6. 对齐按模板: 每种模板的 flexDirection/alignItems/gap 是硬性的，见速查表。
7. Screenshot zero whitespace: line-height: 0 + height: auto。禁止 objectFit:contain。
8. Layout diversity: 相邻场景不同模板，同模板≤2次/视频。

运行 python ~/.claude/skills/remotion-video-development/layout-lint.py . 检查所有规则。

Common components to reuse:

• FadeIn — wraps elements with fade animation + optional direction/distance
• Typewriter — character-by-character text reveal
• Screenshot window — title bar + Img with contain + caption

Phase B: Add Animations

Apply the estimated timeline from docs/timeline-estimated.md. At this stage, use character-ratio for all timing:

// T constants — estimated from character-ratio
// Will be replaced with precise TTS timestamps after voiceover generation
const T = {
  elem1: 15,     // estimated start
  elem2: 210,    // estimated
  elem3: 448,    // estimated
} as const;

// Use in FadeIn
<FadeIn delayFrames={T.elem1}>
  <div>...</div>
</FadeIn>

Subtitles use estimateSubtitles() at this stage:

import { estimateSubtitles } from "../utils/subtitle";
const subtitles = estimateSubtitles(subtitleText, SCENE_DURATION);

This is a temporary scaffold — precise timing replaces it in Phase E.

Phase C: Verify

After each scene:

1. npx tsc --noEmit — must pass, no unused imports
2. python ~/.claude/skills/remotion-video-development/layout-lint.py . — 布局规范检查
3. npx remotion still --frame=30 — spot-check mid-scene frame
4. Check: elements appear when voiceover mentions them
5. git add -A && git commit -m "feat: scene N implemented"

Layout Lint

layout-lint.py 检查 src/scenes/*.tsx 是否遵守 layout-templates.md 中的硬性规范：

检查项	规则	级别
§0.1 内容区	1536×864 + margin auto	ERROR
§0.2 间距	gap/padding 只用 SP 变量	WARN
§0.3 标题区	非Hero场景必须有 height:80 + FS.title	WARN
§0.7 字号	最小 18px，正文≥24px	ERROR
截图	禁止 objectFit:contain	WARN

ERROR 必须修复。WARN 建议修复。

Lint 脚本路径: ~/.claude/skills/remotion-video-development/layout-lint.py 完整规范文件: ~/.claude/skills/remotion-video-design/layout-templates.md

Phase D: Voiceover Generation

After ALL scenes are coded and verified, generate TTS audio.

Why now, not earlier:

• Scene text may change during mockup/animation review
• Layout decisions are final — no risk of wasting TTS credits on changed text
• Audio generation is the slowest step (API calls per sentence)

D.0 Pre-Generation Lint

Before calling TTS API, run these checks on voiceover-text.json. Failures must be fixed before generation:

Check	Rule	Why
Subtitle length	每个逗号分隔的子句 ≤ 30 字	字幕在视频中过长会无法阅读
Double period	每句话结尾必须是 。。	单句号会导致 TTS 吞掉最后一词
Breath marks	无标点的连续复杂短语用 - 分隔	TTS 连读导致断句错误
Segment gaps	相邻句之间无重叠	防止前一句没播完就开始下一句

Lint script (run before generate-voiceover.py):

python ~/.claude/skills/remotion-tools/lint-voiceover-text.py

Output: per-scene pass/fail with specific line numbers and violations. If any ERROR is reported, fix voiceover-text.json before generating audio.

D.1 Generate Audio

python ~/.claude/skills/remotion-tools/generate-voiceover.py

Output: public/voiceover/scene{N}_seg{K}.mp3 + scene{N}_seg{K}_subtitle.json + scene{N}_segments.json

CLI flags: --scene scene3, --segment 1, --force

If Minimax balance is low:

1. python ~/.claude/skills/remotion-tools/prepare-minimax-text.py --scene sceneN
2. User generates at https://www.minimaxi.com/audio/text-to-speech
3. Re-run generate-voiceover.py to build segments.json

Phase E: Replace Precise Timeline

Replace estimated subtitles and durations with actual TTS data:

E.1 Update theme.ts durations — read segments.json → total_frames + 45f buffer:

// theme.ts SCENE_DURATIONS updated from actual audio
export const SCENE_DURATIONS = {
  scene1: 290,   // 245f audio + 45f buffer
  // ...
};

E.2 Replace estimated subtitles with precise TTS timestamps:

For each scene file, replace estimateSubtitles() with loadPreciseSubtitles():

// OLD (estimated):
import { estimateSubtitles } from "../utils/subtitle";
const subtitles = estimateSubtitles(subtitleText, SCENE_DURATION);

// NEW (precise from TTS subtitle JSON):
import { loadPreciseSubtitles } from "../utils/subtitle-precise";
import segmentsMeta from "../../public/voiceover/sceneN_segments.json";
import seg0Sub from "../../public/voiceover/sceneN_seg0_subtitle.json";
import seg1Sub from "../../public/voiceover/sceneN_seg1_subtitle.json";
// ... import all segment subtitle files

const subtitleData: Record<string, { time_begin: number; time_end: number; text: string }[]> = {};
// map each subtitle file by its key
subtitleData["sceneN_seg0_subtitle"] = seg0Sub as any;
subtitleData["sceneN_seg1_subtitle"] = seg1Sub as any;

const subtitles = loadPreciseSubtitles(segmentsMeta, subtitleData);

E.3 Align timeline for reference:

python ~/.claude/skills/remotion-tools/align-timeline.py

Output: docs/timeline-auto.md with frame positions from actual subtitle data.

E.4 Verify — npx tsc --noEmit must pass after all replacements.

Phase F: Add Audio Playback

Each scene loads segments.json and renders multi-segment audio:

import { Audio, staticFile } from "remotion";
import segmentsMeta from "../../public/voiceover/sceneN_segments.json";

// Inside AbsoluteFill, after Subtitles:
{segmentsMeta.segments.map((seg: any, i: number) => (
  <Sequence key={i} from={seg.offset_frames} durationInFrames={seg.frames}>
  <Audio
    src={staticFile(`voiceover/${seg.audio_file}`)}
    volume={0.9}
  />
  </Sequence>
))}

<Audio> with <Sequence> wrapping handles multi-segment playback correctly.

Git checkpoint: git add -A && git commit -m "feat: voiceover + precise timeline + audio playback"

Composition

After all scenes + audio integration (Phase D-F), wire up TransitionSeries:

// TransitionSeries with 15-frame fade between scenes
<TransitionSeries>
  <TransitionSeries.Sequence durationInFrames={SCENE1_DURATION}>
    <Scene1 />
  </TransitionSeries.Sequence>
  <TransitionSeries.Transition presentation={fade()} timing={linearDuration(15)} />
  // ... repeat for each scene
</TransitionSeries>

FULL_DURATION = sum of all scene durations - (N-1) * transition_frames

Voiceover Integration

This section describes Phase D-F in detail.

Why "Code First, Audio Later"

The recommended workflow is:

1. Code all scenes with estimated subtitles (estimateSubtitles())
2. Generate audio after text is stable
3. Replace estimated subtitles with precise TTS timestamps
4. Add Audio components

This is better than "generate audio first" because:

• Scene text often changes during mockup review — no wasted TTS credits
• Layout iteration is faster without waiting for audio
• Audio generation is the slowest step — parallelize it at the end

Pronunciation Pre-check

Before generating voiceover, run the pronunciation pre-check (full process in remotion-video-design Step 1b):

1. Read ~/.claude/tts-rules/tts-replacements.json for existing rules
2. Scan voiceover-text.json text for TTS-prone words NOT already in rules (English brands, mixed en/numbers, hyphenated compounds, all-caps abbreviations)
3. If new risky words found, present suggestions to user and update JSON before generating

TTS Generation (Segment-based)

Use ~/.claude/skills/remotion-tools/generate-voiceover.py:

Text preprocessing (automatic):

1. voiceover-text.json 中的 。。 在发送给 TTS 前会被替换为单个 。，避免 TTS 引擎读出两个句号
2. "-" 分隔符保留原样发送给 TTS，产生自然微停顿
3. 每个句子发送前自动追加 。。 确保最后一词不被吞（如果原文没有双句号结尾）

Audio timeline gap (automatic):

• segments.json 中每个 segment 的 offset_frames 会自动累加，segment 之间没有额外间隔
• theme.ts 的 SCENE_DURATIONS 使用 total_frames + 45 缓冲，确保场景末尾有 1.5s 静音
• 45f 缓冲 = 30f 尾部静音 + 15f 过渡 = 相邻场景间至少 1s 无音频重叠

Output per sentence:

• scene{N}_seg{K}.mp3 — audio file
• scene{N}_seg{K}_subtitle.json — Minimax timestamps
• scene{N}_segments.json — cumulative metadata (frames, offsets, duration)

CLI flags:

• --scene scene3 — only generate a specific scene
• --segment 1 — only regenerate one sentence (requires --scene)
• --force — regenerate even if exists (archives old as _v1, _v2)

Incremental: skips existing segments unless --force. Old files archived, never deleted.

If Minimax API returns insufficient balance:

1. Run python ~/.claude/skills/remotion-tools/prepare-minimax-text.py — outputs pronunciation-corrected text per scene
2. User manually generates at https://www.minimaxi.com/audio/text-to-speech
3. Download MP3 to public/voiceover/scene{N}_seg{K}.mp3
4. Note: Web manual generation does NOT produce subtitle files — use character-ratio for timeline
5. After manual generation, run generate-voiceover.py again to build segments.json from the files

Timeline Alignment

After voiceover generation, run python ~/.claude/skills/remotion-tools/align-timeline.py to:

1. Read *_segments.json to auto-discover scenes
2. Read each segment's subtitle JSON, accumulate frame offsets
3. Output global frame positions for all subtitle parts
4. Generate docs/timeline-auto.md with suggested T constants

This is the same as Phase E.3 — run it for reference/debugging.

Pronunciation Fix Loop

If voiceover is generated but pronunciation is wrong on playback:

1. Identify the mispronounced word and which scene it's in
2. Add rule to ~/.claude/tts-rules/tts-replacements.json
   • Rules sorted by key length (longest first), so compound rules like "Kimi K2.6" take priority over individual "Kimi" and "K2.6"
   • Use enumeration comma ， to create a short TTS pause between connected words (e.g., "个子代理" → "个、子代理")
3. Regenerate the scene:

   python ~/.claude/skills/remotion-tools/generate-voiceover.py --scene sceneN --force
   

   Note: --scene accepts only ONE scene at a time. For multiple scenes, run separately.

CRITICAL — After regeneration, complete this atomic sequence:

4. Verify generation succeeded:

   ls public/voiceover/sceneN_seg*.mp3 | grep -v _v
   

   If any segment is missing (API rate limit, etc.), restore from archived backup:

   cd public/voiceover
   # Restore failed segments from latest archive (_v1, _v2, etc.)
   for f in sceneN_seg*_v*.mp3; do
     base=$(echo "$f" | sed 's/_v[0-9]*\.mp3$/.mp3/')
     [ ! -f "$base" ] && mv "$f" "$base"
   done
   

5. Duration auto-updated: generate-voiceover.py automatically updates theme.ts SCENE_DURATIONS. If it doesn't (e.g., key not found), manually update from segments.json total_frames + 45f buffer.

6. Verify audio files exist:

   ls public/voiceover/sceneN_seg*.mp3 | grep -v _v | wc -l
   

   Segment count must match segments.json. No ffmpeg merge needed — use multi-segment <Audio> in scene code (see Voiceover Integration section).

Failure recovery: If --force archives original files to _v1/_v2 and API fails, the original files are preserved in the archive. Always check segment count matches expected before proceeding.

API sample_rate: The Minimax API rejects sample_rate: 48000. Valid values: 32000, 44100. The generate-voiceover.py script has this configured — update if API changes.

TTS pronunciation fixes: Voiceover text may differ from display text. Keep display text accurate, fix only pronunciation.

Multiple scenes: If regenerating multiple scenes, process one at a time to reduce blast radius from API failures. Verify each scene before moving to the next.

Post-Render Audio Verification

After npx remotion render, run this check to catch silent/missing audio:

VO="public/voiceover"
for s in scene1 scene2 scene3 scene4 scene5 scene6 scene7 scene8 scene9 scene10 scene11 scene12; do
  dur=$(ffprobe -v quiet -show_entries format=duration -of csv=p=0 "$VO/${s}.mp3" 2>/dev/null)
  if [ -z "$dur" ] || [ "$(echo "$dur < 1" | bc 2>/dev/null)" = "1" ]; then
    echo "❌ $s: MISSING or <1s"
  else
    echo "✅ $s: ${dur}s"
  fi
done

All 12 scenes must return ✅ with duration > 1s before considering the render complete.

Deliverables

Output	Files
Scenes	src/scenes/Scene{N}.tsx
Shared components	src/components/{Name}.tsx
Shared animation utils	src/utils/animation.ts (ease, fadeIn)
Composition	src/Composition.tsx (or Root.tsx)
Updated theme	src/styles/theme.ts (if durations changed)

Error Prevention

Check	When	Command
TypeScript	After each scene	npx tsc --noEmit
Layout lint	After each scene	python ~/.claude/skills/remotion-video-development/layout-lint.py .
Unused imports	After layout changes	Check manually or tsc warns
Image filenames	Before first render	ls public/images/ — all ASCII
Segments meta	After voiceover gen	ls public/voiceover/*_segments.json
Timeline match	After animations	python ~/.claude/skills/remotion-tools/align-timeline.py then compare
Audio sync	Full preview	npx remotion studio

Common Mistakes

Mistake	Fix
objectFit: cover on screenshot	Use height: auto with no fixed container height, or overflow: hidden
objectFit: contain in fixed-height container	Remove fixed height, use height: auto to let image determine container size
flex: 1 on screenshot container	Remove flex:1, let image naturally size the container
FadeIn wrapping flex container	Move FadeIn INSIDE the container
Fixed height on auto-content element	Remove height, let content determine
bottom: N pushing content off-screen	Use top: N or flex flow instead
Chinese filename in staticFile()	Rename file to ASCII
All scenes using same card-grid layout	Redesign each scene based on content type
Content touching viewport edges	Wrap in 80% area container (10% margin each side)
Audio overlap between scenes	See "Audio Overlap Prevention" below

Audio Overlap Prevention

<Sequence premountFor={N}> 会让 Audio 组件在场景可见之前 N 帧就开始播放。当 TransitionSeries 的过渡时长叠加时，相邻场景的音频会重叠。

根本原因：

• premountFor={30} → 音频提前 30 帧（1s @ 30fps）播放
• TransitionSeries.Transition fade 过渡占用 15 帧
• 两者叠加 = 相邻场景音频重叠 1.5s

修复步骤：

1. 移除所有场景的 premountFor

   // 删除
   <Sequence premountFor={30}>
     <Audio ... />
   </Sequence>
   
   // 改为
   <Audio ... />
   

2. 在 theme.ts 中为每个场景添加 45f 缓冲

   // 场景时长 = 实际音频时长 + 45f 静音缓冲
   // 45f 缓冲确保：音频结束后 30f 静音 + 15f 过渡 = 场景间至少 1s 无音频重叠
   export const SCENE_DURATIONS = {
     scene1: 212,  // 167 + 45
     // ...
   } as const;
   

3. 验证：在 npx remotion studio 中播放，确认场景切换时没有两段音频同时播放。

Template & Reuse

During implementation, identify components marked as "reusable" in the design doc:

• Extract shared components to src/components/ (e.g., InfoCard.tsx, TitleCard.tsx, MacWindow.tsx)
• Use props for data-driven content (title, subtitle, color, timing)
• Keep animation logic inside the component, controlled by delayFrames prop

Batch Rendering

For template-based videos (same layout, different data), use parameterized rendering:

// Composition with props
<Composition
  id="MyVideo"
  component={MyVideo}
  durationInFrames={300}
  fps={30}
  width={1920}
  height={1080}
  defaultProps={{ title: "Default" }}
/>

# Batch render from dataset
npx remotion render MyVideo --props='{"title":"Variant A"}' out/a.mp4

See remotion-best-practices rules/compositions.md for dataset rendering patterns.

Hybrid Workflow

If Remotion handles only part of the video (e.g., title cards + data viz, but demo parts use screen recording):

1. Render each Remotion segment as separate MP4
2. Record demo parts with OBS
3. Concatenate with FFmpeg:

   # filelist.txt: file 'remotion_intro.mp4'\nfile 'obs_demo.mp4'\nfile 'remotion_outro.mp4'
   ffmpeg -f concat -safe 0 -i filelist.txt -c copy output.mp4
   

Completion

After all scenes implemented and verified:

1. Run npx remotion studio for full preview
2. Compare each scene against timeline.md
3. Dispatch implementation reviewer using implementation-reviewer-prompt.md in this skill directory. Checks timeline alignment, layout correctness, composition math, asset references, component consistency. Only flags issues that would produce wrong video output.
4. Fix any issues found by reviewer
5. git add -A && git commit -m "feat: all scenes implemented + composition wired up"

Transition

→ Invoke remotion-video-review for visual refinement.

If this is a template-based video (multiple variants from same design): → Use renderMedia() with parameterized data to batch render all variants.

If review reveals fundamental design issues (wrong mood, missing scenes): → Go back to remotion-video-design Phase 0 to reassess creative direction.