talking-head-video

name

talking-head-video

description

Turn a raw 口播 / talking-head selfie video plus its script into a finished explainer, vertical 9:16 or landscape 横屏 16:9 — synced burned-in subtitles, animated knowledge graphics in three selectable styles (darkcard 深色科技卡 default, watercolor 纸上水彩, inkline 白纸简笔; both style and orientation are confirmed with the user first), and a circular picture-in-picture avatar that shrinks the speaker to the top-left while a graphic or B-roll fills the screen during explanations. THE pipeline whenever someone has a clip of a person talking to camera and wants it "剪成成片", subtitles auto-added from a 逐字稿/脚本, the 画面 to "切成图/弹图表/做大数字特效" on key points while the "人脸/人头缩小、放到角落、变成圆形头像/画中画/小圈圈", or Remotion/ffmpeg 动态讲解图 cut into a 口播/讲解/知识科普短视频 for 视频号/抖音/B站/Shorts. Also covers standalone 知识卡/概念卡/章节卡 in any of the three styles (水彩/简笔画/ 深色卡) without a source video. Partial asks count: "帮我剪口播", "自动加字幕配动态图", "源视频和脚本都有帮我合成成片". NOT for: writing a 口播脚本, 数字人/AI 主播 reading a script, multi-cam cutting, PPT→video, video compression, audio→text transcription, or subtitling an unrelated movie clip.

Talking-Head Explainer Video

Build a finished 口播 video (vertical 9:16 by default, landscape 16:9 fully supported — see 画幅 section) where the speaker is full-frame by default, and during each explanation beat the speaker shrinks into a circular avatar in the top-left while an animated knowledge graphic (chart / data cards / comparison / flow / big-number) fills the screen. Subtitles are burned in and corrected against the user's verbatim script.

This skill captures a real working pipeline. Read this whole file first, then the references as needed. The design conventions exist for concrete reasons — keep them unless the user asks otherwise.

The core idea (why it's built this way)

The original talking-head video is the spine — its audio plays untouched the whole time, so sync is never an issue. You only overlay graphics + a shrunk avatar during "knowledge windows"; everywhere else it's just the full face. The graphics render as ProRes 4444 (alpha) with a short fade at their edges, so they crossfade onto the face instead of cutting to black.

full face ──┐         ┌── graphic full-frame + avatar top-left ──┐         ┌── full face
            │  cross  │   (one "knowledge window")               │  cross  │
  subtitles burned over EVERYTHING, the whole time

Prerequisites (check / install once)

• ffmpeg with libass + prores_ks (Homebrew ffmpeg has these)
• faster-whisper: python3 -m pip install faster-whisper
• Node + Remotion: the template in assets/remotion/ — npm install inside it. Remotion renders via a headless Chrome it manages.
• A CJK font libass can find (macOS: PingFang SC works out of the box).

Workflow

Set up a working dir (e.g. ~/Downloads/<project>_build/). Copy assets/remotion/ into it as remotion/. Then:

1. Inspect the source

ffprobe the clip for resolution/duration/fps. Vertical 1080×1920 @ 30fps and landscape 1920×1080 @ 30fps are both first-class (see 画幅 section for the parameter set of each). 开工前先和用户确认目标画幅(通常跟源视频走, 但要问一句,别默默定;用户已明说的不再问). Extract a frame (ffmpeg -ss 2 -i src.mp4 -frames:v 1 f.jpg) and look at where the face sits — you need this for the avatar crop.

开工前先问用户(硬规则)

制作视频前,必须和用户确认两个选项,给推荐但由用户拍板;用户已明说的不重复问:

1. 画幅:横屏 16:9 还是竖屏 9:16(参数表见下节);
2. 图卡风格:darkcard / watercolor / inkline(见「图卡风格」节)。

图卡风格 · Graphics styles(三选一)

风格	一句话	适合	字体(fonts.ts 现成栈)
darkcard 深色科技卡(默认)	深底、辉光、边框卡片、大数字	价格/参数/对比/新闻/评测	得意黑(DARK_TITLE)+ DIN(DARK_NUM)+ PingFang
watercolor 纸上水彩	米黄纸、水彩物件、铅字标题、物理点睛	生活类比/故事感/章节卡	汇文明朝体(WC_CN)+ Special Elite(WC_EN)
inkline 白纸简笔	纯白、细线"画出来"、大字+灰注、诗意收尾	原理/定义/一句话洞察	霞鹜文楷双字重(INK)

工艺规范(色板/滤镜/动效语法/选择速查)在 references/styles.md,写组件前先读。darkcard 的六个 archetype 在 references/design-system.md + assets/remotion/src/;watercolor/inkline 的基础件在 src/watercolor.tsx / src/inkline.tsx;三风格各有完整示例在 src/examples/ (同一概念「边际成本」三种表达,当模板抄结构)。字体接线:bash scripts/setup_fonts.sh <remotion项目> (自带 6 个免费字体;Anthropic 官方字体从本机 Claude.app 自动检测,禁止随仓库/产物分发)。 通用红线:文字永不加做旧/毛边滤镜,滤镜只作用于图形线条。

画幅 · Orientation parameter sets (both proven on real videos)

参数	竖屏 9:16 (1080×1920)	横屏 16:9 (1920×1080)
Remotion Composition	width 1080, height 1920	width 1920, height 1080
头像直径 / overlay 位置	300px / 40:120	380px / 44:44
make_avatar.sh 参数	CROP≈1040:1040:0:250 SIZE=300	CROP≈780:780:580:20 SIZE=380(居中脸源)
字幕 ASS PlayRes / MarginV	1080×1920 / 210	1920×1080 / 90(generate_subs.py 传 --land)
图形安全区	避开左上头像 + y≳1540 字幕带	避开左上头像(x<470, y<470)+ 底部字幕带
引用他人视频的 @标签	右上角	右上角(躲开左上头像)

横屏的典型场景:B站/YouTube 知识科普、新闻解析、"口播 + 整屏 B-roll/录屏"结构 (讲解时 B-roll 全屏盖脸、口播音频连续、头像浮左上)。竖屏参数是原始默认, 横屏参数来自金山岭项目实测。

2. Transcribe → timing

ffmpeg -i source.mp4 -ar 16000 -ac 1 -c:a pcm_s16le audio16k.wav
python3 scripts/transcribe.py audio16k.wav asr.json

asr.json gives word/segment timings. Trust the timing, not the text.

3. Build the segment map (the one real judgement step)

Read the user's script + asr.json and decide which time windows are knowledge windows (get a graphic + avatar) vs full-face (rhetorical beats, the hook, the conclusion — let the face carry emotion). A good rhythm alternates: don't make every second a graphic. Each knowledge window maps to one Remotion composition. Capture this as segments.json (see scripts/segments.example.json).

Pick a graphic archetype per window (see references/design-system.md → "Graphic archetypes"). Map the window's narration to the archetype's slots.

4. Build / adapt the graphics (Remotion)

In remotion/src/, each composition is one knowledge graphic. Copy the closest archetype component (RecruitChart / MedicalCards / WetDry / PromptToDrug / Edison / ThesisKeys) and rewrite its content. All of them import the shared theme.tsx — use its tokens, Backdrop, Kicker, Title, useEnter, fadeUp, useClipFade, and respect AVATAR_SAFE (keep the top-left clear). Register each in Root.tsx with durationInFrames = round(windowSeconds * 30).

Iterate fast with stills before full renders: npx remotion still src/index.ts <Comp> out.png --frame=<n>

Critical layout rule: keep all graphic content above y≈1540. The bottom ~360px is reserved for subtitles. Put payoff callouts at bottom: 380–430, not bottom: 150. (This was the #1 bug on the first build.)

5. Render graphics to alpha clips

bash scripts/render_graphics.sh remotion clips <Comp1> <Comp2> ...

Renders ProRes 4444 .mov (alpha) into clips/. Alpha is required for the edge crossfade.

6. Make the circular avatar

bash scripts/make_avatar.sh source.mp4 avatar.mov 1040:1040:0:250 0x22D3A8

Tune the crop so the whole head incl. chin sits in the circle — wider crop = smaller face. Validate on one frame first (the script prints where it wrote). Common mistake: too tight a crop clips the chin.

7. Subtitles (corrected)

python3 scripts/generate_subs.py asr.json subs.ass

Edit CORRECTIONS in the script per video — whisper mis-hears brand names / English terms / 同音字 (e.g. 施实验→湿实验, 礼来, Indeed, Verve, 胰腺癌, PPT). Skim asr.json and add the wrong→right pairs before running.

8. Composite → final

Write segments.json (source, avatar, subtitles, output, and the windows), then:

python3 scripts/build_composite.py segments.json

It generates the ffmpeg filtergraph and renders final.mp4 (graphics overlaid in their windows + avatar top-left during windows + subtitles burned + original audio). Use --dry-run to inspect the filter first.

9. QA (always)

Extract a frame from the final mp4 at the middle of every segment type and look at each:

ffmpeg -ss <t> -i final.mp4 -frames:v 1 qa_<t>.jpg

Check, per frame: avatar shows the whole face and sits in the clear top-left zone; subtitle doesn't collide with graphic content; graphic content isn't cut off; no black flash at window boundaries (mid-crossfade dark is fine). Fix, re-render only the affected clip(s), re-composite.

Editorial conventions worth keeping

• Don't make it all graphics. The face carries the hook and the emotional conclusion. Graphics are for facts/data/structure.
• One subtitle correction pass is mandatory — uncorrected ASR (wrong brand names, 同音字) instantly reads as sloppy.
• Watch the narrative framing in graphics. If the speaker's claim is a slight oversimplification you can't change (it's recorded audio), make the graphic text accurate so it doesn't contradict knowledgeable viewers. Likewise, if a segment risks an unintended read (e.g. "AI replaces humans"), add a bridging line in the graphic that steers toward the speaker's actual point.

Reusable assets in this skill

• assets/remotion/ — full Remotion project: theme.tsx (design system) + 6 archetype components + Root.tsx. Copy into the project and adapt.
• scripts/transcribe.py — faster-whisper word timings
• scripts/generate_subs.py — ASR → corrected styled .ass
• scripts/make_avatar.sh — circular alpha PiP avatar (parameterized crop/ring)
• scripts/render_graphics.sh — batch ProRes-4444 render
• scripts/build_composite.py — segments.json → ffmpeg master composite
• scripts/segments.example.json — the segment-map schema
• references/design-system.md — palette, tokens, avatar/subtitle geometry, and the graphic archetype catalog. Read before designing graphics.