creator-pip-overlay-video

name

creator-pip-overlay-video

description

Produce a 9:16 list-style explainer video where a single fixed-frame AI creator (selfie-cam aesthetic) narrates N numbered items, with each item illustrated by a picture-in-picture sample clip in the top-right corner + a persistent top-center title pill + a per-beat label + a view-count badge inside the PiP. Composes gpt-image-2 (anchor portrait) → ElevenLabs eleven_v3 VO → Seedance 2.0 reference-to-video (creator body motion + gestures) → Veed Fabric 1.0 lipsync (phoneme-accurate mouth) → PIL overlay rendering → ffmpeg per-beat composite with rounded-corner PiP + audio mix + ElevenLabs SFX pop → burn-in-captions (word-flash). Use for "N tools / N formats / N tips" listicles where the creator is showing-and-telling pre-existing example clips. Validated end-to-end on `creator-content/test-videos/01-seven-content-formats/finals/master-final-v3.mp4` (9-beat, 44s, ~$30).

creator-pip-overlay-video

Purpose

The production line for the "creator selfie + PiP sample clips" listicle format — a 9:16 short where a creator faces camera and narrates N numbered items, each accompanied by a picture-in-picture sample clip that swaps in the top-right corner. The creator's body stays mostly fixed; only their head moves (lip sync) and the pointing hand can sustain a gesture for the whole beat. The structure is:

• Hook (no PiP) — title pill appears, creator hooks ("Seven content formats…")
• Beats 1..N — title pill stays, beat label swaps in lower-center, PiP swaps in top-right with view-count badge + white hairline border + rounded corners. Audio: creator VO + PiP-clip audio (ducked to ~0.18) + a pop SFX on swap.
• CTA (no PiP) — title pill clears, creator addresses camera directly.

Use when:

• The user wants "N tools", "N AI workflows", "N content formats", "N apps you should install", "5 tips for X", etc.
• They want an AI creator (no real footage) — i.e. they don't have a live actor available.
• They want to show actual example clips of the things being talked about — PiP is the affordance.

Do NOT use for:

• Live UGC selfie-cam where a real person is on camera — use molecules/ugc-ad/ instead.
• Talking-head podcast clips with a guest — use molecules/podcast-clip-animated-ad or molecules/podcast-repurposing/.
• Lyric-driven music ads — use molecules/music-video-ad/.
• Formats where the visual is screen-recording B-roll rather than a PiP corner — use the (proposed) creator-with-screenroll-list molecule.

Inputs

Required:

• <concept> — paragraph describing what the creator is talking about (the niche of the listicle).

• <beats> — list of N items (typically 5-8 + hook + CTA). Each beat needs:

  • name — short slug (e.g. 01-podcast, 02-broll).
  • vo_text — one or two declarative sentences (~3-5s each beat).
  • label — the on-screen label string for this beat (e.g. "1. Podcast Format").
  • pip_source — local path to a 9:16 (or center-croppable) mp4 to show in PiP. Should exemplify the archetype this beat is about. Use real clips, not AI-generated. If null, the beat shows no PiP (hook + CTA).
  • view_count — string for the badge inside the PiP (e.g. "5.1M", "510k"). Optional; defaults to no badge.
  • gesture_hint — short verb phrase for the Seedance prompt's gesture (e.g. "points up-and-to-the-right at the PiP and holds the point"). Recommended for every beat that has a PiP so the creator gestures TOWARD the PiP.

• <title_pill_text> — the persistent top-center pill text (e.g. "7 content formats to build your brand").

• <voice_id> — ElevenLabs voice ID from VOICES.md. Default for this format: Brad (T4x5CtnhOiichhcqFzgg) — energetic male, creator-tonality.

• <character_descriptor> — verbatim block describing the AI creator's appearance, wardrobe, and home setting. Will be passed to gpt-image-2 as the anchor prompt. Default template in references/character-descriptor.template.md.

• <output_dir> — destination project folder. Follows canonical 5-folder layout (source/, assets/, working/, finals/, archive/).

Optional:

• <atempo> — VO speed factor. Default 1.35 (compresses eleven_v3's natural read by ~26% to hit creator pace). Range: 1.20–1.45. Above 1.45 starts to artifact.
• <caption_style> — burn-in-captions preset. Default word-flash (one big white word, accent #FFD400). Alternatives: frosted-subtle, bold-accent, clean-bubble.
• <emphasis_words> — comma-separated punch words to force-highlight in captions (e.g. "Podcast,B-roll,Clipper,format"). Otherwise the heuristic picks longest non-stopword per cue.
• <music_bed> — path to a low-mix instrumental. Default none (matches reference). If provided, mix at -22 dBFS under the VO.
• <reuse_anchor> — path to a previously-locked anchor portrait. When set, skip gpt-image-2 ($0.19 saved).
• <auto> — bool, default false. When true, advance through human gates with operator's prior preferences.

Composed Atoms

Voiceover:

• atoms/voiceover/create-voiceover-elevenlabs — eleven_v3 with with-timestamps for char-level alignment.

Anchor portrait:

• atoms/image-generation/create-image-gpt-image-fal — --model gpt-image-2 --quality high --aspect-ratio 9:16. Strong photorealism for creator selfie aesthetic. Strictly preferred over Nano Banana 2 for human faces in domestic settings.

Creator body + gesture clips (preserves the pointing pose):

• atoms/video-generation/create-video-seedance-fal — 720p standard, reference-to-video. Anchor PNG as @Image1. Per-beat VO WAV as --ref-audio (a pacing hint that yields natural micro-movements + gestures). One clip per beat.

Phoneme-accurate lip sync (the v3 breakthrough):

• atoms/lipsync/create-lipsync-veed-fal — Veed Fabric 1.0 @ 720p. Takes a still image (pointing-pose frame extracted from the Seedance clip) + audio segment → returns a clip with the body essentially frozen and the face/mouth animated to phoneme-level sync.

SFX:

• ElevenLabs sound-generation HTTP endpoint (/v1/sound-generation) — one short pop SFX, ~$0.05. Same call shape used by talking-goose-creator-video.

Overlays:

• PIL renders (no atom — done inline in the molecule's scripts/render_overlays.py):
  • Title pill (white rounded rect with soft shadow, dark sans-serif text, centered top).
  • Per-beat label PNGs (bold sans-serif, ~72px, white with subtle drop shadow, optional 2-line wrap).
  • View-count badges (dark translucent pill with eye glyph + count, ~28px text).

Per-beat composite:

• Direct ffmpeg filter_complex graph (done inline in the molecule's scripts/composite.py — copied from project 01). Key tricks:
  • Rounded-corner PiP via a geq alpha mask (no external matte needed).
  • drawbox for the white hairline border.
  • amix of VO + PiP audio + SFX pop with alimiter for safety.

Stitch:

• atoms/assembly/stitch-videos-ffmpeg (or inline ffmpeg -f concat -c copy).

Captions (FINAL post step per CLAUDE.md):

• atoms/captions/burn-in-captions — invoked with --style word-flash by default. Word-level Whisper via fal.ai. ASS template, libass burn.

Workflow

S0 — Intake

1. Validate inputs: <beats> has 1 hook + N items + 1 CTA; every PiP-bearing beat has a pip_source that exists on disk; voice_id is in VOICES.md.
2. Validate FAL_API_KEY, ELEVENLABS_API_KEY, KLAP_API_KEY in .env.
3. Run preflight: no LFS pointer files in <pip_source> paths (a silent stub of < 1KB will silently break ffmpeg). Use ffprobe to verify each <pip_source> decodes.
4. Create the canonical project folder: source/pip-candidates/, assets/{characters,overlays,sfx}/, working/{audio,clips,clips-lipsynced,lipsync-frames,segments,scripts}/, finals/, archive/.

S1 — Brainstorm + design-brief

If <concept> is not yet decomposed into <beats>, decompose it first. Hook (~3s), N items (~4-5s each), CTA (~3s). Total target = 30-45s.

Write design-brief.md with:

• audio_strategy: vo-narrator
• climax_line: the CTA line
• assets_manifest: paths listed in "Files of record" below.
• The character descriptor block + voice ID.

S2.5 — Lock script

Compose the script as one continuous take with light eleven_v3 tags:

[curious] <hook line>. [deliberate] This is the <Beat 1 Name> — <one declarative sentence>. This is the <Beat 2 Name> — <…>. ... [emphasized] <CTA line>.

Write script/script-locked.md. Human gate.

S3 — Storyboard

Render one HTML storyboard frame showing the canonical layout: creator selfie + title pill + sample PiP + beat label + badge. Confirm the look before any AI credits are spent. Use references/storyboard.html.template.

S3.5 — Character lock

If <reuse_anchor> is set, copy it to assets/characters/anchor.png and skip. Otherwise:

python3 skills/atoms/image-generation/create-image-gpt-image-fal/scripts/generate.py \
  --prompt "<character_descriptor>" \
  --output assets/characters/anchor.png \
  --model gpt-image-2 \
  --aspect-ratio 9:16 \
  --quality high

Cost: $0.19. Inspect the result. Re-roll if the framing is wrong (too tight = PiP collision; too wide = face too small). HUMAN GATE.

S5a — Voiceover

Use the ElevenLabs HTTP endpoint directly (not the voiceover atom's wrapper) so we get the raw with-timestamps JSON for slicing later:

curl -sS -X POST "https://api.elevenlabs.io/v1/text-to-speech/$VOICE_ID/with-timestamps?output_format=mp3_44100_128" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"...","model_id":"eleven_v3","voice_settings":{"stability":0.40,"similarity_boost":0.75,"style":0.05,"use_speaker_boost":true}}' \
  > working/audio/vo-raw.json

Decode the base64 audio to working/audio/vo-full.mp3 and save the alignment block to working/audio/vo-full-timestamps.json.

S5b — Compress to creator pace

ffmpeg -y -i working/audio/vo-full.mp3 -filter:a "atempo=1.35" -c:a libmp3lame -q:a 2 working/audio/vo-full-1.35x.mp3

atempo=1.35 is the validated sweet spot for Brad — slower starts to read corporate; faster starts to artifact.

S5c — Slice into beat segments

Run scripts/slice_vo.py <project_dir> (in this molecule). The script:

1. Loads the alignment JSON.
2. For each beat, finds the substring index of the beat's first sentence in the concatenated alignment text.
3. Computes raw start/end timestamps from character_start_times_seconds.
4. Divides by atempo (1.35) to get the timestamp on the compressed audio.
5. Writes 9 WAVs to working/segments/beat-NN-<name>.wav (mono, 16kHz, for Seedance + Veed Fabric).
6. PADS short segments to ≥2.0s (Seedance min-audio-duration). CTA is the usual offender.

S5d — Generate Seedance creator clips (gesture + body motion)

For each beat, call atoms/video-generation/create-video-seedance-fal:

python3 skills/atoms/video-generation/create-video-seedance-fal/scripts/generate.py \
  --prompt "@Image1 sits in his home corner, talks to camera, immediately raises right hand and clearly points up-and-to-the-right at an off-camera spot in the upper-right of the frame (where a picture-in-picture sits), holds the point throughout. natural micro-movements, slight smile. soft window light from camera-left. authentic creator iPhone selfie feel. vertical 9:16." \
  --ref-image assets/characters/anchor.png \
  --ref-audio working/segments/beat-NN-<name>.wav \
  --output working/clips/creator-beat-NN-<name>.mp4 \
  --aspect-ratio 9:16 --resolution 720p --duration <ceil(audio_dur)> --tier standard

Prompt rules (validated on this format):

• Always start with @Image1 to bind to the anchor.
• Always describe the pointing gesture in the same direction the PiP will be (upper-right). Use the gesture verbatim from <gesture_hint> or default to "raises right hand and clearly points up-and-to-the-right at an off-camera spot in the upper-right of the frame (where a picture-in-picture sits), holds the point throughout."
• Always include "authentic creator iPhone selfie feel. vertical 9:16." so the framing locks.
• Duration = ceil(audio_dur) clamped to [4, 8]. Seedance's sweet spot is 4-6s; >7s starts to drift.

Parallelize all N+2 jobs (use background tool calls). They each take 1-3 minutes. Cost: ~$1.20-2.10 per clip × ~9 = ~$11-19.

Even with --ref-audio, the lip sync at this stage is a pacing hint only — not phoneme-accurate. Don't ship these clips directly; chain Veed Fabric next.

S5e — Extract pointing-pose frames

For each Seedance clip, extract one still at t = duration × 0.55 (just past middle, where the gesture is most established):

ffmpeg -y -ss <0.55*dur> -i working/clips/creator-beat-NN-<name>.mp4 -frames:v 1 working/lipsync-frames/frame-NN-<name>.png

Inspect each frame. If the hand isn't clearly visible / pointing in any beat, try t = 0.45 or t = 0.65 instead.

S5f — Veed Fabric lipsync (phoneme-accurate mouth)

For each beat, call:

python3 skills/atoms/lipsync/create-lipsync-veed-fal/scripts/generate.py \
  --image working/lipsync-frames/frame-NN-<name>.png \
  --audio working/segments/beat-NN-<name>.wav \
  --output working/clips-lipsynced/lipsync-NN-<name>.mp4 \
  --resolution 720p

Veed Fabric 1.0 freezes the body and animates only the face/head/mouth. The pointing hand stays sustained for the whole beat (which is actually what we want — the creator points at the PiP throughout, matching the reference video grammar).

Parallelize. Each call takes 2-5 minutes. Cost: 9 × ~$0.50-0.85 = ~$5-7 at 720p.

S7a — Render overlay PNGs

Run scripts/render_overlays.py <project_dir>:

• Title pill (one PNG).
• Beat label PNGs (one per beat, auto-wraps long titles into 2 lines).
• View-count badges (one per beat, dark translucent pill + eye glyph + count text).

Use Arial Bold from /System/Library/Fonts/Supplemental/Arial Bold.ttf (macOS). On Linux fall back to DejaVu Sans Bold.

S7b — Generate pop SFX

Call ElevenLabs sound-generation once:

curl -sS -X POST "https://api.elevenlabs.io/v1/sound-generation" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
  -d '{"text":"short clean modern UI pop click, snappy positive feedback, single instance, no reverb, 0.3 seconds","duration_seconds":0.5,"prompt_influence":0.8}' \
  --output assets/sfx/pop.mp3

Cost: ~$0.05. Used as a single shared SFX across all PiP-swap beats.

S7c — Per-beat composite

Run scripts/composite.py <project_dir>. Per beat, the filter graph:

[creator]  scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920  →  [base]
[pip]      scale=W:H,crop=W:H,format=rgba,geq(rounded-corner alpha)              →  [pip]
[base][pip] overlay=PIP_X:PIP_Y                                                  →  [v_pip]
[v_pip]    drawbox(white 0.75 alpha, 3px) around PiP                             →  [v_pip_b]
[v_pip_b][title]  overlay top-center                                             →  [v_title]
[v_title][lbl]   overlay center-lower (y=0.74*H)                                 →  [v_label]
[v_label][badge] overlay inside PiP bottom-left                                  →  [v_final]

Audio: amix=[a_vo][a_pip*0.18][a_sfx*0.45], then alimiter=limit=0.92.

Geometry constants (validated against reference, do not change without testing):

• Output: 1080×1920 9:16
• PiP: 42% width × 32% height, top-right with 28px margin, y=170
• Title pill: top-center, y=60
• Beat label: y=0.74 × H = 1421
• Badge: bottom-left of PiP, with offsets (18, 14) from PiP bottom-left corner
• White PiP border: 3px, alpha 0.75

S7d — Stitch + concat

ffmpeg -y -f concat -safe 0 -i working/concat.txt -c copy finals/master-stitch.mp4

S7e — Burn captions (final post step)

python3 skills/atoms/captions/burn-in-captions/scripts/caption.py \
  --video finals/master-stitch.mp4 \
  --style word-flash \
  --emphasis "<emphasis_words>"
cp finals/master-stitch.captions/master-stitch__word-flash__ass.mp4 finals/master-final.mp4

Per CLAUDE.md hard rule: captions are burned LAST, after all other compositing.

S8 — Review

Run /watch:watch finals/master-final.mp4 and verify:

1. Transcript matches the locked script exactly.
2. Layout reads at all beat boundaries (PiP visible, label visible, no overlap).
3. Lip sync is on-time within ~80ms.

S10 — Deliver

finals/master-final.mp4 is the shippable. Optional platform variants via atoms/export-delivery/ molecules.

Decision Rules

Question	Default	When to override
Number of beats	5-7 + hook + CTA	<5 reads too sparse; >8 the VO can't fit cleanly in 45s
Atempo factor	1.35	Bump to 1.40-1.45 if Brad reads slow at 1.35 for your script; drop to 1.25 for an older voice that artifacts faster
Seedance resolution	720p standard	Drop to fast tier (-20%) if budget is tight; the Veed Fabric pass renders the source moot anyway
Seedance audio ref	always include matching VO WAV	Skip only if you want generic body motion; with the WAV the gesture pattern locks to the VO rhythm
Veed Fabric resolution	720p	480p saves ~50% but face detail noticeably softer
PiP source	real existing mp4s	Don't AI-gen these — the whole point is "look at what other creators already made"
PiP audio gain	0.18	Drop to 0.10 if PiP voice bleeds; raise to 0.25 for stronger texture
Caption style	word-flash	Use frosted-subtle for a more accessible-subtitle look; bold-accent for higher punch
Emphasis words	heuristic	Override when the heuristic picks filler ("will", "to", "format.")
Music bed	none	Add when the VO leaves long silences that need texture
CTA frame duration	match VO segment (~1.5-3s)	Pad to 3.5s by holding the last frame if the CTA feels rushed
Anchor portrait re-use	re-gen per project	Re-use across a series — same creator across 5 ads = strong brand

Output

A populated <output_dir>/ with:

<output_dir>/
├── design-brief.md
├── script/script-locked.md
├── assets/
│   ├── characters/anchor.png
│   ├── overlays/{title-pill, label-NN-*, badge-NN}.png
│   └── sfx/pop.mp3
├── source/pip-candidates/pip-NN-*.mp4
├── working/
│   ├── audio/{vo-raw.json, vo-full.mp3, vo-full-1.35x.mp3, vo-full-timestamps.json, beats.json}
│   ├── segments/beat-NN-*.wav
│   ├── clips/creator-beat-NN-*.mp4
│   ├── lipsync-frames/frame-NN-*.png
│   ├── clips-lipsynced/lipsync-NN-*.mp4
│   ├── clips/composite-NN-*.mp4
│   ├── concat.txt
│   └── scripts/composite.py
└── finals/
    ├── master-stitch.mp4
    └── master-final.mp4         ← shippable

Quality Checks

• VO duration check: raw VO ≤ 60s before atempo. If longer, script is too long — push back at S2.5.
• VO segment min duration: all 9 segments ≥ 2.0s after padding. CTA is usually the offender.
• Anchor portrait check: face takes ~25-35% of frame height. If too tight, the PiP collides; if too loose, the face reads small. Re-roll if outside this range.
• Seedance audio ref check: every audio ref ≥ 2.0s (Seedance hard requirement).
• Lipsync pose check: the extracted frame for each beat shows the creator clearly with the gesture visible. Re-extract at a different timestamp if not.
• Composite preview: sample at least 3 frames across the master (middle of each PiP beat). All overlays present, face uncovered, label not collided with anything.
• Transcript verify: /watch:watch returned transcript matches script-locked.md word-for-word.
• Lip sync verify: mouth visibly moves on every audible syllable. The biggest tell is silent mouths during loud VO — that means Veed Fabric got a wrong audio binding.
• Total spend logged: sum of gpt-image-2 + ElevenLabs + Seedance + Veed Fabric + fal-Whisper. Typical: $25-35.

Failure Modes

Symptom	Likely cause	Fix
Seedance call fails with audio_duration_too_short	beat segment < 2.0s (usually CTA)	Pad with ffmpeg -af "apad=pad_dur=N" -t 2.6 then retry
PiP loads as a 100-byte file	LFS pointer not pulled	git lfs pull or substitute another real mp4
Composite shows green/black PiP	source mp4 codec confused ffmpeg's geq filter	Re-encode the PiP source: ffmpeg -i in.mp4 -c:v libx264 -pix_fmt yuv420p out.mp4
Lip sync still off after Veed Fabric	wrong audio binding (script passed VO segment for a different beat)	Print the path mapping before invocation; ensure frame-NN.png and beat-NN.wav agree on NN
Pointing hand snaps back / disappears mid-beat	Veed Fabric is animating the body — usually triggered by a loose-clothing or wide-framing source still	Re-extract a tighter framing; or accept frozen-body trade-off (the reference video also has frozen bodies)
Caption emphasis lands on filler words	heuristic	Pass --emphasis "<the real punch words>"
PiP audio drowns the VO	audio gain too high	Drop [a_pip] volume=0.18 to volume=0.10 or strip with -an and use SFX only
Beat label overlaps face	y ratio too high	Push label down (raise LABEL_Y_RATIO from 0.74 to 0.80)
Stitched master has stutter at beat boundaries	concat-with--c copy failed because per-beat composites have different keyframe layouts	Drop the -c copy flag — let ffmpeg re-encode at concat: ffmpeg -y -f concat -safe 0 -i concat.txt -c:v libx264 -preset fast -crf 20 -c:a aac -b:a 192k out.mp4
Whole video reads "robotic AI"	atempo too aggressive on a script with long sentences	Drop atempo from 1.35 → 1.25, accept the longer runtime
Veed Fabric output is shorter than input audio	leading silence on the WAV file confused Veed	ffmpeg -ss 0.1 -i in.wav out.wav to trim leading silence

Reference run

Canonical reference: creator-content/test-videos/01-seven-content-formats/

• Concept: "7 content formats to build your brand"
• Beats: hook + 7 (Podcast/B-roll/Clipper/Green Screen/Opinion Container/Raw Native/Direct to Camera) + CTA
• Output: finals/master-final-v3.mp4 (44.0s, 1080×1920)
• Total spend: ~$30
• Pipeline iterations: v1 (Seedance only, $14) → v2 (regen + layout fix, +$10) → v3 (Veed Fabric lipsync chain, +$6.30) — see HOW_TO.md.

Composition

This molecule is intentionally a thin shell over the orchestrator + atoms. The only molecule-local code is:

• scripts/slice_vo.py — VO segment slicing using alignment timestamps
• scripts/render_overlays.py — PIL renders for title pill, beat labels, badges
• scripts/composite.py — per-beat ffmpeg filter graph (rounded-corner PiP + overlays + audio mix)

Everything else is invoked as documented atoms. If any of these three scripts grow beyond simple glue, factor them into proper atoms (e.g. a render-overlay-pip-card atom).

See also

• references/character-descriptor.template.md — template for the gpt-image-2 anchor prompt
• references/storyboard.html.template — single-frame storyboard scaffold
• references/geometry-constants.md — full PiP / title-pill / label / badge coordinates with rationale
• references/cost-breakdown.md — atom-by-atom cost table with worked example
• tests/sample-input.md — fixture inputs (concept + 9 beats) for a smoke test
• tests/expected-output.md — acceptance criteria for a passing render
• creator-content/test-videos/01-seven-content-formats/ — canonical end-to-end reference run