talking-goose-creator-video

name

talking-goose-creator-video

description

Produce a 9:16 fast-paced explainer video starring a felt-puppet mascot (default Goose) lip-synced in a creator-aesthetic apartment, with a clean two-zone overlay layout (big sample-or-text card on top, mascot PIP on bottom — never overlapping), per-tip pop SFX, sequential multi-image reveals at word-level VO timing, and word-level burned-in captions. Orchestrates ElevenLabs v3 VO + ElevenLabs sound-generation SFX + GPT-Image-1 character anchor + Veed Fabric per-scene lip-sync + Playwright overlay rendering + ffmpeg per-scene composite + burn-in-captions atom + ASS-patch step for brand-name spelling fixes. Hard-codes the v03 recipe end-to-end. Default mascot is the felt Goose but the character is a parameter — works for any felt/mascot character with a single high-quality reference still.

talking-goose-creator-video

Originally validated in content-goose at internal/gooseworks-ads/goose-talking-head-01..03/. Adapt paths to your project.

Purpose

The production line for the 9:16 fast-paced explainer-with-mascot format. Codifies the lessons from <your-project>/goose-talking-head-01 → 02 → 03 (three iterative builds of the same format).

Specifically tuned for:

• list-style explainers — "5 tools you need", "3 mistakes to avoid", "the SaaS stack in 30s"
• product / tool walkthroughs where each beat needs a logo + a sample/visual or a stand-alone text card
• founder/mascot rundowns where credibility comes from naming the actual things, not from cinematic production value

NOT the right molecule for:

• Long-form (>40s) talking head — use molecules/podcast-clip-animated-ad
• Brand spots / concept films — use molecules/motion-graphics/create-motion-graphic-video
• Pure UGC aesthetics (handheld, real apartment, no mascot) — use molecules/ugc-ad/create-ugc-diary-ad

Inputs

• <concept> — required. Free-form paragraph describing what the mascot is explaining. Should imply a list (5 tools, 3 steps, etc.).
• <beats> — required if <concept> doesn't enumerate. List of {label, vo_line, tool?, sample_image?, fillText?, logo?, samples?}. The molecule produces one scene per beat plus a hook + CTA. Practical max: 6 tips + hook + CTA = 8 total. More beats and the VO crowds.
• <character> — default goose-finger-puppet. {name, anchor_reference, descriptor_block}. anchor_reference is a path to a single high-quality reference still on solid bg. descriptor_block is a verbatim paragraph naming every distinguishing feature (see references/character-descriptors.md).
• <background_style> — default cozy-library. See references/background-presets.md for the 4 shipped presets.
• <voice> — required. {voice_id, model?, settings?}. Default model: "eleven_v3" with stability 0.34, similarity 0.80, style 0.18. Voice ID must be in VOICES.md OR explicitly approved.
• <tools> — required for tool scenes. List of {name, logo_path, sample_path?, samples?: [array], fillText?, accent_color?}. NEW v03 behavior: if no real sample image makes sense (e.g. "get credits", "ask claude", "use ffmpeg"), pass fillText instead — the spotlight renders a clean beige card with brown text. If a tip has multiple samples (e.g. "look at this and this and this"), pass samples (array) — the molecule will split that beat into N sub-overlays timed to the VO's word boundaries.
• <music_bed> — optional. Path to instrumental track. If absent, molecule emits a TBD warning.
• <output_dir> — destination project folder, e.g. <your-project>/goose-talking-head-NN/.
• <reuse_anchor> — optional path to a previously-locked anchor. When set, skips S3 (anchor lock) entirely. Saves $0.24 + ~30s. All three shipped projects (v01/02/03) used the same anchor, so this is the common case for series content.
• <reuse_lipsync_drafts> — optional bool, default false. If true, builds a cheap draft by concatenating previous projects' files trimmed to v03's duration, instead of calling Veed Fabric. The goose's mouth motion won't match the new VO phonemes but stays alive. Use for fast iteration on layout/overlays before committing to real lip-sync. See "Cheap-draft mode" below.

Composed Atoms

• atoms/voiceover/create-voiceover-elevenlabs — eleven_v3 with audio tags for the main VO
• atoms/image-generation/create-image-gpt-image-fal — character anchor; preserves identity on textured/felt characters
• atoms/image-generation/create-image-nano-banana-fal — fallback for cartoon/illustrated mascots only
• atoms/lipsync/create-lipsync-veed-fal — veed/fabric-1.0 at 720p, always 720p (480p reads soft in PIP)
• atoms/overlays/render-video-overlay-track — Playwright renderer for tool-spotlight + kinetic-title-clean overlays
• atoms/captions/burn-in-captions — frosted-subtle default; pair with a brand-spelling patch step
• atoms/assembly/stitch-videos-ffmpeg — per-scene composite + concat + audio mix (VO + SFX pops via adelay + amix)

In addition the molecule uses the ElevenLabs sound-generation API directly for a single pop SFX mixed at every tip boundary (see references/generate-sfx.sh), and the control-plane scripts at scripts/video_control_plane/{init_project,record_asset,record_state,validate_project}.py.

Workflow

The molecule walks a 9-state machine. Each state's contract and outputs are defined below.

S0 INTAKE  →  S1 SCRIPT LOCK*  →  S2 RAW MATERIALS  →  S3 ANCHOR LOCK*  →  S4 LIP-SYNC
   ↓                                                                            ↓
S8 DELIVER*  ←  S7 CAPTIONS  ←  S6 COMPOSITE + PROMOTE*  ←  S5 OVERLAYS  ←  ─ ─ ─ ─ ─ ─┘

* = human gate

S0 — Intake

1. Validate <character.anchor_reference> exists.
2. Validate <voice.voice_id> against VOICES.md; if missing AND not explicitly approved, FAIL.
3. Validate every <tools[].logo_path> and <tools[].sample_path> if provided.
4. Run init_project.py --video-folder <output_dir> --title "<title>" --format 9x16.
5. Copy <character.anchor_reference> to <output_dir>/raw-materials/reference/.
6. If <reuse_anchor> is set, copy it directly to <output_dir>/generated/character-lock/anchor-portrait.png (+ .meta.json and LOCKED.json) and mark S3 pre-approved.
7. Write idea-brief.md and design-brief.md (templates in references/intake-templates.md).

S1 — Script lock (GATE 1)

1. Author script/script-locked.md with three views: clean VO, tagged VO (light [curious], [deliberate], [emphasized], [reassuring]), per-scene mapping table.
2. Budget: ~60 words ≈ 17–22s clean; with v3 tags expands to ~24–32s. Plan for the longer end.
3. HUMAN GATE. Any change after this gate invalidates every downstream artifact.
4. Sequential reveal pattern: if a beat says "this... and this... and this" (or any N-reveal pattern), keep it as ONE script line — the molecule will derive sub-overlay timings from word-level Whisper timestamps in S5.

S2 — Raw materials

1. VO (main thread): eleven_v3 with with-timestamps endpoint, voice settings from <voice>. Save audio/voiceover.mp3 + audio/voiceover_timestamps.json.
2. Slice into per-scene segments: parse char-level timestamps, find each scene's start anchor (e.g. "One —", "Two —", "Three —"), compute end as the previous-char-of-next-scene-start, slice at silence-midpoints. Write audio/scene_timing.json AND audio/segments/plan.json (with tip1SubBoundaries if any beat has multi-reveal).
3. SFX: run references/generate-sfx.sh to generate ONE pop sound via ElevenLabs sound-generation API. Save to audio/sfx-pop.mp3. ~$0.05.
4. Tool assets (sub-agent if not pre-provided): brief asks for official transparent PNG logo + ONE clean hero image per tool. Hard rule for samples: no promotional collages with text baked in — they fight the spotlight card layout.

S3 — Anchor lock (GATE 2)

1. Skip if <reuse_anchor> is set. Just mark approved.
2. Build prompt from character descriptor + background preset (see references/character-descriptors.md + references/background-presets.md).
3. Generate 3 takes in parallel via gpt-image-1/edit-image at medium quality (~$0.24 total).
4. HUMAN GATE: operator picks one. Copy chosen to generated/character-lock/anchor-portrait.png. Write LOCKED.json with chain_strategy: "anchor-only".

S4 — Lip-sync

1. If <reuse_lipsync_drafts> is true: skip S4 entirely. In S6 the composite uses a concat of prior projects' masters instead. Mark lipsync_method: "reused-draft" in state. Use for cheap layout iteration only.
2. Sequential, with 3s pauses. Do NOT parallelize. The Veed Fabric storage upload endpoint (rest.fal.ai/storage/auth/token) 403s under high parallel load if the account has churned through many calls recently. Sequential with 3s sleep is reliable.
3. 7 calls (one per scene segment) at --resolution 720p. ~$0.15/sec, so ~$5 per 30s ad. Wall time ~25 min.
4. Sub-overlays don't need their own lip-sync clip — the same scene-NN.mp4 covers all sub-reveals; only the overlay PNG swaps in S5.

S5 — Overlays

1. Write production/overlay-plan.raw.json. One entry per overlay segment:
   • Hook + CTA → kinetic-title-clean with {upper, lower, accent: "upper"|"lower"|"both"}. NO caption field — captions are now burn-in (S7).
   • Tool scenes → tool-spotlight with EITHER samples: [path...] (real images) OR fillText: "SHORT POSTER TEXT" (beige card). NEVER both.
   • Multi-reveal beats (e.g. "look at this... and this... and this") → split into N sub-overlays. First sub has noSfx: false (default — gets the pop), subs 2..N have noSfx: true. Sub timings come from char-level "this" word timestamps in voiceover_timestamps.json.
2. Inline-encode logo + sample paths as data: URIs via references/inline-overlay-images.py.
3. Render via node atoms/overlays/render-video-overlay-track/scripts/render-previews.js --plan <path> --out generated/overlays --width 1080 --height 1920 --transparent.
4. Layout invariants (the whole point of the v03 redesign):
   • Sample/fill zone: top ~6% to ~51% of frame
   • Chip: top ~53% (logo + tool name, on a cream pill)
   • Goose PIP zone (composited later, NOT in overlay PNG): bottom ~58% to ~97%
   • Overlay PNG occupies only the top half — bottom half is reserved for the lip-sync PIP. Verify: open any tool overlay PNG, the bottom 40% must be transparent.

S6 — Composite + promote (GATE 3)

1. Stitch the 7 lip-sync clips: ffmpeg concat → scale=-2:1920,crop=1080:1920 → talking-head-master.mp4. (Veed returns 768×1152; this step puts it at exact 1080×1920 9:16.)
2. Cheap-draft path: if <reuse_lipsync_drafts>, concat prior projects' masters and trim to current VO duration (see "Cheap-draft mode" below) — skip step 1.
3. Run references/composite.py. For each overlay:
   • kinetic-title-clean (hook/CTA): full-frame goose + transparent title PNG.
   • tool-spotlight: blur+dim backdrop (boxblur=30:1, eq=brightness=-0.22:saturation=0.55) + sharp goose PIP at 444×790 centered horizontally, bottom-aligned at y=1100 (PIP center is in the lower-third "safe area" the storyboard reserves) + transparent overlay PNG on top.
4. Audio: mix VO directly with SFX pops via [1:a]adelay=BOUNDARY|BOUNDARY,volume=0.55[sN]; [0:a][s0][s1]...amix=inputs=N+1,volume=1.2[aout]. SFX skipped for any overlay with noSfx: true. The first scene also skipped (no pop on frame 1).
5. Concat per-scene composites → mux mixed audio → edits/master-final-v<N>.mp4. Promote by cp v<N>.mp4 master-final.mp4. Never write master-final.mp4 directly.
6. HUMAN GATE. Operator approves the pre-captions video.

S7 — Captions

1. Run atoms/captions/burn-in-captions/scripts/caption.py --video edits/master-final-v<N>.mp4 --style frosted-subtle. fal.ai Whisper transcribes, ASS template renders, libass burns.
2. Always run the brand-spelling patch step afterward via references/patch-captions.py. The patch handles:
   • MarginV (caption Y-position): default 0.18 × 1920 = 345px from bottom. For the talking-goose-creator-video layout, drop this to ~295px (50px lower than default) so captions don't cover the goose's mouth in PIP.
   • Brand-name fixes: each known mishearing (e.g. "Hicksfield" → "Higgsfield"). Add per-project mappings in the patch script's top-level config block.
   • Whisper word splits: compound brand names like "Higgsfield" get tokenized as "Higgs" + "Field" by Whisper. The patch script merges them back (both same-line and cross-cue cases) with summed karaoke kf timings to preserve the fill effect.
3. Re-burn the patched ASS with ffmpeg -i precaption.mp4 -vf "ass=patched.ass" -c:v libx264 -crf 20 -c:a copy patched.mp4.
4. Always keep both master-final-v<N>-precaption.mp4 AND the .captions/ sidecar dir. Lets you re-render captions with different style/wording without re-rendering the composite.

S8 — Deliver (GATE 4)

1. Optional: molecules/platform-variants/* for 1:1 / 16:9 reformats.
2. HUMAN GATE. Operator confirms ship.
3. Record final cost in .control-plane/project-state.json spendUsd.

Layout reference (the v03 redesign)

9:16 frame · 1080 × 1920
┌──────────────────────────────────┐
│                                   │ ← top 5% breathing room
│   ┌──────────────────────────┐   │
│   │                          │   │
│   │  SAMPLE IMAGE            │   │ ← ts-sample: top:120 height:860 (45%)
│   │  or                      │   │   beige card if no sample
│   │  3-IMAGE GRID            │   │
│   │  or                      │   │
│   │  beige fillText card     │   │
│   │  (#F4DEB0 bg, #5A3D1F)   │   │
│   │                          │   │
│   └──────────────────────────┘   │
│  ┌───────────────────────────┐   │
│  │ [logo] TOOL NAME          │   │ ← ts-chip: top:1010 (53%)
│  └───────────────────────────┘   │
│                                   │
│            ┌──────────┐           │
│            │          │           │
│            │  GOOSE   │           │ ← PIP composited at video stitch:
│            │   PIP    │           │   x=318, y=1100, 444×790 (9:16)
│            │ (lipsync)│           │   centered, white border, drop shadow
│            │          │           │
│            └──────────┘           │
│                                   │ ← bottom 2% breathing room
│  ┌─────────────────────────┐     │
│  │ word-level caption pill │     │ ← burn-in captions (S7)
│  └─────────────────────────┘     │   MarginV=295 (lower than default)
└──────────────────────────────────┘

Cheap-draft mode

For fast layout iteration without burning $5 + 25 min on Veed Fabric every run:

1. Concat one or more prior projects' talking-head-master.mp4 files using ffmpeg's concat demuxer.
2. Trim to current VO duration: -t <vo_duration>.
3. Drop the inherited audio (-an) — we'll add our own VO + SFX in S6.
4. Save as edits/talking-head-master.mp4 and run S6 composite normally.

Result: goose's mouth moves throughout (mouth shapes match the OLD VO, not the new one) but the layout/overlays/captions are real. Good enough for review and storyboard. Re-run S4 properly for the final.

Cost saved: ~$5/run. Time saved: ~25 min.

Output

<output_dir>/
├── idea-brief.md
├── design-brief.md
├── script/
│   └── script-locked.md
├── raw-materials/
│   ├── reference/                # character anchor source still
│   ├── logos/                    # transparent PNGs per tool
│   ├── samples/                  # one hero image per tool (or N for multi-reveal beats)
│   └── SOURCES.md
├── audio/
│   ├── voiceover.mp3
│   ├── voiceover_timestamps.json # char-level from ElevenLabs
│   ├── scene_timing.json         # per-scene start/end + tip1SubBoundaries
│   ├── sfx-pop.mp3               # single pop, mixed at every tip boundary
│   └── segments/
│       ├── scene-01.mp3 .. scene-07.mp3
│       └── plan.json
├── generated/
│   ├── anchor-options/           # all takes from S3 (kept for traceability)
│   ├── character-lock/
│   │   ├── anchor-portrait.png
│   │   └── LOCKED.json
│   └── overlays/
│       └── scene-NN-*.png        # one transparent overlay per overlay segment
├── clips/
│   └── scene-NN.mp4              # per-scene lip-synced 720p (or reused-draft)
├── edits/
│   ├── talking-head-master.mp4   # 7 lip-sync clips stitched, 1080x1920
│   ├── segments/                 # per-scene composited
│   ├── master-final-v<N>.mp4     # versioned masters
│   ├── master-final-v<N>-precaption.mp4  # before burn-in-captions (re-burnable)
│   ├── master-final-v<N>-precaption.captions/
│   │   ├── words.json            # Whisper word-level
│   │   ├── captions.{srt,ass}    # raw sidecars
│   │   └── *__frosted-subtle__{ass,fixed}.ass  # rendered + brand-patched
│   └── master-final.mp4          # promoted current
├── production/
│   ├── scene-contract.json
│   ├── overlay-plan.raw.json     # file paths (editable)
│   ├── overlay-plan.json         # data: URIs (renderer-ready)
│   ├── audio-plan.json
│   └── asset-manifest.json
└── .control-plane/
    └── project-state.json

Quality Checks

• All 9 states either ran or were explicitly skipped (with reason in state file).
• Every clips/scene-NN.mp4 duration is within ±10% of its audio segment (if <reuse_lipsync_drafts> is false).
• generated/character-lock/LOCKED.json exists.
• Overlay PNGs: bottom 40% of every TOOL spotlight PNG is transparent (PIP zone uncovered).
• edits/master-final.mp4 exists, is 1080×1920, h264 + AAC, duration matches audio/voiceover.mp3 ±0.5s.
• Captions don't cover the goose's mouth in PIP — MarginV ≥ 275px from bottom.
• Every known brand name in the script appears spelled correctly in the burned-in captions (visual review).
• The .captions/ sidecar dir is preserved.

Failure Modes

Symptom	Likely cause	Fix
Veed Fabric 403 on storage auth	Too many parallel calls recently	Drop to sequential with sleep 3 between calls (see S4 step 2)
FAL "User is locked. Exhausted balance"	Account credits depleted	Top up at fal.ai/dashboard/billing. The error is clear in the response body.
Whisper hears brand name wrong (e.g. Hicksfield)	Compound name with uncommon morphology	Add to patch-captions.py REPLACEMENTS dict; brand-name patching is now a default S7 sub-step
Whisper splits brand into 2 words ("Higgs" + "Field")	Same compound-name issue	patch-captions.py handles both same-line and cross-cue merges with combined karaoke kf timings
Captions cover the goose's mouth	Default MarginV=345 puts captions at ~72% Y; goose PIP head is at ~70% Y in this layout	patch-captions.py drops MarginV to 295 by default for this molecule
Sub-overlay reveal lands BEFORE the word	Slice timing didn't include lead-in	Add 0.15s lead-in past each "this" word: cuts.append(this_end + 0.15)
Tool name wrapping ugly in chip	Multi-word tool name	Shorten to one word + · separator (e.g. "Higgsfield · Marketing Studio" → just "Higgsfield" with marketing in eyebrow)
Caption shows "Higgs" alone at end of cue	Whisper tokenization split across cue boundary	The cross-line merge in patch-captions.py handles this — verify it's enabled
Cheap-draft master is shorter than VO	v01+v02 concat = 56s; trim to VO duration	Already handled — -t <vo_duration> flag. If your VO is longer than the sum of prior masters, add more prior projects to the concat.
Overlay PNG covers the goose PIP	New tool-spotlight component edited away the 40% bottom transparency	Verify CSS: .ts-sample height ≤ 860px out of 1920 (~45%); .ts-chip top ≤ 1080
Character drift on every GPT-image take	Descriptor block missing distinctive shape/proportion words	Add specifics: "tall and slender, NOT chubby"; "long droopy tube beak, NOT triangular bird beak"; "round white felt eyes SEPARATED, NOT touching"

Cost reference (per typical 30s run, real lip-sync)

Line item	Provider	Unit	Count	Cost
ElevenLabs VO (eleven_v3 w/ timestamps)	ElevenLabs	per char	~500	~$0.06
ElevenLabs SFX pop (sound-generation)	ElevenLabs	per request	1	~$0.05
Character anchor takes (skipped if <reuse_anchor>)	FAL gpt-image-1/edit-image medium	$0.08/img	3	~$0.24 (or $0 if reused)
Veed Fabric lip-sync (720p, sequential)	FAL veed/fabric-1.0	$0.15/sec	30s	~$4.50
Burn-in-captions Whisper transcription	fal.ai Whisper	per minute	0.5 min	~$0.005
Playwright overlay rendering	local	—	10	$0
ffmpeg composite + caption burn	local	—	—	$0
Total (with reused anchor)				~$4.65
Total (fresh anchor)				~$4.90
Cheap-draft mode (no lipsync)				~$0.15

When to escalate

• 3 review iterations completed and operator still flagging P0s: escalate to user with punch list.
• Character identity drift > 30% on 6+ takes: surface Soul ID training option.
• Voice ID isn't on VOICES.md AND operator hasn't approved: stop at S0 and ask.
• FAL balance exhausted mid-run: surface immediately with top-up link.

Decision Rules

• Hard cap practical beats at 6 tips + hook + CTA = 8 total scenes. More beats and the VO crowds.
• Prefer gpt-image-1/edit-image for the character anchor on felt/textured mascots; only fall back to Nano Banana for cartoon/illustrated characters.
• Reuse a previously-locked anchor whenever the same mascot is being used (<reuse_anchor> skips S3 entirely — saves ~$0.24 + 30s).
• Run Veed Fabric lip-sync calls sequentially with a 3s sleep between calls — parallel calls 403 from the storage auth endpoint.
• Always pair burn-in-captions with a brand-spelling patch step; Whisper mishears compound brand names (e.g. Higgsfield → "Hicksfield" / "Higgs Field").
• Tool-spotlight overlay PNG must keep the bottom 40% transparent so the goose PIP is uncovered. Verify before composite.
• Use <reuse_lipsync_drafts> for cheap layout iteration only — never ship a draft where mouth motion doesn't match the new VO.

Reference runs

• <your-project>/goose-talking-head-01/ — first build of the format. Old-style overlay (cream card with eyebrow + tagline + sample inside). No SFX, no burn-in captions.
• <your-project>/goose-talking-head-02/ — reused v01 anchor. Same old overlay style. First validation that anchor inheritance saves time.
• <your-project>/goose-talking-head-03/ — the v03 redesign. Clean two-zone layout, beige fillText cards, sequential multi-reveal, SFX pop on tip transitions, burn-in word-level captions with brand-spelling patches. This is the canonical reference for new runs — see the per-project HOW_TO.md.