| name | broll-injector |
| description | Inject cinematic AI-generated b-roll into a talking-head ("spine") video for a visually richer presentation. Reads the presenter transcript (SRT or plain text), picks N cutaway moments, generates one silent clip per moment via the fal-ai-media skill (veo3.1-fast text-to-video, or nano-banana-2 stills turned into Ken Burns clips), and intercuts them over the spine with a deterministic, proven ffmpeg formula — full-frame cutaway or picture-in-picture. Use when the user wants to turn a HeyGen/Mimir/Alaric talking head into a graphics-rich cut, "add b-roll", or "make this video less of a static talking head". Resumable, fail-loud, file-based DAG: plan -> generate -> compose. A --dry-run path proves the whole pipeline with ZERO fal.ai spend by fabricating labeled placeholder clips.
|
| compatibility | Created for Zo Computer |
| metadata | {"author":"marlandoj.zo.computer"} |
B-Roll Injector
Turns a talking-head "spine" video into a visually rich cut with AI b-roll intercut
over it. One conductor (scripts/inject.ts) runs a 3-stage DAG; every stage is
file-based and skips when its artifact already exists, so runs are resumable and
--force re-runs from any point.
plan SRT/transcript -> <run>/00-plan.json (extract-plan.ts via /zo/ask)
generate 00-plan.json -> <run>/broll/<id>.mp4 (gen-broll.ts via fal-ai-media)
compose spine + clips -> <name>-broll.mp4 (compose.py, proven ffmpeg)
Quick start
bun /home/workspace/Skills/broll-injector/scripts/inject.ts \
--base SPINE.mp4 --srt CAPTIONS.srt
bun .../inject.ts --base SPINE.mp4 --srt CAPTIONS.srt --dry-run
bun .../inject.ts --base SPINE.mp4 --plan my-plan.json
The output lands at <run-dir>/<name>-broll.mp4 and the run dir defaults to
<dir-of-base>/broll-<name>/.
Why it exists
A static talking head is visually flat. This skill keeps the presenter's audio as the
backbone and cuts away to concrete, cinematic b-roll at the strongest moments —
without a human editor, and without a new credit surface (it routes through the
existing fal-ai-media skill on the user's own FAL_KEY).
Stages
1. plan — scripts/extract-plan.ts
Parses a word/line-level SRT (HeyGen caption output) or a plain transcript, then asks
this same Zo (headless, via /zo/ask) to choose N b-roll moments: where to cut away,
what to show, and a cinematic prompt for each. Emits:
{ "base_video": "...", "fps": 25,
"moments": [ {"id","start","hold","source","mode","model","prompt","trigger_phrase"} ] }
You can skip this stage entirely by passing --plan my-plan.json (hand-authored).
2. generate — scripts/gen-broll.ts
One silent clip per moment, normalized to a clean 25fps yuv420p mp4 so the compositor
only ever deals with concat-compatible video:
source: "t2v" → fal-media.ts t2v (veo3.1-fast, --no-audio) → normalizesource: "still" → fal-media.ts generate (nano-banana-2) → Ken Burns zoompan
Per-clip caching: a moment whose <id>.mp4 already exists is skipped unless --force.
3. compose — scripts/compose.py
Deterministic ffmpeg compositor. For each moment it time-shifts the clip
(setpts=PTS+start/TB) and gates a single overlay onto the spine
(overlay=...:enable='between(t,start,end)') — proven on the Zouroboros Academy
renders. Audio is always copied straight from the spine. Per-moment hold is clamped
to the generated clip's real duration, which kills the frozen/dark tail. --crossfade
swaps the hard cut for alpha fades (opt-in; hard cut is the proven default).
Modes
| mode | effect |
|---|
fullframe | b-roll covers the whole frame (cover + center-crop). The cutaway. |
pip | b-roll as a corner picture-in-picture over the presenter. |
--mode sets the global default; a per-moment mode in the plan overrides it.
Key options (inject.ts)
| flag | default | meaning |
|---|
--base | (required) | the talking-head spine mp4 |
--srt / --transcript / --plan | — | one source of moments (plan skips extraction) |
--out | <run>/<name>-broll.mp4 | final output |
--run-dir | <dir>/broll-<name> | resumable artifact dir |
--count | 5 | how many moments to auto-pick |
--mode | fullframe | global cutaway mode |
--source | t2v | t2v (motion) or still (Ken Burns) |
--model | veo3.1-fast / nano-banana-2 | fal model |
--crossfade | off | alpha fades instead of hard cuts |
--pip-scale / --margin | 0.42 / 40 | PiP size + corner inset |
--dry-run | off | placeholder b-roll, zero fal spend |
--force | off | re-run every stage |
Cost discipline
--dry-run fabricates a labeled colour-card clip per moment so the full DAG can be
validated end-to-end before paying fal. Always dry-run a new plan first, eyeball the
windows/modes, then drop --dry-run for the real render. A real run spends one
fal.ai generation per moment (veo3.1-fast t2v or a nano-banana-2 still).
See references/pipeline.md for the design, the ffmpeg formula, and the gotchas.
