topic-to-video

# Topic to Video You turn a topic — or a slide deck — into a narrated short video. Source of truth is `storyboard.json` — you edit text/timing; the scripts render the media. ## Tools (all free; verify with `scripts/_common.have_tool`) `soffice` (pptx→pdf), PyMuPDF `fitz`, `python-pptx`, **edge-tts** (preferred neural voice, free, cross-platform — needs network), macOS `say` (offline fallback), `ffmpeg`, PIL. Captions burn via PIL + a bundled font (no libass needed). Optional: OpenAI/ElevenLabs TTS if the user gives an API key. ## Pipeline scripts This is an index. Each line gives what a script is for, when to run it, and the rules that apply at the moment you run it. The full entry for each script is in **`references/pipeline-scripts.md`**: every flag, what each `meta.*`/`slide.*` field means in detail, the measured incident behind each rule, and the library-maintenance tooling. **Open a script's entry there before you** set a field this index does not explain, pass a flag not shown here, act on a gate finding you do not understand, or edit the library (a register, `runtime/`, a gate). **Ingest, voice, palette** - `scripts/doctor.py [--json]`: environment preflight, run once at the START (step 1). It exits non-zero only when a REQUIRED dependency (ffmpeg/ffprobe/PIL) is missing, and prints the stage each missing optional tool gates. - `scripts/ingest.py <input> <out_dir>`: pptx/pdf/image folder → `frames/` + skeleton `storyboard.json` (pulls pptx speaker notes). - `scripts/tts.py <out_dir> [--only N]`: narration → `audio/` + measured durations. edge-tts writes `.mp3` + `audio/NN.words.json` (per-word times, which karaoke, cues and the sync gate need); macOS `say` writes `.wav` and no word times. **`tts.py` REWRITES `storyboard.json`**, so run it (and any duration-editing pass of your own, e.g. tail-padding) **BEFORE you hand-set per-scene fields like `transition`**, or a later re-synth silently clobbers them. Order: finalize audio & durations → THEN set transitions/motion → validate → render (render only READS the board). A scored remotion film's clips also get their tails snapped to the beat grid (see **Sound**; words.json untouched). Every clip is mastered in place (EBU R128 I=-14, trailing dead air trimmed, word times stay valid; `meta.audio_master: false` opts out). **`meta.scene_pause` (default 0.55 s) is the pause the viewer HEARS between scenes, not the pad.** `tts.py` converts it per tier (remotion adds the 0.40 s cross-dissolve overlap, so both tiers sound the same) and targets the film's own 75th-percentile sentence pause PLUS a 0.45 s breath, floored at `scene_pause` and capped at 1.5 s; it prints the number. `meta.audio_breath` sets a RAW pad, only for a board that deliberately wants one. **Keep `scene_pause` ≤ ~0.9 s**: it is a pause, not a reading buffer (a long tail reads as the video hanging); reading time goes into a slower `meta.tts_speed`. - `scripts/theme.py <out_dir> [--mode light|dark] [--accent #hex] [--accent2 #hex] [--bg #hex] [--style <key>] [--brand-logo <png>]`: topic palette → `<out_dir>/theme.css`, which HTML pages link and the programmatic renderers parse (one colour world). **Run it before authoring pages OR programmatic scenes.** Contrast is gamma-correct WCAG: fg/bg ≥7:1, a SAMPLED accent nudged toward ≥3:1, an explicit `--accent` never touched. `--brand-logo`, or `meta.brand.logo` when no accent flag is passed, makes the logo's hue the accent. Re-run it on an out_dir themed before the contrast fix. Printed/filmic styles get fixed-seed grain; a data-heavy page opts out with `<body class="no-grain">`. - `scripts/dna_from_deck.py <out_dir> [--pptx deck.pptx] [--mode light|dark] [--dry-run] [--force]`: remotion tier. Writes `meta.design_dna` (with a `provenance` block) from the deck's OWN pptx theme (`scripts/pptx_theme.py`) and frames. Run it after ingest and BEFORE the shot-director. It never overwrites an authored DNA without `--force` and fails loudly when there is nothing to derive from. The html tier's `theme.py` is untouched. **Imagery** - `scripts/imagegen.py <out_dir> [--key KEY]`: OPTIONAL **paid** step. Generates AI atmosphere art for scenes carrying an `image_prompt`, cached under `.assets/`. It acts only when `meta.image_source == "openai"` (`meta.image_provider` `auto`/`codex`/`openai`; the Codex CLI path needs no key). The images are backplanes BEHIND HTML pages: never a slideshow of generated stills, never text or data, never a stand-in for a real named subject (that is `documentary`). `validate.py` enforces all three. - `scripts/imagefetch.py <query> <out_path> [--prefer W] [--min-width PX] [--require TERM...] [--duotone SHADOW HIGHLIGHT]`: the FREE `documentary` tier. Fetches a license-clean Wikimedia Commons image of the ACTUAL named subject and writes a `<out_path>.json` sidecar with licence + attribution + which field of the file's record named each query term. A candidate is taken only when its title, object name or categories attest EVERY key term of the query (or its description holds the whole phrase); otherwise it is refused, and when all are refused the reasons and a hint print to stderr — rephrase, or pass `--require` with the terms that are the subject when the rest only qualify it. Helpers: `fetch()` returns None when nothing relevant and license-clean exists, `attribution_line()` gives the on-screen `.credit`, `duotone()` tones the image into the palette. Reach for it on CONCRETE topics; the design-director picks `media.treatment: none|atmosphere|documentary` per beat. **Take the credit FROM the sidecar, never from memory**: `validate.py` ERRORS when a beat shows a `media/*` file without crediting its sidecar's author. **Render: html / image tier** (`scripts/render.py`) - `scripts/render.py <out_dir> [--sample N] [--only N] [--vertical] [--reduced-motion] [--preview]`: storyboard → `out.mp4` + `out.srt`. `--preview` is the cheap iteration render (12 fps, its own cache, writes `preview.mp4`, never touches `out.mp4`). Use it while experimenting, and DELIVER the full render. The fields it reads: - **Transitions are SEMANTIC.** `meta.transition` (`dip`|`none`) + `meta.transition_seconds`; per scene `slide.transition` (`dip`|`in`|`out`|`none`) + `slide.transition_seconds`. Map the beat's `relation`: continuation → hard cut; new-topic/payoff/close → an in-edge on the opener PLUS an out-edge on the predecessor (a dip needs both sides; validate lints an unpaired edge). `meta.transition_style` (`dip`|`fadethrough`|`drift`|`push`|`lift`|`wipe`|`blur`) is chosen ONCE per deck from the register. A transition is a claim about how two beats relate, never decoration: budget ~2-3 marked seams per video, never random, no spins, `blur` once per film at most. Both tiers share one seam vocabulary, and **`validate.py` ERRORS on a seam word the renderer cannot resolve** (`references/motion-grammar.md` "Transition rhythm" / "Transition STYLE"). - **Captions.** `meta.caption_mode` defaults to `overlay`: the subtitle sits directly on the frame's own ground, with no strip and no white/black halo. Its ink is the deck's own content colour for that ground, and its outline is the sampled ground. **Authors MUST keep content out of the bottom caption band (`caption_band_px`: 200 px ≈ 18.5% at 16:9, 257 px on 9:16/1:1/4:5 — keyframe_qa's finding prints the exact edge)**; keyframe_qa gates it. `band` is the opt-in footer strip (canvas grows; `meta.caption_band` `flush`|`contrast`). `meta.caption_scale` defaults to 1.0. Karaoke `meta.captions_karaoke` (`auto`|`off`) needs edge word times (CJK matched by char-span) and falls back to line chunks without them. The remotion tier follows the same rule with no scrim (`references/captions.md`). **A 9:16 film uses the platform-safe profile on this tier too** (`meta.safe_zone`, default `"social"`; `"none"` opts out; `band` mode is left as it was): each page is drawn in the scaled frame of `runtime/safeZone.ts`, its edges dissolving into a margin of its own ground (or, for a photo/gradient edge, its own edges extended and blurred), the caption is a size larger, ends short of the button column and sits above the platform's description, the brand bug sits below the tabs, and the full render writes `out.cover.png`. Pages are authored and gated exactly as before (keyframe_qa measures the page; its caption band lands under the caption). - **Timed callouts.** A callout with a `cue` anchor phrase (or `at` 0..1) appears as its region is spoken and holds to the clip end. Untimed callouts show for the whole slide. - **Motion.** `meta.intensity` / `slide.intensity` (`subtle`|`balanced`|`bold` or a non-negative multiplier) scale the Ken Burns move, clamped under the zoom/pan caps. **`shot` is authoring shorthand: translate it into `motion.from/to/zoom`, because render reads `motion`, not `shot`.** - **`meta.audacity`** (`conservative`|`balanced`|`bold`|`experimental`) is the register's RISK BUDGET and the register-contest width (skip / 3 / 5 / 7 candidates). It **NEVER relaxes the floor**: clarity, faithfulness, WCAG and every gate are the same at every level. A register move that breaks one is redone in-bounds. - `--reduced-motion` / `meta.reduced_motion`: the accessibility variant (static frames, opacity-only reveals). It busts the clip cache. - `scripts/scenes.py`: animated programmatic scenes (`type` `title`/`keypoints`/`chart`/`timeline`/`compare`/`html` + a typed `props`; `type` defaults to `slide`). They render ONLY their `props`, so they are faithful by construction, and they read `theme.css`. `props.climax:true` lands a counter at ~68% and holds it. `props.cue` (one anchor phrase per element, in spoken order) lands each bar/node/row as it is spoken. Cues change only WHEN an element appears, never the final frame. - HTML pages tier: `scenes.py:render_html` captures pages authored on `assets/html/base.css` (`references/html-pages.md`) in headless Chrome. For audio-synced reveals, tag each point `data-cue-index="N"` (spoken order) + `data-cue="<spoken anchor>"`. `scripts/cues.py` resolves the anchors from edge word times, or estimates them from text position when there are none. Capture runs over CDP (`scripts/_cdp.py`) and falls back to per-frame screenshots on its own; tune it with `meta.capture` (`cdp`|`screenshot`), `meta.chrome_workers`, `meta.jpeg_quality`. **Render: remotion cinematic tier** (`meta.renderer:"remotion"`) - `scripts/tier_advice.py`: decides from purpose + topic whether a film warrants this tier (interview ③). Remotion is for launch / promo / story films, NOT every video. - `scripts/render_remotion.py <out_dir>`: renders the bespoke-shot library (`remotion/LIBRARY.md`) one beat at a time and keeps unchanged beats in `.remotion_segments/`. `python3 scripts/remotion_segments.py <out_dir>` lists which beats the next render would redo. A visual edit re-renders that beat and the next; a narration edit re-renders every later beat. A render that dies resumes; `--whole`, `meta.segment_cache: false` or `REMOTION_SEGMENTS=0` render in one pass. `"remotion-legacy"` errors (that tier was removed). Run `cd remotion && npm install` once. Rules on this tier: - **The cut has to be authored.** No seam reaches a film unless something writes `transition_in`, so run `python3 scripts/seam_plan.py <out_dir>`. From each beat's `relation` it derives ~2-3 MARKED seams (≥5 beats apart, luminance-safe) and ~2-4 motivated `slide.camera` moves, and cuts the UNMARKED seams by the film's tempo: a launch or promo (`meta.purpose`) cuts every one hard (`cut`), a scored film half, a calm explainer keeps the dissolve (`meta.cuts` names it). An authored `transition_in` always wins. validate WARNS on a ≥12-beat film with no mark, or more than ~1-in-8 marked. Only `carry`, `focus`, `through-dark`, `drift`, `blur`, `container` and `cut` may sit on a dark↔light seam; any other mark there (`portal` included) is silently forced to a fade. The ANCHORED seams — `container` (an element carried across the cut), `grow`, `through-window`, `punch` (once, the payoff) — and the camera hand-off (`camera.handoff`) are in `references/motion-grammar.md`; an anchor is `transition_in.carry`, and `python3 scripts/carry_anchors.py` lists what each register draws. A bespoke direction must declare `meta.design_dna.seam` (e.g. `{type:'wipe',direction:'from-bottom'}` or `{type:'fade'}`), or its unmarked seams fall through to an anonymous cross-fade. - **Camera** is per beat and motivated: `slide.camera:{move:'push'|'pull'|'hold'}` (payoff → push, close/opener → pull, else hold; `handoff: true` carries a push across a continuation cut). A subject with a geography — scales, distances, layers — is a `WorldStage` run instead (Continuous stage, below). `meta.camera_drift` stays off. The camera bed is OFF by default (`meta.camera_bed: true` opts in) and never coexists with `camera_drift`. On a beat of 10 s or more an authored push/pull arrives in steps on the voice's clauses instead of an invisible creep, and on a beat that shows a LIST the item the voice names leads while its peers recede (`render_remotion.attach_phrases`; `meta.phrase_camera: false` or a beat's `"phrase_camera": false` turns both off). Neither adds a camera to a beat nobody gave one: that was measured to make type tremble (pipeline-scripts.md, `phrase_steps.py`). - **The cut carries.** On a film that cuts hard by tempo (kinetic or scored; `meta.cut_carry` says so outright) every hard cut carries one move through the cut frame (runtime/cutCarry): the leaving beat drifts and leans in, the arriving one comes out of the same drift already mid-entrance. `scripts/seam_ledger.py <out_dir>` measures each cut on out.mp4 and fails a kinetic film whose cuts join still frames. - **Footage is shown as itself.** `FilmScreen` plays a delivered film or a recording at its own pixels (a close-up region, the film's beats running beneath, a counted reel); a film's own pictures go in with `workshop.py <out_dir> media add <file> [--at s]`, and `workshop.py <out_dir> motion <Name>` says whether a built shot keeps moving after its reveal. - **The hidden cut.** `transition_in: {type: "sweep"}` (alias `hidden-cut`): a panel passes the lens and the beat changes behind it — two stops of one journey, "and on we go" (motion-grammar.md). - **Time** is grammar too, per beat and named by the words: `slide.time: {hits: [{cue}], hold: 2}` freezes the drawing on an impact (hit-stop), `freeze: {at: {cue}, hold: s}` holds a moment for a look, `sakuga: {until: {cue}, every: 2|3}` moves on held drawings until the climax and on ones from it (runtime/timeGrammar; every pause is repaid within a few frames, so the beat stays on its voice; validate and Film refuse a hold past the bounds). Use it where the beat has an impact, a moment to look at, or a climax — not as texture on every beat. A hit lands on a word where something ARRIVES: on a still frame it shows nothing. - **One structure, many beats.** When the argument IS one structure (a pipeline, a system), name it once in `meta.maps` and build it with `LivingMap` beats (`shot_props: {map, add, focus}`): laid out once, added to beat by beat, the camera moving through it, every map cut invisible (remotion/cards/LivingMap.md). - **Bookends.** A launch/promo film of 6+ beats may open COLD (`ColdOpen`: 2-4 of its own later beats flash past, then the title) and any film that long may end on a `Callback` (its earlier beats gather as cards round the closing line); `shot_director.py` proposes both by position, `shot_props.beats` names the beats (1-based). - **Places** (styled films, `style.surface.world: place`): each beat may name the `world` its subject is in — `orbit`, `sky`, `field`, `room` (`remotion/src/runtime/places.json`) — and is drawn inside it; a beat about an idea names none. Named per beat from the subject, never one place for the film. - **Reach.** After the forms and ideals, each beat's argument is read against the table of shapes the library has a