open-edit

# open-edit — video orchestrator Renders video with `veed-engine-cli` (the veed render engine). Stylized captions over real footage — subtitles across every spoken beat — is the best-travelled path and most of what follows details it, but captions are one capability, not the boundary: edits, reframes, layered motion graphics, and compositions with no footage at all are the same engine and the same gates. The engine renders a `.wv` document, which **is an extension of CSS and can be treated as such**: an HTML fragment plus a `<style>` block, standard CSS throughout, no JavaScript and no proprietary timeline — `@keyframes` and `animation-delay` ARE the timeline. Your CSS knowledge transfers directly; only the engine's unimplemented parts have to be learned (`pipeline/director-brief.md` § ENGINE LIMITS). **INPUTS — any number of videos, INCLUDING NONE.** Footage is a layer inside that document — an optional one. What the input count changes is **how much of the work arrives already scripted, never whether the work is supported.** - **One video** — recommended, and the best-travelled path. The transcript, the canvas (dims + fps) and the base frames are all derived from the file, which is what lets a compiled recipe run at zero tokens. - **Several videos** — ONE batch, not one run each. `npx @veedstudio/openedit-cli transcribe` (either provider) and `npx @veedstudio/openedit-cli prep` both take `<video.mp4> [...]` and write one `runs/<key>` per video, so the provider question, the sign-in and any install happen once; STYLE, DESIGN + RENDER and MUX then run per `runs/<key>`. - **No video — FULLY SUPPORTED, not a degraded mode.** Motion graphics, stills, slides, generated imagery, audio-only sources. Author the `.wv` INLINE per `pipeline/director-brief.md` and run the SAME gates as every other run: `lint-template.ts` → `veed-engine-cli <dir> --verify` → `--record` (the DESIGN + RENDER step's RENDER + VERIFY block — none of it reads `meta.json`). Choose `<key>` from the ask, take the canvas and duration from the ask rather than from ffprobe, and drop only the steps that have no subject: the recipe draw (no footage to derive facets from), `probe-qa` (it diffs frames against source footage) and mux-audio (no audio track). the generate-recipe command is the scripted convenience for 1+ videos, NOT the definition of a supported run — its absence costs you the shortcut, nothing else. The captioned run is **fully scripted end to end**: recipes are COMPILED CODE (`refs/html/<id>/recipe.ts`), so a recipe-backed pick generates, verifies, and renders with zero tokens. The only spawned agent left is the OPT-IN vision-analysis pass (style-refine requests); the CREATIVE path face-1 (the user brought their OWN reference/brand/concept — their materials are the design authority) is authored INLINE by the orchestrator, and creative iteration on a delivered result is REMIXED inline (no subagent). There are **no per-shot intermediates and no user-approval gate**. Read `docs/FLOW.md` for the map. Default run (the FAST PATH) = PREFLIGHT → PREP → SAMPLE ONE STYLE → DESIGN + RENDER → MUX AUDIO (ANALYSE is SKIPPED). Vibe/genre/energy come from the transcript; placement comes from the brief's safe margins; the style is SAMPLED by script; word reveal timings are precomputed (`word-timings.json`). The runtime index is recipes-only, so a default run's DESIGN + RENDER step is always `npx @veedstudio/openedit-cli generate-recipe` — a SCRIPT, no model, no subagent: the recipe already did the design thinking, offline, and the code does the assembly + the full gate chain (lint → verify → record → probe). Creative face-1 is authored INLINE by the orchestrator; the only spawned agent is the opt-in analyse pass. REFINEMENT is declared by `analysis.json` existing (the ANALYSE step ran on user request) — placement then composes from it instead of the safe margins. ## User-facing output — talk like a product, not a pipeline The user asked for a video, not a pipeline tour. Internals are NEVER surfaced: run keys, ref/style ids (`hook-…`), "recipe"/"recipe-backed", seeds, facets, energy scores, beat counts, frame counts, gate names (lint / `--verify` / probe-qa), engine details. A fresh user has no idea what any of that means. The CLASSIC POOL is equally internal: never say "classic", "preset", "route", or a preset id (`simple`, `glass`, …) — "Classic route, 'simple' preset" is exactly the leak this section bars. The user asked for clean captions; say you're on it, then deliver. - **Never name the style — and never expose the mechanics of choosing it.** The ref id, its metadata, and the sampling machinery ("the sampled pick", "the draw", seeds, alternates) are all internal. Talk about "the style" as an abstract thing that exists for this video: "the style centres text mid-frame, so I'm switching to one that anchors low" — not "the sampled pick is…". Describe a delivered look only in plain visual terms (colour, size, placement). - **No step-by-step progress.** Don't announce transcript/prep/sampling/verify/probe/mux as they happen. One line when starting, then the deliverable with the preview URL. Silence in between is fine. - **"Render complete" = audio muxed.** Muxing is not a separate user-visible step; say the render is done only once `final/out.mp4` exists. - **No recap.** The delivery message is the `out.mp4` path plus at most a sentence or two about the result (on creative runs, the look you committed). Never a "What happened" list of stages, gates, counts, or QA results. - **Failures are the exception.** When a gate fails and you must stop, explain in plain terms what is wrong on screen and the options; quote raw FAIL lines only if the user asks. - Questions you must ask (renderer update consent, coverage gaps, probe failures) also stay in plain language: what it means for their video, not exit codes. ## PREFLIGHT — ALWAYS run at session start An installed skill contains this file (`scripts/preflight.sh` is the macOS shim that bootstraps Node); the setup itself is the published CLI's init command, and the full runtime may not exist yet. Resolve **SKILL_ROOT** as the directory containing this `SKILL.md`. Then resolve **WORKSPACE** by the first rule that applies: 1. **If SKILL_ROOT sits inside an Open Edit checkout, WORKSPACE is that checkout** — init reuses it, and the run exercises that code. 2. **Otherwise** WORKSPACE is the user's current project root, or the current directory outside a project — init creates its own runtime at `<WORKSPACE>/.open-edit/runtime` and every step below runs there. Init names which of the two it resolved (`reusing the local checkout at …` or `will use a managed clone at …`); read that line before trusting a run to be testing your changes. Resolve the supplied video to an absolute path from WORKSPACE before changing working directories. **If the session opened with a note saying preflight is ready AND naming `OPEN_EDIT_ROOT`, that note IS the preflight** — the hook ran it. Take the root from the note, read `AGENTS.md`, and start work. Running init again only repeats what has already been answered. Otherwise, at the start of the session, before doing Open Edit work, report what setup would do (init also keeps the workspace's SessionStart hooks current when it applies — idempotent, and a hook problem never blocks a run): ``` npx --yes @veedstudio/openedit-cli init --dry --workspace "$WORKSPACE" ``` Then run bare init to perform all safe, first-time workspace-local setup automatically: ``` npx --yes @veedstudio/openedit-cli init --workspace "$WORKSPACE" ``` This performs the first full runtime clone, installs pinned repository dependencies, and installs the renderer when their prerequisites already exist. It is idempotent. **Approval law — never weaken this:** machine-global dependencies and updates to existing code are never applied by bare init. If `--dry` or bare init prints `APPROVAL REQUIRED`, communicate EVERY exact action to the user and wait for an explicit affirmative response. Only when the user approves ALL reported actions may you run: ``` npx --yes @veedstudio/openedit-cli init --auto-approve --workspace "$WORKSPACE" ``` `--auto-approve` means the user agreed to every currently proposed global install and clean update. Never infer approval from the original render request. If the user approves only selected actions, perform only those exact commands yourself, then rerun `--dry`. If nothing needs approval, do not mention preflight. Exit **0** means stdout is **OPEN_EDIT_ROOT**; use it for every repo-relative command below. It does **not** mean setup is finished — a `--dry` run exits 0 while listing the `WOULD APPLY LOCALLY` work that bare preflight performs itself, and then ends on `not ready yet — run bare preflight …`. Read the final `preflight:` line, not the exit code: `ready — OPEN_EDIT_ROOT=…` means go. Exit **10** means **only** that `APPROVAL REQUIRED` was printed and the user must approve every listed action first. Exit **1** is a hard invariant/install error. For development, `--repository <URL-or-local-path> --ref <branch>` overrides the initial clone source. A managed clone records its origin, branch, and commit and rejects conflicting later overrides. A clean checkout is offered a fast-forward update; any local or untracked changes are reported and left untouched. Immediately after resolving OPEN_EDIT_ROOT, read `$OPEN_EDIT_ROOT/AGENTS.md` completely and follow it before running any repository command. Do this explicitly on every agent; never rely on Claude, Codex, Gemini, or another client discovering instructions inside the newly cloned runtime automatically. ## The flow Written for the footage case. PREFLIGHT, FOOTAGE, DESIGN + RENDER and PREVIEW hold for every run; **PREP** (transcript, frames, meta), **SAMPLE ONE STYLE** (the style draw) and **MUX AUDIO** derive from a source file, so a run with no video simply has no subject for them — see INPUTS: authoring, lint, `--verify` and `--record` are unchanged. **CUT** runs only when there is an edit to make, and it splits PREP: the sources are transcribed, the cut is assembled, and everything downstream then sees the edited timeline. ### PREFLIGHT — completed above · SCRIPT Do not run a second dependency implementation. `pipeline/scripts/preflight.mjs` (and its `.sh` twin) is only a compatibility wrapper around the init command. If `node` itself is missing, treat installing it as an APPROVAL REQUIRED action (macOS: `brew install node` — the shim reports this itself; Windows: `winget install --id OpenJS.NodeJS.LTS`). The provider choice — and any sign-in or install it implies — remains the interactive PREP step. ### CUT — assemble the footage you actually want · SCRIPT (only when there is an edit to make; splits PREP) Applies whenever the deliverable is not simply one source file played end to end: something to REMOVE (dead air, filler, a weak take), something to REORDER, or several clips to JOIN into one piece. When the ask is a single clip captioned as-is, this step has no subject — skip it and run the PREP step normally. **Where it sits.** `retime-transcript` needs the SOURCES already transcribed, so this step splits PREP rather than preceding it: transcribe the sources (the PREP step's own batch — one command, all files), then CUT, then run `prep` on the assembled file. Never transcribe the assembled file. **Measure the cut points; do not take them from the transcript.** ``` npx @veedstudio/openedit-cli speech-probe <video> [--range a:b] [--gap 250] [--window 10] [--json] ``` Reports the measured noise floor, the speech onset and decay, and every sub-threshold gap at least `--gap` long. `--gap` and `--window` are MILLISECONDS; `--range` and every number in the EDL are seconds. A transcript's word boundaries are not cut points: ASR can report a gap between two words where the waveform shows unbroken voicing, and cutting there slices a phoneme. When the probe finds no gap at a boundary, there is no cut there: keep the filler word rather than make an audible seam. Optional when the in- and out-points come from somewhere else entirely — a storyboard, or the user naming the takes. **Write the edit down, then apply it.** ```json { "sources": { "<id>": "<path to video>" }, "transcripts": { "<id>": "<path to that source's transcript.json>" }, "ranges": [ { "source": "<id>", "start": 1.6, "end": 7.05, "note": "free text, ignored" } ] } ``` `transcripts` is optional — without it each source's transcript is read from where the transcribe command wrote it (`runs/<key>/transcript.json` under the runtime root, `<key>` from the source video's filename). `note` is optional. Every source path must exist for both tools: the ranges are snapped to each source's frame grid, and the grid comes from the file. Ranges play in the order written: reorder them freely, a beat does not have to keep its chronological place. ``` npx @veedstudio/openedit-cli apply-edl --edl edl.json --out cut.mp4 [--crossfade 40] [--crf 20] ``` One encode, one canvas (the first range's source's), joins crossfaded so the room tone carries across them, and the sources' colour tags kept — sources that disagree on colour or frame rate are refused rather than relabelled. `--crossfade` is milliseconds. Each range is snapped to the frame grid, so the assembled timeline is exactly the sum of the snapped ranges, and the retimed transcript below lands on the same instants: picture, sound and captions agree to the frame. This is the joiner for an EDIT — parts of clips, in an order you chose. The FOOTAGE step's joiner is for whole generated clips whose shapes disagree, and it hands back an ordinary source file you then transcribe; this one hands back a cut whose transcript you RETIME instead. **Move the timings; do not buy them again.** ``` npx @veedstudio/openedit-cli retime-transcript --edl edl.json --out "$OPEN_EDIT_ROOT/runs/cut/transcript.json" ``` A cut changes WHEN words were said, never WHICH, so the per-word timings you already have are the timings of the new file. Write it where `prep` will look for it: `$OPEN_EDIT_ROOT/runs/<key>/transcript.json`, where `$OPEN_EDIT_ROOT` is the runtime root preflight printed (NOT the project directory under a managed clone) and `<key>` is the assembled file's name without its extension, whitespace turned into underscores — `runs/cut/` for `cut.mp4`. Skipped when there is no transcript at all: a run with no speech has nothing to retime. **Never transcribe the assembled cut.** It is the most expensive mistake available in this flow, and it buys nothing the arithmetic above does not already give you. Every transcription route — hosted, local and the mapper for your own service — refuses to overwrite an existing `transcript.json` without `--force`, and checks before it uploads or runs anything, so a stray one costs nothing. Then continue with the PREP step, giving it `cut.mp4`. ### FOOTAGE — a video to work from, generate one, or none · SCRIPT (only when the user brought none; runs before PREP) This step is about VIDEO only — stills, screenshots, slides, images and audio are inputs too, and a run can have them with no video at all. If the user supplied a video, continue to the PREP step unchanged. Otherwise do NOT assume a video is needed — read the ask first: - **They have a clip, or will record one** → take the path, waiting for the filename if it is still coming, then continue to the PREP step unchanged. - **VEED Fabric** (recommend this when they want a talking head) → a talking-head clip from a script, billed to one of their VEED workspaces. **Fabric REUSES VEED transcription's authentication** — the same veed.io account, the same OAuth login, the same stored token. There is no Fabric connector and no second sign-in: if they are already signed in for VEED transcription, they are signed in for this. Continue below. - **Another model** (Veo, Kling, Luma, anything on fal) → their auth and their bill, not ours; take the finished file into the PREP step. Say this in the SAME BREATH as that option, every time: captions come from TRANSCRIBING the clip's audio, so the clip must contain SPEECH. Veo 3 does. Veo 2, Kling, Luma and most of fal's catalogue are SILENT, and a silent clip yields an empty transcript and no captions. This is a warning, not a decision — say it, then let them proceed. - **No video — work with other sources** → raster graphics (stills, screenshots, photos), vector graphics (logos, shapes, SVG), motion graphics (titles, kinetic type, animation), or generated imagery, in any combination. Build the piece from those: go straight to the DESIGN + RENDER step, which reads no footage. If they have AUDIO it can still be transcribed for captions; PREP, the style draw and MUX are skipped for want of a video subject. Only when the ask is FOR a video of something but none is attached is there a real question — and even then "no video" sometimes just means they forgot to attach the file, so if it is ambiguous, ask which of these it is rather than guessing; a no-video answer is as good as any clip. On the Fabric path exactly three things stop and ask: this footage question, WHOSE credits, and the credit approval. Everything else — logging in, generating, reporting the charge — is a step: do it, say what happened, keep moving. **LOG IN BEFORE THE FIRST FABRIC COMMAND.** Every command below needs the VEED token — the SAME token VEED transcription uses, not a second one — so establish the login here rather than discovering it is missing mid-flow. If a command reports "No VEED login found", run the browser flow YOURSELF exactly as the PREP step's LOGIN block below describes — you launch it, the user never runs a command and never pastes a token. It is skipped when a token is already stored; one login covers generation AND transcription and lasts about a month. Draft the script yourself from their prompt and show it for edit. This is **two commands, and the script is typed only in the first one.** **WHOSE credits.** Generation spends the AI Playground credits of ONE workspace. With exactly one on the account there is nothing to decide, so it is used and NAMED with what it holds; with several and no prior answer the CLI stops and asks, and never picks. Run the confirm command with NO workspace flag first: `npx @veedstudio/openedit-cli generate --script "<the script>" --key <key>` With no workspace chosen it stops having spent nothing (exit 1) and prints every workspace with its name and credit balance. Put that choice to the user in plain terms (the names and what each has left, not ids if you can avoid them), then re-run naming the one they picked — that re-run is the PREP step below. That choice is remembered at `veed/.veed-workspace.json`, but a remembered choice is never a settled one: a spend pass whose workspace was only remembered REFUSES until the command names it again. Put the remembered workspace and its balance to the user, get a yes, and carry `--workspace <id>` on the spend command — the same flag switches it whenever they want a different one. **WHAT IT COSTS.** Generating draws AI Playground credits TWICE: the speech is synthesized first, then handed to **Fabric One Lipsync** (`veed/fabric-one-lipsync`), and both debits land on the same credit allowance. - **Fabric One Lipsync** — ~4 credits per SECOND of finished video, measured. - **Speech synthesis** — 2 credits per minute of generated audio, rounded up to the whole minute, so any read up to a minute costs 2. The quoted figure is the SUM of both. The script LENGTH is the lever, because it decides how long the read is — but how long is a property of the VOICE, and measured voices run from about 11 to 18 characters a second. So a 900-character script is a minute of video in one voice and a minute and a half in another, which is the difference between roughly 200 and 320 credits. The tool quotes at the rate it has measured for that voice, and quotes a RANGE when it has never heard it; repeat the range rather than flattening it to its low end, and never anchor the user on a small number. Too expensive → redraft a shorter script, or any of the other answers to the footage question; never a different workspace. The figure quoted before the spend is OUR estimate; VEED quotes no per-job price. **THE PRESENTER CAN BE THEIRS.** The 24 presets are a menu, not the boundary — the model takes an image, and it does not care where it came from. `--image <url|path>` uses the user's own still INSTEAD of a preset: a URL is fetched by VEED, a local file is uploaded from here. Reach for it whenever they brought a face, a logo, a character sheet or a frame they like. A preset carries a default voice and a user image does not, so `--voice` is required with `--image`. **A SET of images is ONE approval.** Several stills is one video made of several shots, so it is one question, not N. Write a shots file — `[{ id, script, image | character, voice }, …]` — and confirm the whole set at once: `npx @veedstudio/openedit-cli generate-set --shots shots.json --key <key> --workspace <id>` It prints every shot with its own share of the cost and ONE total, then spends the lot on a single `--yes`. The approval is hashed over the whole set: edit a line, reorder two shots, swap an image or a voice, and it refuses rather than buying something nobody saw. Each shot still runs under its own key, so a failure halfway leaves the shots already paid for alone and `--resume` collects them. Then join: `npx @veedstudio/openedit-cli concat-videos <out.mp4> <clip1.mp4> <clip2.mp4> [...]` It fits each clip into one canvas and pads the rest rather than cropping, because stills of different shapes produce clips of different sizes and nothing should lose its framing to a join. The result is an ordinary source file: transcribe it, caption it, render it like any other footage. **That joiner is for SOURCE clips that disagree, and only those.** It re-encodes and normalises the frame rate, which is right for generated clips of different shapes and wrong for anything else. The finished chapters of a long piece are joined by `npx @veedstudio/openedit-cli concat-chapters`, which stream-copies and refuses parts whose format differs rather than transcoding a whole film — see the DESIGN + RENDER step. Reaching for the wrong one costs a re-encode and silently resamples a 24 or 25 fps film to 30. **WHO presents it.** If the user has no opinion about the presenter, do not paste 24 thumbnails at them: `npx @veedstudio/openedit-cli sample-presenter --key <key> [--gender male|female] [--locale <locale>] [--portrait|--landscape]` PROPOSES one character + voice, prints two or three alternate PRESENTERS — each a face with a voice already suited to it, so a whole row can be swapped in — plus more voices for the chosen face, all with thumbnail and audio-preview links, and ends with the ready-to-run confirm command carrying that pair. The voice is drawn from what suits the face that won, so a male face never comes back with a female voice; gender-neutral voices suit either. `--gender` therefore narrows the FACE, and the voice follows it. `--portrait`/`--landscape` is how FRAMING gets chosen (the character IS the framing — there is no aspect parameter), so pass the one the user's format needs. It proposes, it never decides — it costs 0 credits, writes nothing, and the user overrules it with `--seed N` or by editing the two ids. Show them the pick and the alternates and get a yes before you run the confirm command. 1. CONFIRM (spends NOTHING): `npx @veedstudio/openedit-cli generate --script "<the script>" --key <key> --workspace <id>` It prints the script, the character, voice, framing ("portrait 9:16"), the workspace being billed with its balance, and the exact credit cost, records that approval at `runs/<key>/.fabric-pending.json`, and prints the exact next command. Show the user the cost in plain terms and get an explicit yes. NOT ENOUGH CREDITS is checked HERE too, before anything is written: if the workspace's balance is below the