ffmpeg-skill

# ffmpeg-skill Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script you are about to run is the cheapest full flag list. A reference file costs as much to read as this file; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end). Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (the same trimmed to status/output/verified plus a `summary`; prefer it on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later; refuses if an input changed). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale): unset, SDR is x264 and HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses an HDR source (`color.py --to-sdr` first). Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe — they just don't write their artifact, nor side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per-tool: `contract --json`'s `dry_run` field. ## Workflow (always follow this order) 0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails on its own with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok` and the tool's `usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow. 1. **Probe what you must plan from.** Run `probe.py` on each input you plan the edit from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions. 2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; `--accurate` on `cut.py` only for frame-exact cuts. 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use it before long encodes and to report exact facts. `--fast` is preview quality, `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then. 4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions and overlays, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json** — one call, one JSON, one number for the user to change. 5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change the content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix those only when the request implies the answer, otherwise state the choice and its cost in one line. Mention WARNs; do not chase them. 6. **Verify the output.** Confirm duration, resolution, fps and audio match the request — from the writing tool's own `--json`/`--json-brief` probe, or `probe.py` — and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message. 7. **Keep the user's originals.** Never overwrite the source; write new files next to the input or where the user asked. Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment you run these scripts in: an existing output path is then refused (`kind: input`) instead of warned about, with `--overwrite` the one way to say "yes, replace it". It is the recommended agent setting, and 2.0's default. 8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled a clip to the first clip's frame, a `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption on someone's face. Audio-only jobs write `Look: not needed`. What to look for splits like `check.py`'s rows in step 5: - **Mechanical (this skill's own job to verify and report):** the specified text/logo is at the specified position, subtitles appear at the specified timestamps, dimensions are even. Letterboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag. - **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, text sits over a face, colours look washed out, a transition lands. These need deciding what the subject *is*, which belongs to the calling agent — say what you see in one line and let them judge it. With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, and don't stall waiting for a capability that isn't there. ## Before you run anything: what to ask, what to assume Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect, length and captions open), don't ask one per turn: propose one bundle with your defaults and let the user change any part ("Reels: 9:16 padded, 60 s, -14 LUFS, no captions — OK?"). One question, one answer, then the run. Never ask for what `probe.py` can tell you. - **Destination** decides aspect, length limit, loudness and codec; "for Reels" answers all four and names a template. No destination named and a plain cut/caption: keep the source format and say so. If the user says "export", "post" or "deliver", ask where. - **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, and say which. Ask when the content is a talk (trimming loses words) and the change is large. - **Captions** without a text source: `--transcribe` if a local whisper exists, otherwise ask for the text or a timed file; never invent dialogue. - **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once and reuse it. - **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns that off); `--lang ja|ko` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job. - **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of a silent centre guess, or ask which edge to keep. - Anything else (transition type, caption style): pick the conventional default, say what you picked, offer the alternative in one line. ## What this skill does and does not decide This skill cuts, joins, measures, syncs, exports and checks files — it executes an edit, it does not decide one. What belongs to the human, the calling agent or another skill: - **Which cut is right, or whether a deliverable is approvable** — this skill measures and reports (`check.py`'s PASS/WARN/FAIL, `cut.py`'s duration error); the user or a production agent decides whether that ships. - **What makes a highlight interesting** — `scenes.py --highlights` ranks by a measured proxy (audio energy, duration): candidates, not a verdict. - **Thumbnail or cover composition** — a design decision, not a measurement. - **Understanding what a video is *about*** — there is no vision here beyond `look.py`'s contact sheets, which exist for the calling agent's eyes, not for this skill to interpret. - **Judging what looks good** — "apply this LUT", "correct exposure by +0.3 stops" (`color.py`) is mechanical; "grade this scene to look cinematic" belongs to a colour-grading skill ([`color-grading-skill`](https://github.com/kajisho5/color-grading-skill)) that decides the parameters and then calls `color.py`. - **The words in a caption** — cue text is burned as written. Too long for the frame means a smaller size, `--max-lines`, or the user's own edit; never rewrite, shorten or paraphrase it, even when asked to "make it fit" — say so, offer `--max-lines 1` at a smaller size. - **Picking a subject or region you were not given** — "crop to x=200,y=0" is mechanical once the box is known; "crop to keep the speaker in frame" needs deciding *what* the speaker is — a judgement for the calling agent (from a `look.py` sheet). The line: same input + same explicit parameters always producing the same verifiable output belongs here; anything depending on taste, content understanding or what looks or sounds good belongs to whoever makes that judgement. If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option — never guess a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards), so it is never the fallback when a script's flag doesn't cover something. ## Request → script This table and `doctor --json`'s `tools` list are the source of truth for what exists: name only a script you have seen in one of them (there is no `doctor.py`, no `trim.py`, no `subtitle.py`). Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`, and the times in cue and chapter files) take seconds, `mm:ss(.fff)`, `hh:mm:ss(.fff)` or SMPTE `hh:mm:ss:ff` with `@fps` (`00:01:02:15@29.97`) — paste an NLE cue sheet as it is; length flags (`--min-silence`, `--margin`, `--fade`) are plain seconds. | User says | Do | |-----------|----| | "what's in this file", "how long is it" | `probe.py input.mp4` | | "cut from 1:20 to 2:05", "trim the first 10 s" | `cut.py input.mp4 --start 1:20 --end 2:05` | | "keep only these parts", "remove the middle" | `cut.py input.mp4 --segments 0-1:00,1:30-2:00` | | "make it exactly 60 seconds" | `fit.py input.mp4 --duration 60` (speed) or `--method trim` | | "make it vertical / 9:16 / square" | `fit.py input.mp4 --aspect 9:16 --fit pad` (or `--fit crop`; `--pad-fill blur` for blurred bars) | | "resize to a height/width" | `fit.py input.mp4 --height 1080` (or `--width`, or both for an exact frame) | | "crop to this exact box" (known x/y/w/h) | `crop.py input.mp4 --x 100 --y 0 --width 1080 --height 1920` | | "are there black bars on this?" | `cropdetect.py input.mp4` | | "this old footage is interlaced" | `deinterlace.py input.mp4` | | "it's grainy/noisy, clean it up" | `denoise.py input.mp4 --strength medium` | | "blur/pixelate this face/plate" (known box) | `redact.py input.mp4 --x 820 --y 140 --width 240 --height 240 --mode pixelate` | | "flat view out of this 360 video" (known yaw/pitch/fov) | `sphere.py insta360.mp4 --yaw 90 --pitch 0 --h-fov 100 --v-fov 70` | | "the horizon is tilted" (known degrees) | `straighten.py tilted.mp4 --degrees -2.5` | | "turn this image into a clip", "title card" | `insert.py title.png --duration 3` | | "slow zoom on a photo", "Ken Burns" | `insert.py photo.jpg --duration 6 --zoom in --pan right --width 1920 --height 1080` | | "rotate 90 degrees", "mirror it" | `fit.py input.mp4 --rotate 90` / `fit.py input.mp4 --flip h` | | "reverse this clip" | `reverse.py input.mp4` | | "stabilize this shaky footage" | `stabilize.py input.mp4` | | "make a blank/colour background clip" | `background.py -o bg.mp4 --duration 3 --width 1920 --height 1080 --color 0x101010` | | "turn these numbered frames into a video" | `sequence.py --dir frames --pattern "frame_%04d.png" --fps 24` | | "waveform/spectrum video for this track" | `waveform.py podcast.wav -o waveform.mp4` | | "hold on this frame", "freeze the last frame" | `freeze.py clip.mp4 --hold 2` | | "add black at the start" | `pad.py clip.mp4 --start 1.5` | | "speed up here, slow-mo there" (known segments) | `speedramp.py action.mp4 --segment 0-3:1.0 --segment 3-4:0.25 --segment 4-8:2.0` | | "loop this clip to fill 30 seconds" | `loop.py bg_loop.mp4 --duration 30` | | "cut to the product shot 0:12-0:16", "B-roll over this bit" | `broll.py talk.mp4 --insert product.mp4 --at 12 --end 16` (repeat `--insert/--at`; `--audio b\|mix`) | | "add chapters", "chapter markers for YouTube" | `metadata.py episode.mp4 --chapters chapters.txt` (`TIME TITLE` per line; streams copied; a `render.py` project spells it `"chapters"`). `--auto-chapters` proposes them from measured pauses/scene cuts, titled `Chapter N` for you to rename | | "set the title / artist / comment" | `metadata.py episode.mp4 --title "Episode 12" --artist "Studio"` | | "put these videos in a 4x2 grid" | `grid.py t1.mp4 ... t8.mp4 --cols 4 --rows 2` | | "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` | | "caption it with these lines" (text with times) | `caption.py input.mp4 --text cues.txt` | | "keep the subtitles toggleable", "mux in an SRT" | `caption.py input.mp4 --srt subs.srt --mode mux`; repeat `--srt file:lang` for several languages, `.mkv` for more than two | | "the captions are tiny / three lines on a Short", "don't chop the sentence" | `caption.py` shrinks the size until the cue fits `--max-lines` before splitting it (`--fit-size off` for 1.16 behaviour, `--min-size` sets the floor). Keep the template's `--max-lines` (2 on a vertical) and let the size drop; raising it to dodge a shrink stacks two words per line | | "our logo top-right", "a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` | | "a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` | | "webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` | | "remove the green screen" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` | | "turn this podcast into a video", "audiogram" | `render.py --template audiogram ep.m4a --image cover.png` — waveform over a still or colour plate; give an image or colour, nothing is fetched |