frame-extraction

name	frame-extraction
description	Domain expertise for slicing a downloaded YouTube video into image frames using ffmpeg — with sampling strategies tuned to the downstream task (file reconstruction, PRD evidence, diagram capture). Sub-skill of `youtube-extraction`.

Frame Extraction

Sub-skill of youtube-extraction. You receive a path to the downloaded video, <basename>, and a mode flag from the caller.

Goal

Place individual frame images at:

docs/youtube-extraction/<basename>/frames/

…using a sampling strategy appropriate for the caller's mode. Frames are disposable — youtube-extraction deletes this directory after the operation completes.

Why mode matters

Different downstream tasks need different sampling.

Mode	Caller	Strategy	Why
file-reconstruction	/recreate-files	Scene change + 1 fps cap	File panes scroll; you need enough overlap between frames to stitch lines, but not so many that OCR cost explodes.
prd-evidence	(other operations)	Fixed 5-second interval	PRD evidence wants a coarse, predictable index — every screen shown for ≥5s gets at least one frame.
diagram-capture	(other operations)	Scene change only	Diagrams are static — one frame per scene is enough.

For youtube-extraction's recreate-files operation, always use file-reconstruction.

Strategy: file-reconstruction

ffmpeg -i docs/youtube-extraction/<basename>/video.mp4 `
  -vf "select='gt(scene,0.2)+not(mod(n,30))',showinfo" `
  -vsync vfr `
  -q:v 2 `
  docs/youtube-extraction/<basename>/frames/frame_%05d.jpg

Decoded:

• gt(scene,0.2) — keep frames where the scene-change score exceeds 0.2. This catches IDE scrolls, slide transitions, terminal clears.
• not(mod(n,30)) — also keep every 30th frame (≈1 fps at 30 fps source) so we have temporal anchors even during long static screens.
• vsync vfr — variable frame rate output so timestamps stay correct.
• -q:v 2 — high JPEG quality (lower number = better). Code OCR is sensitive to compression artefacts; the storage cost is acceptable for tutorials.

Strategy: prd-evidence (5-second interval)

For completeness, since other operations need it:

ffmpeg -i docs/youtube-extraction/<basename>/video.mp4 `
  -vf "fps=1/5" `
  -q:v 3 `
  docs/youtube-extraction/<basename>/frames/frame_%05d.jpg

Build a frame index

Whichever strategy you used, write frames.index.json alongside the frames mapping filename → seconds_into_video. The downstream frame-content-recognition sub-skill needs timestamps to cross-reference with the transcript.

[
  { "file": "frame_00001.jpg", "t": 0.000 },
  { "file": "frame_00002.jpg", "t": 4.733 },
  …
]

You can derive timestamps from ffprobe or from ffmpeg's showinfo filter output. Store seconds as a float; the parent skill formats [mm:ss] for display.

Performance notes

• Resolution stays native. Don't downscale — OCR is the bottleneck and you want every pixel ffmpeg gives you.
• JPEG vs PNG. JPEG with -q:v 2 is fine for OCR and roughly 1/5th the disk of PNG. Use PNG only if the video is short (< 5 min) and you want pixel-perfect frames for screenshot reuse later.
• Long videos. A 60-minute tutorial in file-reconstruction mode can produce 1500–3000 frames. That's expected. The cleanup step at the end of the parent operation deletes them.

Idempotence

If frames/ already contains files matching the expected pattern and frames.index.json exists, skip extraction unless the caller forces a refresh.

What to surface back

• The frames directory path
• Path to frames.index.json
• Frame count and total duration covered
• Mode used