processing-video

name	processing-video
description	Audio and video processing with ffmpeg. Use when: user asks to convert, trim, merge, compress, or transcode video or audio files; extract audio from video; create GIFs or animated WebP from video; add subtitles or watermarks to video; change video resolution, framerate, or codec; normalize audio loudness; extract frames from video; concatenate clips; create thumbnails from video; strip or add audio tracks; convert between audio formats (MP3, AAC, FLAC, Opus, WAV); adjust volume; apply video filters; stabilize shaky video; generate waveform or spectrum visualizations; probe media file metadata. Triggers on 'ffmpeg', 'video', 'audio', 'transcode', 'MP4', 'MKV', 'WebM', 'MP3', 'AAC', 'FLAC', 'Opus', 'WAV', 'GIF from video', 'extract audio', 'add subtitles', 'video to gif', 'compress video', 'trim video', 'merge videos', 'normalize audio', 'framerate', 'resolution', 'bitrate', 'codec', 'ffprobe', 'waveform', 'spectrogram'.
metadata	{"version":"0.1.2"}

ffmpeg Toolkit

ffmpeg 6.1.1 is pre-installed with a full-featured build. Also available: ffprobe (media analysis) and ffplay (playback, limited use in container).

Before writing custom Python for media tasks, check whether ffmpeg handles it in a single command.

To interpret video content (summarize, describe, find scenes) rather than transform it, use the parsing-video skill — it samples frames into timestamped contact sheets that can be read as images.

Task Reference

Probe & Inspect Media

ffprobe — always start here to understand what you're working with.

ffprobe -v quiet -print_format json -show_format -show_streams input.mp4
ffprobe -v quiet -show_entries format=duration,bit_rate -of csv=p=0 input.mp4
ffprobe -v quiet -select_streams v:0 -show_entries stream=width,height,r_frame_rate,codec_name -of csv=p=0 input.mp4

Video Format Conversion

ffmpeg -i input.avi output.mp4                             # container swap (re-encode)
ffmpeg -i input.avi -c copy output.mp4                     # container swap (no re-encode, fast)
ffmpeg -i input.mp4 -c:v libx265 -crf 28 output.mp4       # H.265
ffmpeg -i input.mp4 -c:v libvpx-vp9 -crf 30 -b:v 0 out.webm  # VP9 WebM
ffmpeg -i input.mp4 -c:v libsvtav1 -crf 35 output.mp4     # AV1 (SVT, fastest AV1 encoder)
ffmpeg -i input.mp4 -c:v libjxl output.jxl                # JPEG XL (single frame)

Encoder selection guide:

Goal	Encoder	Typical flags
Compatibility	libx264	-crf 23 -preset medium
Better compression	libx265	-crf 28 -preset medium
Web delivery	libvpx-vp9	-crf 30 -b:v 0
Best compression	libsvtav1	-crf 35 -preset 6
Lossless archival	libx264	-crf 0 -preset veryslow

Lower CRF = higher quality. x264 default 23, x265 default 28, SVT-AV1 default 35 are visually similar.

Audio Format Conversion

ffmpeg -i input.wav -c:a libmp3lame -q:a 2 output.mp3     # MP3 VBR ~190kbps
ffmpeg -i input.wav -c:a libopus -b:a 128k output.opus    # Opus (best quality/size)
ffmpeg -i input.wav -c:a aac -b:a 192k output.m4a         # AAC
ffmpeg -i input.wav -c:a flac output.flac                  # FLAC lossless
ffmpeg -i input.mp3 -ar 44100 -ac 2 output.wav            # to WAV, set sample rate/channels

Extract Audio from Video

ffmpeg -i video.mp4 -vn -c:a copy audio.aac               # extract without re-encoding
ffmpeg -i video.mp4 -vn -c:a libmp3lame -q:a 2 audio.mp3  # extract as MP3
ffmpeg -i video.mp4 -vn -c:a libopus -b:a 128k audio.opus

Trim & Cut

ffmpeg -i input.mp4 -ss 00:01:30 -to 00:03:00 -c copy clip.mp4        # fast, keyframe-aligned
ffmpeg -ss 00:01:30 -i input.mp4 -to 00:01:30 -c copy clip.mp4        # -ss before -i = faster seek
ffmpeg -i input.mp4 -ss 00:01:30 -to 00:03:00 -c:v libx264 -c:a aac clip.mp4  # frame-accurate (re-encode)

-ss before -i seeks by keyframe (fast, may be imprecise). After -i decodes from start (slow, precise). For frame-accurate cuts, re-encode.

Concatenate / Merge

Demuxer method (same codec, no re-encode):

# Create file list
printf "file '%s'\n" clip1.mp4 clip2.mp4 clip3.mp4 > list.txt
ffmpeg -f concat -safe 0 -i list.txt -c copy merged.mp4

Filter method (different formats, re-encodes):

ffmpeg -i clip1.mp4 -i clip2.mp4 -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" -map "[v]" -map "[a]" merged.mp4

Resize & Scale

ffmpeg -i input.mp4 -vf "scale=1280:720" output.mp4                   # exact size
ffmpeg -i input.mp4 -vf "scale=1280:-1" output.mp4                    # width 1280, auto height
ffmpeg -i input.mp4 -vf "scale=-1:720:flags=lanczos" output.mp4       # height 720, Lanczos
ffmpeg -i input.mp4 -vf "scale=iw/2:ih/2" output.mp4                  # half size
ffmpeg -i input.mp4 -vf "pad=1920:1080:(ow-iw)/2:(oh-ih)/2" output.mp4  # letterbox to 1080p

Framerate

ffmpeg -i input.mp4 -r 30 output.mp4                       # simple (drops/dupes frames)
ffmpeg -i input.mp4 -vf "fps=24" output.mp4                # filter-based
ffmpeg -i input.mp4 -vf "minterpolate=fps=60" output.mp4   # motion interpolation (slow)

Animated GIF from Video

Use the two-pass palette method for quality:

ffmpeg -i input.mp4 -vf "fps=10,scale=480:-1:flags=lanczos,palettegen" palette.png
ffmpeg -i input.mp4 -i palette.png -lavfi "fps=10,scale=480:-1:flags=lanczos[x];[x][1:v]paletteuse" output.gif

Single-pass (simpler, lower quality):

ffmpeg -i input.mp4 -vf "fps=10,scale=480:-1" output.gif

Animated WebP from Video

ffmpeg -i input.mp4 -vf "fps=15,scale=480:-1" -c:v libwebp -lossless 0 -q:v 75 -loop 0 output.webp

Extract Frames

ffmpeg -i input.mp4 -vf "fps=1" frame_%04d.png             # 1 frame/second
ffmpeg -i input.mp4 -vf "select='eq(pict_type,I)'" -vsync vfr keyframe_%04d.png  # keyframes only
ffmpeg -i input.mp4 -vf "thumbnail=300" -frames:v 1 thumb.png   # best thumbnail from first 300 frames
ffmpeg -ss 00:00:05 -i input.mp4 -frames:v 1 screenshot.png     # single frame at timestamp
ffmpeg -i input.mp4 -vf "select='gt(scene,0.3)'" -vsync vfr cut_%04d.png  # first frame of each detected cut

The scene score is a frame-pair difference: it catches hard cuts but misses gradual transitions (dissolves/fades) and within-shot content changes; pans can false-positive. For robust shot detection use PySceneDetect (scenedetect -i input.mp4 list-scenes).

Subtitles

ffmpeg -i input.mp4 -vf "subtitles=subs.srt" output.mp4           # burn in SRT
ffmpeg -i input.mp4 -vf "ass=subs.ass" output.mp4                 # burn in ASS (styled)
ffmpeg -i input.mp4 -i subs.srt -c copy -c:s mov_text output.mp4  # soft subs in MP4

Subtitle rendering uses libass (full ASS/SSA styling support).

Text & Watermark Overlays

ffmpeg -i input.mp4 -vf "drawtext=text='Hello':fontsize=48:fontcolor=white:x=10:y=10" output.mp4
ffmpeg -i input.mp4 -i watermark.png -filter_complex "overlay=W-w-10:H-h-10" output.mp4
ffmpeg -i input.mp4 -vf "drawtext=text='%{pts\:hms}':fontsize=24:fontcolor=white:x=10:y=H-30" output.mp4  # timestamp

Audio Processing

ffmpeg -i input.mp4 -af "volume=1.5" output.mp4                    # volume boost
ffmpeg -i input.mp4 -af "loudnorm=I=-16:TP=-1.5:LRA=11" output.mp4  # EBU R128 normalization
ffmpeg -i input.mp4 -af "afade=t=in:d=2,afade=t=out:st=58:d=2" output.mp4  # fade in/out
ffmpeg -i input.mp4 -af "highpass=f=200,lowpass=f=3000" output.mp4  # bandpass
ffmpeg -i input.mp4 -an output_silent.mp4                           # strip audio
ffmpeg -i video.mp4 -i audio.mp3 -c:v copy -map 0:v -map 1:a output.mp4  # replace audio track

Video Stabilization (libvidstab, two-pass)

ffmpeg -i shaky.mp4 -vf "vidstabdetect=shakiness=5:accuracy=15" -f null -
ffmpeg -i shaky.mp4 -vf "vidstabtransform=smoothing=10:input=transforms.trf" stabilized.mp4

Crossfade & Transitions

ffmpeg -i clip1.mp4 -i clip2.mp4 -filter_complex "xfade=transition=fade:duration=1:offset=4" output.mp4

Transition types: fade, wipeleft, wiperight, slideup, slidedown, circlecrop, dissolve, and ~40 more.

Waveform & Spectrum Visualization

ffmpeg -i audio.mp3 -filter_complex "showwavespic=s=1280x240:colors=0x1e90ff" -frames:v 1 waveform.png
ffmpeg -i audio.mp3 -filter_complex "showspectrumpic=s=1280x720" -frames:v 1 spectrum.png

Compress / Reduce File Size

ffmpeg -i input.mp4 -c:v libx264 -crf 28 -preset faster -c:a aac -b:a 128k smaller.mp4
ffmpeg -i input.mp4 -c:v libx265 -crf 32 -preset medium -c:a libopus -b:a 96k smallest.mp4

CRF is the primary quality knob. Preset trades encoding speed for compression efficiency.

Available Codecs & Libraries Summary

Video encoders: libx264, libx265, libvpx (VP8), libvpx-vp9, libsvtav1, librav1e, libaom-av1, libjxl (JPEG XL), gif, png, apng, libwebp Audio encoders: aac, libmp3lame, libopus, libvorbis, flac, ac3, pcm_s16le Subtitle: ASS/SSA (libass), SRT, DVB, DVD, MOV text 426 formats, 562 filters, hardware accel stubs (vaapi, vulkan, opencl — limited use in container)

Key Constraints

• No GPU acceleration — CUDA device count is 0; vaapi/vulkan/opencl listed but no hardware available. All encoding is CPU-only.
• Container is ephemeral — long encodes on large files are feasible but the container resets between tasks. Work in /home/claude/, deliver to /mnt/user-data/outputs/.
• 4 CPU cores, 9 GB RAM — encoding is parallel but constrained. Use -preset faster or -preset veryfast for large files to avoid timeouts.
• ffplay exists but display is unavailable — use for probing only, not playback.
• No GPU-accelerated filters — stick to CPU filter variants.