ffmpeg

name	ffmpeg
description	Expert guide for FFmpeg video/audio processing, conversion, streaming, and filtering. Use when working with video, audio, multimedia files, transcoding, format conversion, or when the user mentions ffmpeg, media processing, video conversion, audio conversion, streaming, or codecs.

FFmpeg Expert

Expert guide for FFmpeg command-line tool for video/audio processing, transcoding, streaming, and filtering.

Quick Start

Basic format conversion:

ffmpeg -i input.mp4 output.avi

Convert video with specific codec:

ffmpeg -i input.mp4 -c:v libx264 -c:a aac output.mp4

Extract audio from video:

ffmpeg -i input.mp4 -vn -acodec copy output.aac

Instructions

Step 1: Understand the User's Goal

First, clarify what the user wants to do:

• Format conversion: Change container format (e.g., MP4 to MKV, AVI to MP4)
• Transcoding: Re-encode video/audio with different codecs
• Stream extraction: Extract video, audio, or subtitle streams
• Stream mapping: Select specific streams from input
• Filtering: Apply video/audio filters (scale, crop, overlay, etc.)
• Streaming: Process live streams or network protocols
• Screen recording: Capture desktop or application windows
• GIF creation: Convert video to GIF or create from images

Step 2: Analyze Input Files

Use ffprobe to inspect media files:

ffprobe -v error -show_format -show_streams input.mp4

Key information to gather:

• Container format (MP4, MKV, AVI, etc.)
• Video codec (H.264, H.265, VP9, etc.)
• Audio codec (AAC, MP3, Opus, etc.)
• Resolution, frame rate, bitrate
• Duration
• Number and type of streams

Step 3: Build the Command

FFmpeg follows this syntax:

ffmpeg [global_options] {[input_file_options] -i input_url} ... {[output_file_options] output_url} ...

Input Options (placed before -i)

• -ss position: Seek to position
• -t duration: Limit input duration
• -r fps: Set frame rate (input)

Output Options (placed before output url)

• -c[:stream_specifier] codec: Set codec (use copy for streamcopy)
• -b[:stream_specifier] bitrate: Set bitrate
• -s size: Set frame size (WxH)
• -r fps: Set frame rate (output)
• -vf filtergraph: Video filter
• -af filtergraph: Audio filter
• -map input_file_id[:stream_specifier]: Map streams
• -an / -vn / -sn: Disable audio/video/subtitle

Step 4: Handle Common Operations

Format Conversion (Streamcopy)

Fast, no quality loss:

ffmpeg -i input.mp4 -c copy output.mkv

Video Transcoding

# CRF-based quality (recommended)
ffmpeg -i input.mp4 -c:v libx264 -crf 23 -c:a aac output.mp4

# Bitrate-based
ffmpeg -i input.mp4 -c:v libx264 -b:v 2M -maxrate 2M -bufsize 4M -c:a aac output.mp4

Resolution Scaling

# Scale to specific size
ffmpeg -i input.mp4 -vf scale=1280:720 output.mp4

# Scale maintaining aspect ratio
ffmpeg -i input.mp4 -vf scale=-1:720 output.mp4  # Height 720, auto width
ffmpeg -i input.mp4 -vf scale=1280:-1 output.mp4  # Width 1280, auto height

Extract Streams

# Extract audio
ffmpeg -i input.mp4 -vn -acodec copy audio.aac

# Extract video
ffmpeg -i input.mp4 -an -vcodec copy video.h264

# Extract subtitle
ffmpeg -i input.mkv -map 0:s:0 subs.srt

Trim Video

# Using -ss (inputseek - fast)
ffmpeg -ss 00:01:00 -i input.mp4 -to 00:02:00 -c copy output.mp4

# Using -ss (outputseek - accurate but slow)
ffmpeg -i input.mp4 -ss 00:01:00 -to 00:02:00 -c copy output.mp4

Combine Videos

# Concatenate with filter
ffmpeg -f concat -safe 0 -i filelist.txt -c copy output.mp4
# filelist.txt contains:
# file '/path/to/video1.mp4'
# file '/path/to/video2.mp4'

Create GIF

# High quality GIF
ffmpeg -i input.mp4 -vf "fps=10,scale=320:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif

# Simple GIF
ffmpeg -i input.mp4 -vf "fps=10,scale=320:-1" output.gif

Screen Recording

# macOS
ffmpeg -f avfoundation -i "1:0" -r 30 output.mp4

# Linux (X11)
ffmpeg -f x11grab -s 1920x1080 -r 30 -i :0.0 output.mp4

# Windows
ffmpeg -f gdigrab -framerate 30 -i desktop output.mp4

Step 5: Use Stream Specifiers

Apply options to specific streams:

• :v - All video streams
• :a - All audio streams
• :s - All subtitle streams
• :v:0 - First video stream
• :a:1 - Second audio stream
• :0:v:0 - Video stream 0 from input 0

Examples:

# Different codecs for different streams
ffmpeg -i input.mkv -c:v:0 libx264 -c:v:1 libx265 output.mkv

# Copy audio, re-encode video
ffmpeg -i input.mp4 -c:v libx264 -c:a copy output.mp4

Step 6: Apply Filters

Video Filters (via -vf)

# Deinterlace
ffmpeg -i input.mp4 -vf yadif output.mp4

# Crop
ffmpeg -i input.mp4 -vf crop=1280:720:0:0 output.mp4  # W:H:X:Y

# Rotate
ffmpeg -i input.mp4 -vf "transpose=1" output.mp4

# Overlay text
ffmpeg -i input.mp4 -vf "drawtext=text='Hello':fontsize=24:fontcolor=white:x=10:y=10" output.mp4

# Overlay image on video
ffmpeg -i video.mp4 -i overlay.png -filter_complex overlay output.mp4

# Stack videos horizontally
ffmpeg -i left.mp4 -i right.mp4 -filter_complex hstack output.mp4

Audio Filters (via -af)

# Volume adjustment
ffmpeg -i input.mp4 -af "volume=1.5" output.mp4

# Normalize audio
ffmpeg -i input.mp4 -af "loudnorm" output.mp4

# Fade in/out
ffmpeg -i input.mp4 -af "afade=t=in:st=0:d=2,afade=t=out:st=50:d=2" output.mp4

Step 7: Advanced Scenarios

Two-Pass Encoding

# Pass 1
ffmpeg -i input.mp4 -c:v libx264 -pass 1 -an -f null /dev/null

# Pass 2
ffmpeg -i input.mp4 -c:v libx264 -pass 2 -b:v 2M output.mp4

Hardware Acceleration

# NVIDIA (NVENC)
ffmpeg -hwaccel cuda -i input.mp4 -c:v h264_nvenc output.mp4

# Intel (QSV)
ffmpeg -hwaccel qsv -i input.mp4 -c:v h264_qsv output.mp4

# macOS (VideoToolbox)
ffmpeg -hwaccel videotoolbox -i input.mp4 -c:v h264_videotoolbox output.mp4

HLS Streaming

ffmpeg -i input.mp4 -c:v libx264 -c:a aac -f hls -hls_time 4 -hls_playlist_type vod output.m3u8

Extract Frames

# Extract one frame per second
ffmpeg -i input.mp4 -vf "fps=1" frame_%04d.png

# Extract frame at specific time
ffmpeg -ss 00:00:05 -i input.mp4 -frames:v 1 frame.png

Best Practices

Quality Settings

• CRF (Constant Rate Factor): Recommended for quality-based encoding

  • Range: 0-51 (lower = better quality, larger file)
  • Recommended: 18-28 for H.264
  • Default: 23 for H.264

• Presets: Balance encoding speed vs compression

  • ultrafast, superfast, veryfast, faster, fast
  • medium (default)
  • slow, slower, veryslow

Performance Tips

1. Use streamcopy (-c copy) when not transcoding - much faster
2. Avoid re-encoding when possible - quality loss accumulates
3. Use hardware acceleration when available
4. Presets: veryfast for live, veryslow for final encode
5. Thread count: -threads option (usually auto is best)

Common Pitfalls

• Aspect ratio: Always consider aspect ratio when scaling
• Audio sync: Use -vsync options if audio drift occurs
• Codec compatibility: Ensure target device supports chosen codecs
• Bitrate vs quality: Don't set both CRF and bitrate
• File extensions: Use correct extension for output format

Common Codecs

Video Codecs

• libx264 - H.264 (most compatible)
• libx265 / hevc - H.265 (better compression)
• libvpx-vp9 - VP9 (WebM)
• libaom-av1 - AV1 (next-gen)
• mpeg4 - MPEG-4 Part 2 (legacy)
• copy - Streamcopy (no re-encoding)

Audio Codecs

• aac - AAC (standard)
• libmp3lame - MP3
• libopus - Opus (modern)
• libvorbis - Vorbis (WebM)
• pcm_s16le - Uncompressed PCM
• copy - Streamcopy

Container Formats

• mp4 - Most compatible
• mkv - Feature-rich
• webm - Web optimized (VP8/VP9)
• avi - Legacy
• mov - QuickTime
• flv - Flash video

Stream Selection Examples

# Map all streams from input
ffmpeg -i input.mp4 -map 0 output.mkv

# Map only video from first input
ffmpeg -i input.mp4 -map 0:v output.mp4

# Map specific streams
ffmpeg -i1.mkv -i2.mp4 -map 0:0 -map 1:1 output.mp4

# Negative mapping (exclude)
ffmpeg -i input.mp4 -map 0 -map -0:a output.mp4  # Exclude all audio

# Map by language
ffmpeg -i input.mkv -map 0:m:language:eng output.mp4

Troubleshooting

Command fails

• Check if input file exists and is readable
• Verify codec is supported: ffmpeg -codecs
• Check syntax: options go before the file they affect

Quality issues

• Increase CRF value for better quality (lower number)
• Use slower preset for better compression
• Check bitrate settings

Audio sync issues

• Try different -vsync modes
• Use -async 1 to sync audio to video
• Check frame rate settings

Performance issues

• Use hardware acceleration
• Try faster preset
• Reduce resolution first, then encode
• Check CPU usage with -threads

Advanced Reference

For complex filtergraphs, see filter documentation

For codec options, use:

ffmpeg -h encoder=libx264
ffmpeg -h decoder=h264

For format options:

ffmpeg -h muxer=mp4
ffmpeg -h demuxer=mp4