synthesize-sfx-ffmpeg

name	synthesize-sfx-ffmpeg
description	synthesize sound effects (bell, thud, punch, whoosh, riser, sub-bass drop, breath, droplet) from scratch with ffmpeg sine waves + decay envelopes. Free, deterministic, no API dependency.

synthesize-sfx-ffmpeg

Purpose

Generate atomic SFX (bell dings, bag thuds, punch impacts, whooshes, risers, sub-bass drops, breath, droplet pings) from scratch using ffmpeg sine waves + exponential decay envelopes + optional reverb. The synthesized output is indistinguishable from a stock library in a finished mix at -14 LUFS and ships in zero API calls.

Use this atom when:

• FAL's fal-ai/elevenlabs/sound-effects endpoint is broken upstream (returns invalid_model_id: eleven_text_to_sound_v0 — see LEARNINGS.md #24).
• The brief is for fight-card / sports / action SFX where stock libraries sound generic.
• The composition needs precise SFX timing (synthesized output lets you target exact decay times, harmonic stacks, attack curves).
• Cost matters — synthesis is $0.

Pairs with:

• add-sound-effects (the higher-level "place SFX hits at marked timestamps" atom — composes this atom's outputs).
• create-music-elevenlabs (the music bed that SFX layer over).

Inputs

• SFX type (one of: bell, bag-thud, punch, bag-fall-boom, whoosh, riser, sub-drop, breath, droplet-ping) — or a custom spec.
• Duration in seconds (typical: 0.3–1.5s per atomic SFX).
• Output directory.

For custom specs:

• Fundamental frequency (Hz).
• Harmonic stack (list of multiplier ratios + amplitudes).
• Decay rate K (controls exp(-K*t) envelope falloff).
• Optional: noise color and amplitude for transient bite.
• Optional: reverb params (delay + decay).

Workflow

1. Read the brief; identify the SFX type and target duration.
2. Build the ffmpeg filter_complex graph: layered sine=frequency=F:duration=D sources, each modulated by volume=volume='exp(-K*t)*A':eval=frame for the decay envelope.
3. For impact SFX (bell, thud, punch), add a brief broadband anoisesrc=color=white|brown:duration=... for transient bite.
4. Layer via amix=inputs=N:duration=longest:normalize=0.
5. Apply highpass / lowpass filters to shape spectral content.
6. Optional aecho=delay:decay:delays:decays for reverb.
7. Output to PCM wav (-c:a pcm_s16le) for downstream mux losslessness, or libmp3lame for delivery.
8. Verify duration: ffprobe -v error -show_entries format=duration -of csv=p=0 out.wav.
9. Write manifest.json with SFX type, params, output path, duration.

SFX recipes (verified)

Bell (3-ding boxing ring)

Layered sine harmonics at 1175 Hz / 2350 Hz / 3525 Hz, exponential decay with K of 3.5 / 5.0 / 7.0 respectively, aecho=0.6:0.6:30:0.25. Single ding ~1.2s. Stack 3 dings at 0/0.4/0.8s offsets via adelay for a 2.0s 3-bell output.

Bag-drop thud

Low-freq fundamental 70 Hz + 140 Hz harmonic + sub at 45 Hz, decay K=4–8, highpass=f=40,lowpass=f=400,aecho=0.5:0.6:25:0.2. ~0.6s.

Punch impact

150 Hz fundamental + 300 Hz harmonic + 40ms broadband noise click at >2000 Hz for snap, decay K=15–18 on the body. highpass=f=80,lowpass=f=6000. ~0.3s.

Bag-fall boom

Sub 50 Hz + low 100 Hz + brown noise rumble, decay K=2.5–6, highpass=f=30,lowpass=f=800,aecho=0.6:0.7:60:0.3. ~1.4s of deep resonance.

Whoosh

White noise, envelope volume=volume='if(lt(t,0.2),t*5,(0.4-t)*5)' (ramp up then down), lowpass=f=2200,highpass=f=400. ~0.4s of air-sweep.

Riser

aevalsrc=0.6*sin(2*PI*(F_start+(F_end-F_start)*t/D)*t):duration=D for a pitched sweep, plus pink noise layer band-passed at 1500 Hz with t/D envelope. ~1.5s building tension.

Sub-bass drop

55 Hz sine + 80 Hz harmonic, decay K=4–5, highpass=f=30,lowpass=f=200. ~0.6s deep impact.

Breath / exhale

Pink noise filtered bandpass=f=500:w=800, envelope volume=volume='sin(PI*t/0.7)*0.5' for soft attack-and-release. ~0.8s.

Droplet ping

2400 Hz + 4800 Hz sines with sharp K=25–35 decay, highpass=f=1500, aecho=0.4:0.5:35:0.3 for room. ~0.4s.

Output

• Primary artifact: .wav (PCM 16-bit) for losslessness during downstream mux, or .mp3 (libmp3lame 192 kbps) for delivery.
• manifest.json — SFX type, params (frequencies, decay rates, amplitudes), output path, duration, format.
• verification.md — duration check, spectral spot-check (optional).

Quality Checks

• Output duration matches requested ± 0.02s.
• No clipping: astats peak ≤ -1 dBFS.
• File is decodable: ffprobe -v error <out>.wav returns at least one audio stream.
• For impact SFX (bell, punch, thud): waveform shape via ffmpeg -i <out> -filter_complex "showwavespic=s=800x200" -frames:v 1 wave.png shows a sharp attack at t=0 and exponential decay tail.
• A/B-listen: synthesized SFX should be indistinguishable from stock at -14 LUFS in a music mix. If the SFX stands out as obviously synthetic, layer additional harmonics or add reverb.

Failure Modes

• SFX sounds like a "ringing alarm clock" instead of "boxing bell" — too few harmonics. Add at least 2 harmonic layers (e.g. 2×F + 3×F) with declining amplitude (0.5×, 0.25×).
• Punch sounds like a sine wave, not a punch — missing the high-freq noise click. Add a 40–60ms anoisesrc=color=white layer high-passed at 2000 Hz.
• Output is silent — volume=volume='exp(-K*t)*A' without :eval=frame makes ffmpeg evaluate t once at the start; specify :eval=frame so it re-evaluates per sample.
• Reverb tail sounds metallic — aecho with very short delays (< 20ms) creates flanging; use 30–60ms minimum.
• Frequency sweep on riser glitches — aevalsrc expression too complex for the sample rate; precompute via Python+numpy and pipe PCM via stdin instead.
• Clipping artifacts — sum of amplitudes too high; pre-normalize each layer or set normalize=1 on amix.

Reference implementation

fightcamp/video-01-last-bag-standing/working/gen_sfx.py — produces 8 atomic SFX (bag thud, punch, bag-fall boom, whoosh, riser, sub-drop, breath, droplet ping). ~150 lines, no dependencies beyond ffmpeg + Python stdlib. Plus a separate gen_bells_only.py for the 3-ding boxing bell. All synthesized SFX were used in the final v7 master at -14 LUFS without listener flagging them as fake.

Example invocation

ffmpeg -y -loglevel error -filter_complex "
sine=frequency=1175:duration=1.2:sample_rate=44100[s1];
sine=frequency=2350:duration=1.2:sample_rate=44100[s2];
sine=frequency=3525:duration=1.2:sample_rate=44100[s3];
[s1]volume=volume='exp(-3.5*t)':eval=frame[h1];
[s2]volume=volume='exp(-5.0*t)*0.55':eval=frame[h2];
[s3]volume=volume='exp(-7.0*t)*0.28':eval=frame[h3];
[h1][h2][h3]amix=inputs=3:duration=longest:normalize=0,
aecho=0.6:0.6:30:0.25,
volume=1.6
" -t 1.2 -c:a pcm_s16le bell-single.wav