Use this skill when adding audio to programmatic videos - generating narration with ElevenLabs TTS, sourcing royalty-free background music, creating SFX with FFmpeg, implementing audio ducking, or mixing multiple audio layers in Remotion. Triggers on ElevenLabs, text-to-speech, voice generation, background music, sound effects, audio mixing, and volume ducking.
Use this skill when adding audio to programmatic videos - generating narration with ElevenLabs TTS, sourcing royalty-free background music, creating SFX with FFmpeg, implementing audio ducking, or mixing multiple audio layers in Remotion. Triggers on ElevenLabs, text-to-speech, voice generation, background music, sound effects, audio mixing, and volume ducking.
When this skill is activated, always start your first response with the :speaker: emoji.
Video Audio Design
Video audio design is the practice of layering narration, sound effects, and
background music into programmatic video compositions. Great audio transforms
a slide-deck video into a polished production - narration guides the viewer,
music sets the emotional tone, and SFX punctuate key moments. This skill covers
generating speech with ElevenLabs and alternative TTS providers, creating
synthetic sound effects with FFmpeg, sourcing royalty-free background music,
implementing audio ducking so speech stays intelligible, and mixing all layers
together in Remotion compositions with frame-accurate timing.
When to use this skill
Trigger this skill when the user:
Wants to add narration or voiceover to a programmatic video
Needs to generate speech with ElevenLabs, OpenAI TTS, or Edge TTS
Asks about voice selection, voice settings, or voice cloning
Wants to add background music or needs royalty-free music sources
Asks about creating sound effects programmatically
Wants to implement audio ducking (lowering music during speech)
Needs to mix multiple audio layers in Remotion
Asks about audio timing, volume levels, or frame-based audio sync
Do NOT trigger this skill for:
Video scripting or storyboarding - use the video-scriptwriting skill
Remotion component architecture or rendering - use the remotion-video skill
Professional audio production in a DAW (Ableton, Logic, Pro Tools)
Music composition or MIDI programming
Key principles
Layered audio architecture - Every video has three audio layers:
narration on top (loudest), SFX in the middle (accent volume), and
background music at the base (lowest).
Narration drives timing - Generate narration first, measure its
duration, then set scene timing to match. Never fit narration into
arbitrary scene lengths.
Duck music during speech - Background music must drop 50-60% when
narration plays. Use smooth ramps (10-15 frames) to avoid jarring jumps.
SFX as accents, not distractions - Keep SFX short (under 0.5s),
subtle in volume, and relevant to on-screen action.
Test audio in context - Always preview the full mix with all layers
together. Listen for muddy speech, volume spikes, or dead silence.
Core concepts
3-layer audio architecture
Layer
Role
Base Volume
During Narration
Narration
Conveys information, drives pacing
0.8-1.0
N/A (top layer)
SFX
Accents transitions and actions
0.3-0.5
0.3-0.5 (unchanged)
Background Music
Sets emotional tone, fills silence
0.3-0.5
0.15-0.25 (ducked)
ElevenLabs API model
ElevenLabs provides neural TTS via a REST API. The core flow:
Pick a voice (pre-made or cloned) - each has a voice_id
Send text + voice settings to /v1/text-to-speech/{voice_id}
Receive raw audio bytes (mp3 by default)
Write to file and measure duration for scene timing
Voice settings:
Setting
Range
Low
High
Recommended
stability
0-1
More expressive, variable
More consistent, monotone
0.4-0.6
similarity_boost
0-1
More creative
Closer to original voice
0.6-0.8
style
0-1
Neutral delivery
Exaggerated style
0.3-0.6
Audio ducking concept
Audio ducking reduces background music volume when narration starts and
restores it when narration ends. In Remotion, use interpolate():
Music volume: 0.4 ---\ /--- 0.4
\ /
0.15 \__________/
narration start → end
Ramps should take 10-15 frames (~0.3-0.5s at 30fps).
Frame-based audio sync in Remotion
useCurrentFrame() returns the current frame number
interpolate() maps frame ranges to value ranges (e.g., volume)
<Sequence from={frame}> places audio at a specific frame
<Audio volume={fn}> accepts a static number or a per-frame function
Convert seconds to frames: frames = seconds * fps.
Common tasks
1. Set up ElevenLabs API key and generate narration
Implement audio ducking - drop music 50-60% during speech
Hardcoding ElevenLabs API key
Key leaks into version control
Use environment variables: process.env.ELEVEN_LABS_API_KEY
Using TTS without measuring duration
Scene timing wrong, narration cut off
Measure audio duration with ffprobe after generation
SFX louder than narration
Distracts from content
SFX at 0.3-0.5, narration at 0.8-1.0
No fade on music start/end
Abrupt start/stop sounds like a bug
Add 0.5-1s fade-in at start and fade-out at end
Using low-quality TTS model
Robotic voice undermines quality
Use eleven_multilingual_v2 or tts-1-hd
Ignoring audio file format
Some formats add silence padding
Use MP3 for narration, WAV for SFX
Gotchas
ElevenLabs rate limits and character quotas - The free tier has a monthly character limit. Cache generated audio aggressively and only regenerate when text changes. Use a hash of the text as the cache key.
MP3 encoder padding adds silence - MP3 files often have 20-50ms of silence at the start. Trim with ffmpeg -af silenceremove=1:0:-50dB or account for the offset in frame timing.
Remotion Audio volume is per-component, not global - Two <Audio> components at volume 1.0 can clip. Keep total volume across simultaneous layers under 1.0.
FFmpeg SFX sound different across systems - Always specify -ar 44100 -sample_fmt s16 for consistent output across machines.
Voice consistency across scenes - ElevenLabs can produce different tones for the same settings with varying text. Use stability >= 0.5 for multi-scene narration.
References
For detailed patterns on specific audio sub-domains, read the relevant file
from the references/ folder:
references/elevenlabs-api.md - advanced ElevenLabs API patterns including
voice cloning, streaming TTS, websocket API, pronunciation dictionaries,
and quota management
references/audio-mixing-patterns.md - advanced mixing patterns including
multi-segment ducking, crossfades between scenes, volume automation curves,
and mastering the final mix
references/sfx-generation.md - comprehensive SFX generation with FFmpeg
including complex synthesis, layering multiple generators, and building a
reusable SFX library
Only load a references file if the current task requires it - they are
long and will consume context.
Companion check
On first activation of this skill in a conversation: check which companion skills are installed by running ls ~/.claude/skills/ ~/.agent/skills/ ~/.agents/skills/ .claude/skills/ .agent/skills/ .agents/skills/ 2>/dev/null. Compare the results against the recommended_skills field in this file's frontmatter. For any that are missing, mention them once and offer to install:
