audio-process

name

audio-process

description

Merge ElevenLabs MP3 chunks into a master track, trim silences, and tempo-normalize per voice preset. Use this skill whenever the user supplies multiple MP3 files from ElevenLabs (or any sequential narration chunks) and asks to combine, stitch, glue, склеить, объединить, собрать them; or asks for WPM analysis, темп речи, скорость речи, нормировать темп, normalize tempo, speed up voiceover, ускорить озвучку, замедлить озвучку, убрать паузы, trim pauses, remove silences. Trigger when the user mentions ElevenLabs chunks, narration MP3s, voiceover audio that needs joining, a specific TTS voice (e.g. "Hidden 6"), or asks to calibrate / register a new voice.

audio-process — Director

You orchestrate a voice-aware audio pipeline:

1. Merge ElevenLabs chunks → one master MP3.
2. Transcribe master via Replicate Whisper-diarization → word-level timings.
3. Analyze baseline WPM (informational).
4. Apply voice preset — pause-trim + tempo-normalize using a calibrated recipe stored per-voice in voice_presets.json. Produces the final voice-ready file.

For an unknown voice, branch to a calibration sweep first (see "Calibration workflow"), then save the chosen recipe as a new preset and proceed with step 4.

Voices and presets

Per-voice recipes live in:

/Users/alex/NewChapter/WhiteboardStack/audio-process/voice_presets.json

Each preset is a complete final-render recipe: pause-trim mode/ratio + tempo ratio that delivers the user's preferred final WPM for that voice. The user calibrated each one by listening to a sweep — the agent must NOT re-pick or "improve" registered presets without explicit user request.

Currently registered

Voice slug	Pause-trim	Tempo	Target WPM	Source
hidden_6	mode=ratio, ratio=0.45, min_keep=0.12	×1.038	187	chosen 2026-05-09
hidden_1	mode=ratio, ratio=0.55, min_keep=0.12	×1.1267	180	chosen 2026-05-12

Detecting which voice the user is processing

In priority order:

1. Filename pattern match — each preset has a filename_patterns list. If any chunk's filename contains a pattern, that's the voice. (Hidden 6_gen_sp112_… matches hidden_6.)
2. User-stated — they said "это hidden 6 голос" or similar.
3. Ask explicitly — "Какой это голос? Зарегистрированные: hidden_6. Если новый — нужна калибровка (1 раз на голос, ~3 минуты + Whisper ~$0.07)."

If filename matches a registered voice, you may proceed without asking.

Standard pipeline (voice has a registered preset)

SKILL_DIR=/Users/alex/NewChapter/WhiteboardStack/audio-process
SRC_DIR=/abs/path/to/episode/script
EP="<episode-friendly-name>"   # used for the merged file name
VOICE=hidden_6                 # from filename or user

Step 1 — Merge chunks

python "$SKILL_DIR/scripts/merge_chunks.py" "$SRC_DIR/chunk1.mp3" ... \
  --output "$SRC_DIR/$EP.mp3"

(See "Merge details" below if needed.)

Step 2 — Transcribe

python "$SKILL_DIR/scripts/transcribe_audio.py" "$SRC_DIR/$EP.mp3"

Skip if <merged>.transcript.json already exists (re-transcription costs money).

Step 3 — Analyze (informational)

python "$SKILL_DIR/scripts/analyze_wpm.py" "$SRC_DIR/$EP.mp3"

Show the user the baseline WPM. This is useful context but does NOT decide the pipeline — the voice preset does.

Step 4 — Apply voice preset

Read the preset for $VOICE from voice_presets.json. Then:

# 4a — pause trim per preset
python "$SKILL_DIR/scripts/trim_pauses.py" \
  "$SRC_DIR/$EP.mp3" \
  "$SRC_DIR/${EP}_paused.mp3" \
  --mode <preset.pause_trim_mode> \
  --ratio <preset.pause_trim_ratio> \
  --min-keep <preset.pause_trim_min_keep>

# 4b — tempo normalize per preset
python "$SKILL_DIR/scripts/normalize_tempo.py" \
  "$SRC_DIR/${EP}_paused.mp3" \
  --ratio <preset.tempo_ratio> \
  --output "$SRC_DIR/${EP}_voice-${VOICE}.mp3"

Tell the user where the final file is and report achieved WPM.

The intermediate _paused.mp3 can be kept (useful for debugging) or removed. The pre-tempo <EP>.mp3 master is always preserved.

Calibration workflow (unknown voice)

When the user introduces a voice not in voice_presets.json:

1. Run steps 1–3 of the standard pipeline (merge + transcribe + analyze) so a <merged>.analysis.json exists with word_count and natural WPM.

2. Run the sweep:

   python "$SKILL_DIR/scripts/calibrate_voice.py" \
     "$SRC_DIR/$EP.mp3" --voice-name <slug-the-user-named>
   

   Produces 8 variants in _calibration_<slug>/:

   Variants
   r40_none r40_to185 r40_to187 r40_to190
   r45_none r45_to185 r45_to187 r45_to190

   Plus a calibration.json with all params/durations/WPMs and a markdown table printed to stdout.

3. Show the user the table. Tell them to listen to each MP3 and pick.

4. When the user names a winner (e.g. "r45_to187"):

   • Look up the variant's pause_ratio, tempo_ratio, target_wpm in _calibration_<slug>/calibration.json.
   • Append a preset entry to voice_presets.json following the schema of hidden_6 exactly. Required fields:

     "<voice_slug>": {
       "description": "<one-line description from user>",
       "filename_patterns": ["<patterns the agent should match in chunk names>"],
       "natural_wpm_baseline" <baseline overall_wpm before any processing>
        
          
          < or >
          
          <ratio used;  if 'none' chosen>
          <// or >
       
        
        
     
     

After registration, future episodes for that voice take the fast standard pipeline (no sweep).

Calibration sweep grid is fixed

The sweep is pinned to 2 pause ratios × 4 tempo targets = 8 variants:

• pause ratios: 0.40, 0.45
• tempo targets: none, 185, 187, 190 WPM

These were validated by the user as the right exploration range. Don't add more or substitute different numbers — extra variants make the listening test harder, not easier. If a voice genuinely needs a different range (e.g. very fast natural baseline already >185 WPM), discuss with the user first and add an explicit override.

Defaults

• Default target WPM: 181 (channel-tuned baseline, sits between natural slow narration and the faster competitor average). Per-voice presets override this; analyze_wpm.py --target N overrides per-call.
• Output suffix: voice-applied final = _voice-<slug>.mp3. Plain tempo- normalized fallback (no preset) = _normalized.mp3.
• Silence between merged chunks: 0.5 s.
• Whisper model: thomasmol/whisper-diarization pinned (~$0.006 / min).
• Tempo filter: ffmpeg atempo (preserves pitch). Single pass for ratios 0.5–2.0×; chained automatically beyond that range.
• Pause-detect threshold: noise=-30dB, min duration 0.3s. Override via --detect-thr / --detect-min only if a recording has unusual noise floor.

Hard checkpoints

• For a known voice, the standard pipeline auto-runs through to the final file. Report the final path + achieved WPM. The user pre-approved the recipe at calibration time.
• For an unknown voice, stop after the calibration sweep and wait for the user to name a winner before registering and promoting.
• If the merged output filename already exists, ask before overwriting (the merge script supports --force).
• Never re-run transcription if <merged>.transcript.json already exists.

Working language

Communicate with the user in their language (Russian by default). Script output is English diagnostic.

File map

audio-process/
├── SKILL.md                          (this file — agent instructions)
├── voice_presets.json                voice registry (read by agent every run)
└── scripts/
    ├── _common.py                    shared helpers
    ├── merge_chunks.py               step 1
    ├── transcribe_audio.py           step 2
    ├── analyze_wpm.py                step 3 (informational)
    ├── trim_pauses.py                step 4a — pause trimming, multiple modes
    ├── normalize_tempo.py            step 4b — atempo
    ├── calibrate_voice.py            sweep tool for unknown voices
    └── requirements.txt

SKILL_DIR=/Users/alex/NewChapter/WhiteboardStack/audio-process.

Environment

All scripts auto-load .env from parent of cwd, then cwd, then skill's parent. They require REPLICATE_API_TOKEN (Whisper). The user's existing .env lives at /Users/alex/NewChapter/WhiteboardStack/.env.

If replicate / httpx / python-dotenv are missing: pip install -r "$SKILL_DIR/scripts/requirements.txt". If ffmpeg / ffprobe are missing, tell the user brew install ffmpeg and stop.

Merge details

python "$SKILL_DIR/scripts/merge_chunks.py" /abs/chunk1.mp3 /abs/chunk2.mp3 ...
# or:
python "$SKILL_DIR/scripts/merge_chunks.py" --dir /abs/folder

• Sorts inputs by filesystem mtime (filename timestamps are unreliable).
• Inserts 0.5 s silence between chunks (none at start/end).
• Uses ffmpeg's concat filter with explicit aformat normalization so mismatched sample rate / channel count between chunks is handled inside the graph (the older concat demuxer approach broke on this).
• Re-encodes seam-clean: libmp3lame -q:a 4 -ar 44100 mono.
• Default output = <source_folder>/<source_folder_name>.mp3. Use --output to override (recommended when chunks live in a generic subfolder like script/ — name the master after the episode).
• Source MP3s are never modified.

trim_pauses.py modes — reference

ratio, piecewise, power, fixed. Detection defaults noise=-30dB / d=0.3. The voice-preset workflow uses mode=ratio exclusively; the other modes exist for experimentation. See --help.

Output artefacts (per run)

In the source folder:

• <EP>.mp3 — merged master (preserved)
• <EP>.mp3.transcript.json — Whisper output
• <EP>.mp3.analysis.json — baseline WPM report
• <EP>_paused.mp3 — pause-trimmed intermediate (optional to keep)
• <EP>_voice-<slug>.mp3 — final voice-ready file (the deliverable)

For unknown-voice calibration:

• _calibration_<slug>/ — 8 variant MP3s + calibration.json

What you do NOT do

• Do NOT modify the merged master MP3 in place — every transformation goes to a new file.
• Do NOT delete or overwrite source ElevenLabs chunks.
• Do NOT re-transcribe an audio that already has <audio>.transcript.json.
• Do NOT pick the calibration winner yourself — the user listens and decides.
• Do NOT silently change a registered voice's preset. If the user wants to re-tune, run a fresh calibration sweep and replace the entry explicitly.
• Do NOT add unknown variants to the calibration sweep — the grid is fixed at 0.40/0.45 × {none, 185, 187, 190}.