split-audio-by-words

name	split-audio-by-words
description	Slice a long-form audio file (podcast, interview, talk) into per-scene mp3 splits using Whisper word-level timestamps so cuts land on clean word boundaries, never mid-syllable. Accepts explicit timestamp markers OR auto-detects scene breaks from long pauses. Emits per-scene mp3s with proper afade-out tails and a manifest.json mapping scenes to source timestamps + caption text.

split-audio-by-words

Purpose

Cutting a long-form podcast clip into per-scene mp3s with raw timestamps always lands on a mid-word boundary sometimes. The klarify scene-07 split clipped at source-49.2s but the word "stakes" started at 49.4s — the per-scene file ended at "...higher" and the rest of "stakes situations" was lost. We didn't notice for two iterations.

This atom solves it the right way: transcribe the source with word-level timestamps, then snap every requested cut point to the nearest word boundary (with operator-tunable preference for snapping before/after the boundary word).

Inputs

• <audio> — required. The long-form source mp3/m4a/wav.
• <scenes> — required. Either:
  • explicit: a JSON file or --scene flag with [{ "scene": 1, "caption": "...", "start": 15.1, "end": 18.1 }, ...]
  • auto-pause: pass --auto-split and an optional --min-pause-duration 0.6 (seconds). The atom uses ffmpeg silencedetect + Whisper word timing to identify sentence-ending pauses and emit one split per sentence.
• <provider> — groq (default — cheapest, fastest), fal, or openai. Composes atoms/source/transcribe-audio-fal for the fal path; uses inline Groq/OpenAI clients otherwise.
• <snap_mode> — nearest (default), before, or after. Controls which word boundary to snap to when a requested cut lands inside a word.
• <padding> — default 0.20s. Appended to each scene's end with afade=t=out:st=...:d=0.20 so the cut tail fades smoothly instead of clipping abruptly.
• <output_dir> — default <audio_parent>/voiceovers/. Per-scene files written as scene-NN-<speaker?>.mp3.

Workflow

Step 1 — Transcribe with word timestamps

If <provider>=fal, compose atoms/source/transcribe-audio-fal (passing chunk_level=word). Otherwise call Groq Whisper or OpenAI Whisper inline with timestamp_granularities=[word]. Output: words.json with [{word, start, end}, ...].

This cache is reused for subsequent runs on the same source — re- transcribing wastes credits.

Step 2 — Snap cut points to word boundaries

For each requested (start, end) pair:

def snap(t, words, mode):
    """Snap timestamp t to nearest word boundary based on mode."""
    # Find word w such that w.start <= t < w.end
    inside = next((w for w in words if w["start"] <= t < w["end"]), None)
    if not inside:
        # t is in a silence — already on a clean boundary, no snap needed
        return t
    if mode == "before":
        return inside["start"]
    if mode == "after":
        return inside["end"]
    # nearest
    return inside["start"] if (t - inside["start"]) < (inside["end"] - t) else inside["end"]

Write snapped timestamps to the manifest with a snapped_from field so the operator can see what changed.

Step 3 — Extract per-scene mp3s

ffmpeg -y -ss "$snapped_start" -to "$snapped_end" \
  -i "$audio" \
  -af "afade=t=out:st=$((snapped_end - snapped_start - padding)):d=$padding" \
  "$output_dir/scene-$NN-$speaker.mp3"

The afade=t=out masks any tail noise and provides a clean audio-cut boundary downstream (especially important when the audio gets muxed under B-roll where there's no visual cue covering an abrupt cut).

Step 4 — Write manifest.json

{
  "source": "audio/source-practice-stakes-low-51s.mp3",
  "provider": "groq",
  "scenes": [
    {
      "scene": 1,
      "speaker": "TAMMER",
      "caption": "you don't start",
      "text": "You don't start practicing layups",
      "clip_start": 15.10,
      "clip_end": 18.06,
      "snapped_from": { "start": 15.10, "end": 18.10 },
      "duration": 2.96,
      "file": "scene-01-tammer.mp3"
    }
    ...
  

Step 5 — Verification report

For each scene, run word-level transcription on the extracted file (small, cheap) and verify:

• First word starts within the first 200ms
• Last word ends at least 200ms before the file end (i.e. the afade caught the tail)
• Transcribed text matches the manifest's text field within Levenshtein 0.9

Surface failures to stderr; operator decides whether to re-snap or accept.

Output

• <output_dir>/scene-NN-<speaker>.mp3 per scene
• <output_dir>/manifest.json
• <output_dir>/words.json (full source transcript — reusable for burn-in-captions later)
• <output_dir>/verification.md (per-scene transcribed-text check)

Quality checks

• Every scene's actual file duration matches clip_end - clip_start within ±20ms
• No two scenes overlap in source timestamps
• Each scene's afade tail is present (last ~200ms of audio amplitude decays smoothly to silence)
• Verification step confirms transcribed text aligns with manifest

When to use

• Splitting a podcast clip into per-scene VO files for lipsync (klarify pipeline)
• Re-cutting an existing master into scene-aligned chunks for B-roll experiments
• Inside molecules/podcast-clip-animated-ad (step 2)

Do NOT use for:

• General audio editing where word-boundary precision isn't needed (use raw ffmpeg -ss/-to)
• Splitting music tracks (this atom is speech-focused; for music use atoms/audio-editing/)

Failure modes

• Whisper misses a word — the snapped timestamp may be off by one word. Verification step catches this. Operator can pass explicit timestamps to override.
• Long source with many speakers — Whisper's word stream blends them; pass --speaker-diarization to use a smarter provider (currently unsupported — flagged for future work).
• Auto-split detects too many micro-pauses — increase --min-pause-duration from 0.6s to 1.0s or 1.5s.

Defaults rationale

Knob	Default	Why
provider	groq	$0.001/min — cheapest, fastest
snap_mode	nearest	Minimizes timestamp drift from operator intent
padding	0.20s	Long enough for an inaudible afade-out, short enough not to clip the next scene's first word
min-pause-duration	0.6s	Catches sentence boundaries while ignoring breath pauses

Example

python3 .../skills/atoms/source/split-audio-by-words/scripts/split.py \
  --audio audio/source-practice-stakes-low-51s.mp3 \
  --scenes-json scenes.json \
  --output-dir voiceovers/ \
  --provider groq \
  --padding 0.2

Where scenes.json is:

[
  {"scene": 1, "speaker": "TAMMER", "start": 15.10, "end": 18.10, "caption": "you don't start", "text": "You don't start practicing layups"},
  {"scene": 2, "speaker": "TAMMER", "start": 18.10, "end": 23.60, "caption": "when the game starts", "text": "and basketball, when the game starts because the pressure is a little bit higher."}
]

Output:

• voiceovers/scene-01-tammer.mp3 (2.96s, snapped from operator's 3.0)
• voiceovers/scene-02-tammer.mp3 (5.50s)
• voiceovers/manifest.json
• voiceovers/words.json
• voiceovers/verification.md

Implementation note

Script at scripts/split.py. Composes atoms/source/transcribe-audio-fal when provider=fal; otherwise calls Groq/OpenAI directly. Caches the full-source words.json in the output dir so a re-run with different scene boundaries doesn't re-transcribe.

The companion atom atoms/review/review-transcript-integrity should be run on the final master to catch any per-scene splits that ended up truncating despite the snap (rare but possible if Whisper missed the boundary word).