diagnostic-stem-delivery

# Diagnostic Stem Audio Production Workflow This skill provides a resilient pattern for audio production that emphasizes **diagnostic analysis before editing**, **explicit timecode extraction from documents**, **incremental verification**, **fail-fast principles**, and **mandatory deliverable verification**. Each major step produces verified outputs before proceeding, with comprehensive audio diagnostics at specified timecodes. ## Overview Follow these steps in strict order. Each step must complete successfully and pass verification before proceeding to the next: 1. **Parse timecodes from source documents** - Extract edit spots/timecodes from DOCX/text sources 2. **Perform diagnostic audio analysis** - Analyze reference audio at each timecode (pitch, clicks, frequency) 3. **Calculate timing parameters** - Derive section transitions from BPM and duration 4. **Verify reference audio** - Validate input file properties and extract target duration 5. **Generate and verify each stem individually** - One stem at a time with immediate verification 6. **Detect and resolve duration mismatches** - Apply appropriate extension strategy 7. **Apply edits based on diagnostics** - Make informed edits using analysis results 8. **Mix with verification** - Combine stems and verify mix integrity 9. **Export and verify deliverable** - Generate final output with comprehensive checks ## Key Principles - **Diagnostics first**: Analyze audio at edit points BEFORE making any changes - **Document-driven**: Parse timecodes directly from source documents (DOCX, TXT) - **Incremental verification**: Verify each stem immediately after generation - **Fail-fast approach**: Stop and report errors at each step - **Mandatory export**: Final step MUST produce verified deliverable file - **Tool reliability**: Use run_shell with inline Python for audio processing (avoid execute_code_sandbox for audio) ## Step 0: Parse Timecodes from Source Documents Extract edit spots and timecodes from document sources. Use python-docx via run_shell for reliable DOCX parsing: ```bash # Parse DOCX file for timecodes and edit spots python3 -c " from docx import Document import re import sys doc_path = sys.argv[1] if len(sys.argv) > 1 else 'Bass Edit Spots.docx' doc = Document(doc_path) edit_spots = [] timecode_pattern = r'(\d{1,2}:?\d{2}:?\d{2}[.:\d]*)|(\d+[.:\d]+)s' for para in doc.paragraphs: text = para.text.strip() if not text: continue # Look for timecodes in various formats matches = re.findall(timecode_pattern, text, re.IGNORECASE) if matches: for match in matches: timecode = match[0] if match[0] else match[1] if timecode: edit_spots.append({'timecode': timecode, 'context': text[:100]}) # Also check tables for table in doc.tables: for row in table.rows: for cell in row.cells: cell_text = cell.text.strip() matches = re.findall(timecode_pattern, cell_text, re.IGNORECASE) for match in matches: timecode = match[0] if match[0] else match[1] if timecode: edit_spots.append({'timecode': timecode, 'context': cell_text[:100]}) print(f'Found {len(edit_spots)} edit spots:') for i, spot in enumerate(edit_spots, 1): print(f'{i}. {spot[\"timecode\"]} - {spot[\"context\"][:50]}...') " ``` ## Step 1: Perform Diagnostic Audio Analysis at Timecodes Before any editing, analyze the reference audio at each identified timecode: ```python import numpy as np import soundfile as sf import librosa def analyze_audio_at_timecode(filepath, timecode_str, sample_rate=48000): """ Perform comprehensive diagnostic analysis at a specific timecode. Returns dict with: - pitch_estimate: Dominant frequency/pitch - click_pop_score: Likelihood of clicks/pops (0-1, higher = more likely) - frequency_spectrum: Dominant frequency bands - amplitude: RMS amplitude at timecode - issues: List of detected issues """ # Parse timecode to seconds timecode_str = timecode_str.replace(':', '.').strip() if 's' in timecode_str: timecode_str = timecode_str.replace('s', '') try: parts = timecode_str.split('.') if len(parts) == 3: seconds = int(parts[0]) * 3600 + int(parts[1]) * 60 + float(parts[2]) elif len(parts) == 2: seconds = int(parts[0]) * 60 + float(parts[1]) else: seconds = float(parts[0]) except: return {'error': f'Invalid timecode format: {timecode_str}'} # Load audio data, sr = sf.read(filepath) if sr != sample_rate: data = librosa.resample(data, orig_sr=sr, target_sr=sample_rate) sr = sample_rate # Extract window around timecode (±50ms for analysis) window_samples = int(0.1 * sample_rate) # 100ms window start_sample = max(0, int(seconds * sample_rate) - window_samples // 2) end_sample = min(len(data), start_sample + window_samples) window = data[start_sample:end_sample] if len(window) < 100: return {'error': 'Window too short for analysis'} # Pitch detection (using autocorrelation for monophonic content) def estimate_pitch(signal, sr): # Simple autocorrelation-based pitch detection signal = signal - np.mean(signal) # DC removal autocorr = np.correlate(signal, signal, mode='full') autocorr = autocorr[len(autocorr)//2:] # Find first significant peak after zero lag for i in range(1, min(len(autocorr) // 2, int(sr / 50))): if autocorr[i] > 0.3 * autocorr[0]: for j in range(i + 1, min(len(autocorr), int(sr / 20))): if autocorr[j] > autocorr[i]: period = j freq = sr / period return freq return None pitch = estimate_pitch(window, sr) # Click/pop detection (sudden amplitude changes) def detect_clicks(signal): diff = np.diff(np.abs(signal)) threshold = 5 * np.std(diff) click_positions = np.where(np.abs(diff) > threshold)[0] click_score = min(1.0, len(click_positions) / len(signal) * 1000) return click_score, click_positions click_score, click_positions = detect_clicks(window) # Frequency analysis spectrum = np.abs(np.fft.rfft(window)) freqs = np.fft.rfftfreq(len(window), 1/sr) dominant_freqs = [] for band in [(20, 200, 'sub'), (200, 2000, 'mid'), (2000, 20000, 'high')]: mask = (freqs >= band[0]) & (freqs < band[1]) if np.any(mask): band_power = np.sum(spectrum[mask]) dominant_freqs.append({'range': f'{band[0]}-{band[1]}Hz', 'power': float(band_power), 'label': band[2]}) dominant_freqs.sort(key=lambda x: x['power'], reverse=True) # Amplitude rms = np.sqrt(np.mean(window ** 2)) # Detect issues issues = [] if click_score > 0.3: issues.append(f'High click/pop probability ({click_score:.2f})') if rms < 0.001: issues.append('Near-silence detected') if rms > 0.9: issues.append('Potential clipping') if pitch and pitch < 40: issues.append(f'Very low frequency content ({pitch:.1f}Hz)') return { 'timecode': timecode_str, 'seconds': seconds, 'pitch_hz': pitch, 'click_pop_score': click_score, 'frequency_spectrum': dominant_freqs[:3], 'amplitude_rms': float(rms), 'issues': issues, 'window_length': len(window) } # Analyze all edit spots # edit_spots from Step 0 for i, spot in enumerate(edit_spots): print(f'\\n=== Analyzing edit spot {i+1}: {spot["timecode"]} ===') analysis = analyze_audio_at_timecode('reference.wav', spot['timecode']) if 'error' in analysis: print(f'ERROR: {analysis["error"]}') else: print(f'Pitch: {analysis["pitch_hz"]} Hz' if analysis["pitch_hz"] else 'Pitch: N/A (complex/noisy)') print(f'Click/Pop Score: {analysis["click_pop_score"]:.3f} (0=none, 1=certain)') print(f'Amplitude (RMS): {analysis["amplitude_rms"]:.6f}') if analysis['issues']: print(f'Issues: {", ".join(analysis["issues"])}') for freq in analysis['frequency_spectrum']: print(f' {freq["label"]} band ({freq["range"]}): power={freq["power"]:.2f}') ``` ## Step 2: Calculate Timing Parameters (Early) Calculate all timing parameters **before** generating any audio: ```python def calculate_section_transitions(bpm, total_duration_sec, sections): """Calculate beat-aligned transition points for song sections.""" beats_per_second = bpm / 60.0 section_durations = {} cumulative_time = 0 for section_name, beat_count in sections.items(): duration = beat_count / beats_per_second section_durations[section_name] = { 'start': cumulative_time, 'end': cumulative_time + duration, 'beats': beat_count, 'start_beat': cumulative_time * beats_per_second } cumulative_time += duration return section_durations # Configuration BPM = 120 DURATION = 137 SECTIONS = {'intro': 16, 'verse': 32, 'chorus': 32, 'bridge': 16, 'outro': 16} timing = calculate_section_transitions(BPM, DURATION, SECTIONS) print('Timing calculated:') for section, data in timing.items(): print(f' {section}: {data["start"]:.2f}s - {data["end"]:.2f}s ({data["beats"]} beats)') ``` ## Step 3: Verify Reference Audio Validate the reference file exists and has expected properties: ```python import soundfile as sf import os def verify_reference_file(filepath, expected_sample_rate=None, min_duration=None): """Verify reference audio file and return info dict.""" if not os.path.exists(filepath): raise FileNotFoundError(f'Reference file not found: {filepath}') info = sf.info(filepath) errors = [] if expected_sample_rate and info.samplerate != expected_sample_rate: errors.append(f'Sample rate mismatch: expected {expected_sample_rate}, got {info.samplerate}') if min_duration and info.duration < min_duration: errors.append(f'Duration too short: expected >= {min_duration}s, got {info.duration}s') if errors: raise ValueError(f'Reference file validation failed: {"; ".join(errors)}') print(f'Reference verified: {info.duration:.2f}s @ {info.samplerate}Hz, {info.channels}ch, {info.subtype}') return { 'sample_rate': info.samplerate, 'duration': info.duration, 'channels': info.channels, 'subtype': info.subtype } # Verify reference ref_info = verify_reference_file('reference.wav', expected_sample_rate=48000, min_duration=130) TARGET_DURATION = ref_info['duration'] # Use actual reference duration as target ``` ## Step 4: Generate and Verify Each Stem Individually Generate one stem at a time, verify it immediately before proceeding to the next: ```python import numpy as np def generate_stem(name, duration_sec, sample_rate, subtype='FLOAT', section_timing=None): """Generate a single stem with explicit sample type.""" frames = int(duration_sec * sample_rate) t = np.linspace(0, duration_sec, frames) # Generate stem-specific content (customize per stem type) if name == 'bass': freq = 110 # A2 audio_data = np.sin(2 * np.pi * freq * t) * 0.8 elif name == 'guitars': freq = 440 # A4 audio_data = np.sin(2 * np.pi * freq * t) * 0.6 elif name == 'synths': freq = 880 # A5 audio_data = np.sin(2 * np.pi * freq * t) * 0.5 elif name == 'bridge': freq = 220 # A3 audio_data = np.sin(2 * np.pi * freq * t) * 0.7 else: audio_data = np.sin(2 * np.pi * 440 * t) * 0.5 # Ensure proper data type if subtype == 'FLOAT': audio_data = audio_data.astype(np.float32) elif subtype == 'PCM_24': audio_data = np.clip(audio_data, -1, 1) * (2**23 - 1) audio_data = audio_data.astype(np.int32) filepath = f'{name}_stem.wav' sf.write(filepath, audio_data, sample_rate, subtype=subtype, format='WAV') return filepath, audio_data def verify_stem(filepath, expected_sample_rate, expected_duration, tolerance_sec=1.0): """Verify a single stem meets specifications.""" if not os.path.exists(filepath):