| name | voice-note-to-midi |
| description | Convert voice notes, humming, and melodic audio recordings to quantized MIDI files using ML-based pitch detection and intelligent post-processing |
| author | Clawd |
| tags | ["audio","midi","music","transcription","machine-learning"] |
🎵 Voice Note to MIDI
Transform your voice memos, humming, and melodic recordings into clean, quantized MIDI files ready for your DAW.
What It Does
This skill provides a complete audio-to-MIDI conversion pipeline that:
- Stem Separation - Uses HPSS (Harmonic-Percussive Source Separation) to isolate melodic content from drums, noise, and background sounds
- ML-Powered Pitch Detection - Leverages Spotify's Basic Pitch model for accurate fundamental frequency extraction
- Key Detection - Automatically detects the musical key of your recording using Krumhansl-Kessler key profiles
- Intelligent Quantization - Snaps notes to a configurable timing grid with optional key-aware pitch correction
- Post-Processing - Applies octave pruning, overlap-based harmonic removal, and legato note merging for clean output
Pipeline Architecture
Audio Input (WAV/M4A/MP3)
↓
┌─────────────────────────────────────┐
│ Step 1: Stem Separation (HPSS) │
│ - Isolate harmonic content │
│ - Remove drums/percussion │
│ - Noise gating │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────┐
│ Step 2: Pitch Detection │
│ - Basic Pitch ML model (Spotify) │
│ - Polyphonic note detection │
│ - Onset/offset estimation │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────┐
│ Step 3: Analysis │
│ - Pitch class distribution │
│ - Key detection │
│ - Dominant note identification │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────┐
│ Step 4: Quantization & Cleanup │
│ - Timing grid snap │
│ - Key-aware pitch correction │
│ - Octave pruning (harmonic removal) │
│ - Overlap-based pruning │
│ - Note merging (legato) │
│ - Velocity normalization │
└─────────────────────────────────────┘
↓
MIDI Output (Standard MIDI File)
Setup
Prerequisites
- Python 3.11+ (Python 3.14+ recommended)
- FFmpeg (for audio format support)
- pip
Installation
Quick Install (Recommended):
cd /path/to/voice-note-to-midi
./setup.sh
This automated script will:
- Check Python 3.11+ is installed
- Create the
~/melody-pipeline directory
- Set up the virtual environment
- Install all dependencies (basic-pitch, librosa, music21, etc.)
- Download and configure the hum2midi script
- Add melody-pipeline to your PATH
Manual Install:
If you prefer manual setup:
mkdir -p ~/melody-pipeline
cd ~/melody-pipeline
python3 -m venv venv-bp
source venv-bp/bin/activate
pip install basic-pitch librosa soundfile mido music21
chmod +x ~/melody-pipeline/hum2midi
- Add to your PATH (optional):
echo 'export PATH="$HOME/melody-pipeline:$PATH"' >> ~/.bashrc
source ~/.bashrc
Verify Installation
cd ~/melody-pipeline
./hum2midi --help
Usage
Basic Usage
Convert a voice memo to MIDI:
./hum2midi my_humming.wav
This creates my_humming.mid with 16th-note quantization.
Specify Output File
./hum2midi input.wav output.mid
Command-Line Options
| Option | Description | Default |
|---|
--grid <value> | Quantization grid: 1/4, 1/8, 1/16, 1/32 | 1/16 |
--min-note <ms> | Minimum note duration in milliseconds | 50 |
--no-quantize | Skip quantization (output raw Basic Pitch MIDI) | disabled |
--key-aware | Enable key-aware pitch correction | disabled |
--no-analysis | Skip pitch analysis and key detection | disabled |
Usage Examples
Quantize to eighth notes
./hum2midi melody.wav --grid 1/8
Key-aware quantization (recommended for tonal music)
./hum2midi song.wav --key-aware
Require longer minimum notes
./hum2midi humming.wav --min-note 100
Skip analysis for faster processing
./hum2midi quick.wav --no-analysis
Combine options
./hum2midi recording.wav output.mid --grid 1/8 --key-aware --min-note 80
Processing MIDI Input
You can also process existing MIDI files through the quantization pipeline:
./hum2midi input.mid output.mid --grid 1/16 --key-aware
This skips the audio processing steps and goes directly to analysis and quantization.
Sample Output
═══════════════════════════════════════════════════════════════
hum2midi - Melody-to-MIDI Pipeline (Basic Pitch Edition)
[Key-Aware Mode Enabled]
═══════════════════════════════════════════════════════════════
Input: my_humming.wav
Output: my_humming.mid
→ Step 1: Stem Separation (HPSS)
Isolating melodic content...
Loaded: 5.23s @ 44100Hz
✓ Melody stem extracted → 5.23s
→ Step 2: Audio-to-MIDI Conversion (Basic Pitch)
Running Spotify's Basic Pitch ML model on melody stem...
✓ Raw MIDI generated (Basic Pitch)
→ Step 3: Pitch Analysis & Key Detection
Notes detected: 42 total, 7 unique
Note range: C3 - G4
Pitch classes: C3, E3, G3, A3, C4, D4, G4
Dominant note: G3 (23.8% of notes)
Detected key: G major
→ Step 4: Quantization & Cleanup
Octave pruning: removed 3 harmonic notes above 67 (median+12)
Overlap pruning: removed 2 harmonic notes at overlapping positions
Note merging: merged 5 staccato chunks into legato notes (gap<=60 ticks)
Grid: 240 ticks (1/16)
Notes: 38 notes
Key: G major
Key-aware: 2 notes corrected to scale
Tempo: 120 BPM
✓ Quantized MIDI saved
═══════════════════════════════════════════════════════════════
✓ Done! Output: my_humming.mid
═══════════════════════════════════════════════════════════════
📊 ANALYSIS SUMMARY
─────────────────────────────────────────────────────────────
Detected Notes: C3, E3, G3, A3, C4, D4, G4
Detected Key: G major
Quantization: Key-aware mode (notes snapped to scale)
MIDI Info: 38 notes, 7 unique pitches, 120 BPM
Pitches: C3, E3, G3, A3, C4, D4, G4
Notes & Limitations
Audio Quality Matters
- Clear, loud melody produces the best results
- Background noise can cause false note detection
- Reverb and effects may confuse pitch detection
- Close-mic'd vocals work significantly better than room recordings
Musical Considerations
- Monophonic sources work best (single melody line)
- Polyphonic audio (chords, multiple instruments) will produce messy results
- Vibrato and pitch bends may be quantized to stepped pitches
- Rapid note passages may be missed or merged
Technical Limitations
- Tempo is fixed at 120 BPM in output (time positions are preserved, but tempo may need adjustment in your DAW)
- Note velocities are normalized but may need manual adjustment
- Very short notes (<50ms) may be filtered out by default
- Extreme pitch ranges may cause octave detection issues
Post-Processing Recommendations
After generating MIDI, you may want to:
- Import into your DAW and adjust tempo to match your original recording
- Quantize further if stricter timing is needed
- Adjust note velocities for dynamics
- Apply swing/groove templates if the rigid grid sounds too mechanical
- Edit individual notes that were misdetected (common with fast runs)
Supported Audio Formats
Input formats supported via FFmpeg:
- WAV, AIFF, FLAC (uncompressed, best quality)
- MP3, M4A, AAC (compressed, acceptable)
- OGG, OPUS (open source formats)
- Most other formats FFmpeg supports
Troubleshooting
No notes detected
- Check that input file isn't silent or corrupted
- Try increasing
--min-note threshold
- Verify audio has clear melodic content (not just noise)
Too many notes / messy output
- Enable octave pruning and overlap pruning (on by default)
- Use
--key-aware to constrain to musical scale
- Check for background noise in source audio
Wrong key detected
- Key detection works best with at least 8-10 measures of music
- Chromatic passages may confuse the detector
- Manually review and adjust in your DAW if needed
Notes in wrong octave
- Basic Pitch sometimes detects harmonics instead of fundamentals
- The pipeline includes pruning, but some may slip through
- Use your DAW's transpose function for simple octave shifts
References
License
This skill integrates Basic Pitch by Spotify, which is licensed under Apache 2.0. The pipeline script and documentation are provided under MIT license.
