mp3-to-vgm

name	mp3-to-vgm
description	Convert MP3/WAV/audio recordings into AY-3-8910 VGM chiptunes (.vgm/.vgz) playable by the MSX VGM Analyzer, and modify the converter itself. Use when the user wants to turn audio into a VGM/chiptune file, transcribe music to the AY-3-8910/YM2149 PSG, tune the conversion (voices, noise, gain, fps, stem separation), debug bad output, or extend the mp3_to_vgm package or its VGM/AY register handling.

MP3 → AY-3-8910 VGM

What this does (read this to the user)

Give it a song (MP3, WAV, …) and it produces a chiptune — a .vgm/.vgz file that plays on the General Instrument AY-3-8910/YM2149 PSG, the sound chip in the MSX, ZX Spectrum, Atari ST and many arcade boards. Load the result in the browser-based MSX VGM Analyzer to hear and inspect it.

Two quality levels:

• Fast (default) — analyses the raw mix and grabs the loudest notes each moment. Quick, no heavy dependencies. Good for sparse, melodic material.
• Best (--separate) — first splits the song into vocals / bass / other / drums stems with Demucs, then gives each stem its own chip voice (vocals→lead, other→harmony, bass→bass voice, drums→noise). Dramatically cleaner on full mixes. Costs ~10–60 s of separation per track (cached afterwards) and a one-time Demucs/PyTorch install.

Set expectations honestly: a .vgm is not audio (see Core fact). Even at best quality the chip has only 3 tone voices + 1 noise channel, so lush arrangements thin out. Tell the user roughly how their material will fare.

Core fact (do not lose sight of this)

A .vgm file is not audio — it is a timed log of register writes to the sound chip. "Converting MP3 to VGM" is therefore lossy automatic music transcription: full polyphonic audio is reduced to 3 square waves + 1 noise generator (16 volume levels each). Sparse, melodic material transcribes well; dense mixes and vocals come out approximate even with stem separation.

Where the code lives

The converter is a single Python package at the repository root: mp3_to_vgm/. This skill is just documentation that drives it — there is no copy of the code under the skill directory. Run every command from the repo root (where mp3_to_vgm/ and requirements.txt are), so python -m mp3_to_vgm.cli resolves.

Setup

pip install -r requirements.txt   # numpy, librosa, soundfile, demucs (pulls torch)
# ffmpeg must be on PATH (macOS: brew install ffmpeg) — required to decode MP3

demucs is only needed for --separate; the fast path runs without it. Its first --separate run downloads the htdemucs model weights (~80 MB).

Converting

# fast path
python -m mp3_to_vgm.cli song.mp3                       # -> song.vgm and song.vgz
python -m mp3_to_vgm.cli song.mp3 out.vgm --self-check

# best quality: stem separation (one source per voice)
python -m mp3_to_vgm.cli song.mp3 out.vgm --separate --self-check

Hand out.vgm or out.vgz to the user to load at https://hrubix.github.io/VGM_Analyzer/. Always run with --self-check when producing a deliverable — it validates the header and command stream and parse-verifies the result (frame count, first audible note, noise frames).

Stem separation (--separate)

Splits the mix with Demucs (default model htdemucs) and maps stems to voices adaptively, logging the plan it chose:

• bass present → its own monophonic, low-clamped voice (channel C).
• vocals present → monophonic lead on the first free channel.
• other → fills the remaining tone channels polyphonically (the harmony).
• drums → noise generator, but only if drums are a real part of the mix (see noise gating below).

For an instrumental the vocals stem is near-silent, so the plan falls back to other covering lead + harmony across the front channels. Stems are cached in .stems_cache/<model>/<track>/ next to the output, so re-runs that only change --max-voices/--gain/--no-noise are instant (no re-separation).

Options: --demucs-model <name> (default htdemucs), --demucs-device mps|cpu|cuda (auto-detected if omitted).

Tuning (when output sounds wrong)

Symptom	Try
Muddy, full-mix sludge	--separate (biggest single quality win)
Too quiet / silent channels	--gain 2.0 (raises pre-quantisation volume)
Harmonics fighting for voices	--max-voices 1 or 2
Constant hiss / spurious drums	--no-noise (or check the drums-share log)
Missing drums	source too tonal, or drums <8% of mix so the noise gate muted them
Wrong pitch/timing region	use --fps 60 for NTSC timing

Defaults are MSX standard: clock 1789772 Hz, 50 fps (--clock, --fps).

Noise gating (why drums sometimes don't play)

Demucs always emits a drums stem; on a kit-less acoustic track it holds only breath/bow/bleed. Routing that to noise unconditionally produces a wall-to-wall hiss. So transcribe_stems gates noise three ways (constants at the top of transcribe.py): a global gate (drums must be ≥DRUMS_PRESENCE_FRAC of the whole-mix energy, else noise is off entirely) and a per-frame gate (drums must dominate the frame's energy, clear a broadband floor, and sit on an onset transient). The CLI logs drums share of mix = X%. If a user expects drums and gets none, check that number first.

Verifying output

--self-check does both jobs: it validates the header/stream structure and replays the command stream, printing a [verify] line with the frame count, channel A's first audible note (decoded from its tone period), and the noise-frame count — so you can confirm the result is musically sensible, not just well-formed. Mixer logic is the easy trap: in register R7, enable bits are active LOW. A channel's noise is ON when (R7 & 0x38) != 0x38, not when it equals 0. See reference.md for the standalone parsing snippet.

Modifying the converter

The pipeline is five stages — change the one that owns the behaviour:

• mp3_to_vgm/separate.py — (optional, --separate) Demucs split into 4 stem WAVs via get_model/apply_model + soundfile; disk-cached
• mp3_to_vgm/audio.py — decode + STFT framing (hop = sr/fps, one column/frame)
• mp3_to_vgm/transcribe.py — transcribe (mix: peak-pick + voice continuity + percussion→noise) and transcribe_stems (one source per voice + noise gating)
• mp3_to_vgm/ay.py — freq→period, amplitude→4-bit log volume, register delta model
• mp3_to_vgm/vgm.py — VGM v1.51 header + A0/63/66 command stream + .vgz

Both transcribe paths emit the same Frame list (defined in ay.py), so the RegisterFile/VgmWriter stages are shared and untouched by the stem work. For exact VGM header offsets, AY register map, conversion formulas, stem→voice mapping and the Demucs install gotchas, read reference.md before editing.

Workflow checklist

1. Confirm ffmpeg + deps are installed (demucs too if using --separate).
2. Pick the path: default for sparse/melodic, --separate for full mixes / "best quality" requests.
3. Run the converter (default params first).
4. Run --self-check; for a deliverable, read the parse-verify line — sane first note, and a noise-frame count that matches whether the song has drums.
5. If output is poor, tune per the table above before changing code.
6. Tell the user the exact .vgm/.vgz paths and the Analyzer URL.