gloop-grind

name	gloop-grind
version	1.0.0
description	The long-march roadmap for GLOOPY — the compact JUCE 9 / C++17 linear-arranger DAW at ~/git/gloopy (plugin hosting, composition-as-code projects, OSC + gRPC control, automation, embedded sfizz, scriptable/headless workflows). Invoke when the user says "grind", "keep going on Gloopy", "what's next", "pick the next feature", or wants to continue the roadmap across sessions. Loads the north star, the principles that judge each slice, the current shipped state, the working discipline (build/test/commit/verify-headless), and the full ordered backlog of features we have actually committed to building. NOTE: this is the *Gloopy* grind (project-local). A separate global `grind` skill belongs to the Plume Lisp project — don't confuse them.
allowed-tools	["Bash","Read","Write","Edit","Glob","Grep","Agent"]

grind — the march to make Gloopy the scriptable, composition-as-code DAW

This skill is the durable map and process for Gloopy (~/git/gloopy): a compact JUCE 9 / C++17 linear-arranger DAW that hosts VST3/LV2 plugins and an embedded sfizz SFZ engine, and that is driven end-to-end by a control API (OSC UDP 9000 + gRPC 127.0.0.1:50051) over projects stored as a diff-friendly composition-as-code text format.

Three living documents, in order of authority:

• AGENTS.md (repo root) — the hard-won landmines: proto3-omits-zero, one instance at a time (pkill -x Gloopy), NewProject resets to the 5-track kit, manual level staging, the sfizz vendoring, sample-path resolution. Read it every session; it is the operational ground truth.
• docs/ROADMAP.md + ideas.md — the design record of what shipped (Phases 1–4, composition, recording, presets) and the raw idea pool (14 Feature Ideas + 16 Ardour borrowings). This skill is the curated subset of that pool that we will actually build, in order. ideas.md is the superset; this file is the commitment.
• This skill — the shape of the journey and how to walk it. Stable. The volatile per-feature detail lives in docs/ROADMAP.md and the git log.

The north star (what Gloopy is trying to be)

The DAW you can drive from a script and store in git. Not a full DAW with an API bolted on — a DAW whose primary surface is the composition-as-code text format and the OSC/gRPC control API, with the GUI as one client among Python, Common Lisp, CI, and future AI agents. Every capability is reachable and verifiable headlessly: grpcurl builds it, --render bounces it, a script asserts the WAV. Gloopy stays smaller, more scriptable, and more composition-repo friendly than a typical DAW — that constraint is the product, not a limitation.

The principles (how every slice is judged)

1. API-first, GUI-second — but GUI is not optional. A feature isn't done until it's reachable over gRPC (and OSC where it's a live knob) with a stable id. The UI is a client of that surface, built after — and never the only way to do the thing. If it can't be scripted, it isn't Gloopy-shaped. AND, per explicit user direction (2026-07): everything we add must have a desktop UI where appropriate — a control, menu item, mouse gesture, or shortcut a desktop user can actually reach. A slice is NOT done at "backend + gRPC + smoke green"; it also needs the desktop affordance, wiring the same shared transform, screenshot-validated on Xvfb. Do not ship API-only features (the chord tool was caught being grpcurl-only). "Where appropriate" excludes pure non-visual plumbing (format internals, scan/validate CLI).
2. Everything lives in the composition text format. New state serialises to readable TOML / .notes / .points under the content-addressed dirty-write model (Source/Composition.cpp), so it diffs and travels. No new hidden project blobs. A dir → runtime → dir round-trip must stay byte-stable.
3. Every slice is headless-verifiable. Prove it with grpcurl + --render + a WAV/text assertion, wired into tests/smoke.sh where it fits. "I built the UI, it looks right" is not proof. Favour work whose correctness a script can see.
4. The audio thread is sacred. No allocation, locks, file I/O, or blocking in renderBlock. New realtime data reaches the audio thread lock-free or under the engineLock try-lock; structural edits go on the message thread (callOnMessageThread, pushUndoSnapshot() first). Build the object off-thread, then swap it in under the lock — the presets/instrument-swap pattern.
5. Stay curated, not comprehensive. Gloopy hosts VST3/LV2, so built-ins are a small high-value set, not a catalog. Borrow systems and workflows from bigger DAWs (Ardour, trackers) — never their code or assets without a license review against Gloopy's AGPL-3.0 (Ardour is GPL; direct copying needs review). Keep the JUCE layout simple; don't grow a window-heavy workspace or a per-device C++ subsystem for every controller — keep mappings data-driven. Don't embed a second scripting VM (no Lua): scripts drive the existing gRPC API. Never name other named DAWs/trackers as the source in code, docs, or commits — borrow the idea, not the brand.

Current state (verify against git log + docs/ROADMAP.md each session)

[x] done · [~] partial · [ ] not started

• [x] Phases 1–4 (docs/ROADMAP.md): MIDI recording into clips, master limiter + clip meters, mixer-insert control API, MIDI hot-plug, undo/redo (snapshot-based), parameter automation (breakpoint lanes, SetAutomation/GetAutomation), structural change events on Subscribe, region/stem render (RenderToFile + range/target), Python client + CL client, expanded dual-osc + resonant filter + LFO synth, groove/swing + piano-roll velocity editing, unit tests (GloopyTests, ctest) + smoke test + CI under xvfb.
• [x] Composition-as-repo format (Source/Composition.cpp, gloop-compositions.md): TOML manifests + .notes/.points + WAV/plugin sidecars; content-addressed dirty writes; zip read; .gloopy / folder / .zip all first-class in the File menu; SaveComposition/LoadComposition RPCs.
• [x] Audio recording (Source/Recording.cpp, recording.md): live-input capture to a WAV take via ThreadedWriter; referencing audio clips (Clip.audioFile/takeId); monitoring, count-in, punch, take lanes / take management, loop recording; GLOOPY_REC_TEST_TONE_HZ test seam; SetLoop RPC.
• [x] Embedded sfizz (Source/SfizzGenerator.h, third_party/sfizz/, BSD-2, vendored no-submodule): native loadSfzFile/renderBlock, disk-streaming, portable sample-path resolution, freewheeling for offline bounce.
• [~] Presets (Source/Presets.cpp): synth, effect-chain, and general instrument (synth or SFZ, cross-type) presets as composition-friendly TOML under <userAppData>/Gloopy/presets/ (GLOOPY_PRESET_PATH override); ListPresets + save/load RPCs + Python client. Remaining: sampler + plugin instrument presets, plugin effects inside effect-chain presets, and all preset UI (menus, browser, New-From-Template). These fold into Wave 6 items #16 below.

So the model layers (tracks, mixer inserts, effects, automation, plugins, sfizz, composition I/O, recording, presets, session-launch grid, group/submix routing, the OSC/gRPC surface) all exist. The backlog below is about depth, musical model, analysis, product surface, and the session-view/engine tails — not foundational plumbing.

The working discipline (the meta-process that works — follow it)

• Read AGENTS.md and git log --oneline -15 first. The landmines there are non-negotiable; re-learning them costs real time.
• One slice = one feature = one commit. Pick the next unchecked backlog item. Implement the smallest coherent version that is API-reachable and headless-provable, not the whole grand vision at once.
• The full stack of a slice (most features touch all of these — miss one and it's half-built):
  1. Model / audio — a new Source/<Feature>.cpp (or extend an existing one); realtime data lock-free / under engineLock, structural ops via callOnMessageThread + pushUndoSnapshot().
  2. Serialisation — extend toValueTree/loadFromTree and the composition mapping in Source/Composition.cpp (readable TOML / .points), preserving the dirty-write round-trip. Add to tests/smoke.sh's round-trip check.
  3. proto + gRPC — add messages/RPCs to proto/gloopy.proto (proto3; --experimental_allow_proto3_optional), implement the override in Source/GrpcServer.cpp (the main.apiX(...) → set_ok/set_error pattern, js() for std::string→juce::String). Remember proto3 omits zero/false — always .get('x', 0), and use OSC for "set to 0 / turn a bool off".
  4. OSC — add a /gloopy/... lane for anything that's a live knob (OscControl.h).
  5. Clients — regenerate stubs (python/gen.sh) and add thin wrappers to python/gloopy/client.py (and the CL client where it earns it).
  6. Register the source in CMakeLists.txt target_sources if it's a new .cpp.
  7. Document it — every new feature/capability lands in the user-facing manual (docs/, Material for MkDocs) as part of the slice, not later: what it does and how to use it, in the right Diátaxis bucket (User guide for musicians, Control & scripting for integrators). Document the domain model in and link to it — don't re-explain nouns per language. Keep it out of implementation-language framing (describe what it does, not "the C++/JUCE…"). must stay green. This is not optional: a slice that isn't in the manual is half-built, same as one missing its gRPC or desktop surface.

The backlog — the real list we're building, ordered

The through-line: make the scriptable composition-as-code surface deep and complete before making it pretty. Waves are roughly sequential; within a wave, order by leverage. Each item is a slice: model+serialise+proto+client → headless proof → commit. ✦ marks a design fork worth a prior-art check first. Effort: S ≈ hours, M ≈ a day or two, L ≈ a week+.

★ Reprioritized near-term order (updated 2026-07-28) — read this first

Directive (user, 2026-07-28): do Waves 9 → 10 → 11 FIRST, then go back to the beginning. Those three are the cross-cutting reach & infrastructure epics — version control, platform, agent control — that widen who and what can drive Gloopy and refresh the foundation. Front-load them, then return to feature depth (the Phase-B tails below, which pick up the pre-existing near-term queue).

Already shipped since the 2026-07-26 plan (dropped from the queue; see "Current state"): the Surge XT featured-synth front (DONE/paused — hosted plugin default + bundled presets; removal blocked by the .fxp-into-hosted-plugin blocker, docs/surge-embed.md), the Session view epic (slices 1–4), the grouping / submix-routing epic (complete), the mixer channel model, and the UI elevation-over-borders pass. Also DONE: undo/redo (already existed — snapshot stack, Ctrl+Z/Y, commit c3eedf0; the fine-grained Wave 7 #22 rework stays optional/deferred).

Phase A — do these now, in this order:

1. Wave 9 #30 — Git as project management. IDE-grade source control (status, commit, commit-graph, branches, tags, checkout any version, diff, stash, remote/push); shell out to the system git. Central to project management + realizes the "store it in git" half of the north star. XL, 12 slices — start at slice 1 (availability + status) and walk them in order.
2. Wave 10 — build & platform. #31 upgrade JUCE off the 8.0.15 FetchContent pin (low-risk; do it before the port so Windows builds on the latest) → #32 Windows build (CI-native recommended, else mingw-w64 cross-compile).
3. Wave 11 #33 — MCP stdio service. AI agents drive Gloopy over stdio; a thin gRPC adapter — the cheapest of the three (pure adapter code over the existing control API).

Phase B — then go back to the beginning (resume the pre-existing near-term queue, in order): 4. Strong time types (Wave 7 #21) — foundation landed (Source/Time.h); NEXT = incremental adoption: migrate the beatToSamples/samplesToBeats layer + clip/loop/marker positions onto the types, add a Sample domain, one subsystem per slice. Before more timeline/tempo work. 5. AudioBufferPool (Wave 7 #24) — not started; kill audio-thread allocations; cheap, and a prerequisite for any graph rework. 6. Finish the Session view epic — slices 5 (control API: OSC/gRPC launch clip/scene, stop, query session state) and 6 (polish: clip colours, follow-actions, capture, quantize menu). 7. Browser sidebar tail (Wave 6 #16: Favorites tab + first-class drag-and-drop onto tracks/inserts — Templates/Demos/Plugins/Samples/Presets tabs already landed), then the rest of Waves 2–5.

The heavy engine items (off-thread graph swap #5, multicore rendering #6) stay deferred and gated on a real profiling trigger — do not start them until Gloopy is measurably single-core-bound on a heavy multi-track session. See Wave 7 for why.

Wave 1 — Keystone model work (everything downstream rides on these)

1. Universal parameter model — ParamModel / ParameterRef. ✦ L Idea #3. One parameter abstraction: { stable id, name, min, max, default, scaling (linear|log|dB), unit, current value, thread policy }. Back UI knobs, automation targets, MIDI/OSC mapping, gRPC state, plugin generic editors, and save/load with it. Reconcile the existing TargetRef (automation/insert params) into it so ids are stable across composition files and external clients. Done when: ListParameters returns stable ids for track/insert/effect/plugin params; GetParameter/SetParameter work by id; an automation lane and an OSC map both address the same id; ids survive a composition round-trip. ✅ DONE (2026-07-25, focused session across 3 slices) — all four "done when" clauses met: ListParameters covers track/insert/effect/synth and plugin (VST3/LV2) params; Get/Set/SetParameterNormalized work by id; automation, the modulation matrix, and MIDI/ OSC controllers all address the same id via the shared applyParamValue; and the numeric track id is now serialised so track/<id>/... ids survive a composition round- trip. See the slice log below.

   • [~] Minimal slice landed (Source/Parameters.cpp, commit): flat string-id read/write layer — ListParameters/GetParameter/SetParameter RPCs + ParameterInfo{id,name,value,min,max,default_value,unit,scaling}, delegating to the existing thread-safe setters. Covers track/<id>/{volume,pan,mute,solo}, track/<id>/synth/<name>, insert/<i>/{volume,pan,mute,solo}, effect/<i>/<slot>/<param>. Proven via smoke.sh (102 params; cutoff set/get) + grpcurl.
   • [x] Automation-by-id + stable track ids landed (commit, focused session): an automation lane can now address a ParamModel id string (AutoLaneSnap.target), written each block through the same lock-held applyParamValue that controllers/modulation use — so an automation lane and a controller/LFO map address the same id (the keystone reconciliation). apiSetAutomationById(target,points) + apiAddAutomationPointById (the keyframe primitive) + / RPCs + Python; serialised on LANE + composition (). Legacy tuple lanes still work. the numeric track id is now serialised ( in TRACK + composition track TOML) and preserved on load ( keeps loaded ids, bumps past them), so param ids survive a composition round-trip. the mixer param right-click menu gains "Automate at playhead" / "Clear automation" — screenshot-validated. smoke proves a cutoff sweep on moves the render, the lane round-trips through SaveComposition/LoadComposition, and the RELOADED project still sweeps (stable id keeps the lane live).

Wave 2 — Turn the control surface into the product

3. Export profiles / named render targets. ✦ M Ardour #2 + Idea #9. Named export presets stored in composition text. Targets: full mix, named range (Wave 1 #2), selected tracks, stems, current clip, loop region. Delivery presets: WAV master, FLAC archive, streaming WAV, loop pack, stems. Default outputs to exports/, deterministic filenames. Done when: DefineExportPreset + RunExport(name) RPCs; a preset renders the right files to exports/ at correct levels; presets round-trip in the composition.

   • [x] Landed (Source/Exports.cpp, commit): ExportProfile{name,target, rangeName,format,trackId,tailSeconds} + DefineExportProfile(upsert)/ ListExportProfiles/RemoveExportProfile/RunExport RPCs. Targets mix (whole song), range (named location → window), track (solo), stems (one file per instrument track). Reuses apiRenderToFile + apiResolveRange; output to <project>/exports/ (gitignored) with deterministic <name>.wav / stems/<id>-<slug>.wav; returns the file list. Serialised in toValueTree/loadFromTree + composition exports.toml. Python client define/list/remove/run_export. smoke.sh: master mix renders a file + profile survives composition round-trip. Verified: 6 stems one-per-track, range export shorter than mix.
   • [x] FLAC export landed (commit): apiRenderToFile now picks the encoder from the output extension (.flac → juce::FlacAudioFormat, else WAV; both 24-bit), and apiRunExport chooses .flac/.wav from the profile's format field (which already existed). The forced-.wav line in the render path now only defaults unknown extensions. smoke.sh proves a FLAC mix is valid (fLaC magic + ffprobe codec=flac) and losslessly smaller than the WAV. Not yet: streaming/loop-pack/ current-clip targets, per-format bit depth, MP3/OGG (no encoder linked).

Wave 3 — Editing + routing depth (still API-first)

6. Clip / region operations. M Ardour #12. Split at playhead / at marker, clip gain, normalize, reverse audio, per-clip fades (audio), duplicate-to-grid, crop-to-range, consolidate, bounce-in-place. All through the API and expressible in the composition. Done when: each op has an RPC; a split/gain/reverse sequence round-trips and renders correctly.

   • [x] Split/duplicate/reverse + note read-back landed (Source/ClipOps.cpp, commit): apiSplitClip (absolute beat → two clips, notes distributed, straddlers truncated), apiDuplicateClip (at beat, or butt-up-after), apiReverseClip (mirror notes in the content window; reverse the audio buffer for audio clips), apiGetClipNotes (read-back — also covers #5's deferred note-export). RPCs SplitClip/DuplicateClip/ReverseClip/GetClipNotes + Python client. Clips already serialise, so no new persistence. Verified precisely via GetClipNotes (split 8→4+4 with correct offsets; reverse 60→beat3/63→beat0; duplicate→3 clips) + smoke.sh.
   • [x] Repeat clip (tile) landed (commit): apiRepeatClip(trackId,index,copies) appends N back-to-back duplicates after a clip (each one clip-length further along), turning a 1-bar loop into an N-bar run — the "duplicate-to-grid" variant from #6. Copies share audio buffers (read-only) and duplicate MIDI notes. RepeatClip RPC (→SliceResult) + Python. Desktop UI: a "Repeat ▸ x2/x4/x8/x16" submenu on the clip menu (adds N-1 copies for an N× total) — screenshot-validated. smoke proves a 2-beat clip repeated x4 saves as 4 clips at beats 0/2/4/6.
   • [x] Loop-to-clip landed (commit): apiSetLoopToClip(trackId,index) sets the transport loop region to a clip's [start,end) and enables looping (audition a clip on repeat). TransportState/GetTransport gained loop_enabled/loop_start/loop_end so clients (and the smoke test) can read the loop back. SetLoopToClip RPC (ClipRef) + Python; transport() now returns the loop fields. Desktop UI: "Loop this clip" on the arrange-view clip menu — screenshot-validated (Loop button lights + the ruler draws the region over the clip). smoke proves SetLoopToClip on a clip at [5,8) sets the loop region there (verified via the new GetTransport loop fields).
   • [x] Audio-clip split fixed (commit): apiSplitClip on an audio clip previously copied the full buffer into both halves, so the right half REPLAYED from the buffer start at the cut instead of continuing. Now each half's buffer is trimmed to its own span (source-sample offset via the tempo-aware beats→seconds→samples map, same as crop); peaks rebuilt, inner fades cleared, / cleared so the halves embed. Fixes the existing "Split at playhead" clip-menu control for audio. smoke proves the split is audibly TRANSPARENT (whole-render mean-abs diff 0.000000 before vs after a split of a two-distinct-halves clip — a replay bug would corrupt the second half).

Wave 4 — Musical model & modulation

9. Modulation matrix — LFO/envelope → any ParamModel. ✦ M (needs Wave 1 #1) Idea #12. Modulation sources (LFO, envelope) route to any parameter id with tempo-sync, phase, shape, depth, smoothing, bipolar/unipolar. Persist routes as readable composition files; expose over gRPC. Done when: an LFO modulating a filter-cutoff param audibly moves it in a render; routes round-trip.

   • [x] LFO landed (Source/Modulation.cpp, commit): Mod{target,rate,depth, center,shape} LFO on any ParamModel id — value = center + depth*osc(rate·t), shapes sine/tri/saw/square. evaluateModulation(timeSeconds) runs on the audio thread under engineLock, mirroring evaluateAutomation's shape, driven off the playhead so renders are deterministic; a new lock-held applyParamValue writes any ParamModel id directly (audio-thread-safe, no message thread). apiSetModulation(upsert)/apiRemoveModulation/apiListModulations. Serialised (MODS/MOD + composition mods.toml). RPCs + Python. Verified: cutoff LFO changes the render (mean abs diff 0.033), shapes differ, round-trips.
   • [x] Tempo-sync landed (commit): Mod.syncBeats (>0 = cycle length in beats). The pure phase/osc math moved to Source/Lfo.h (lfoPhaseCycles/lfoOsc) so it's unit-testable without the engine — a synced LFO takes its phase from the beat position (evaluateModulation now gets tc.sampleToBeat(playhead)), so its period tracks the tempo map; a free LFO still uses rate·seconds. sync_beats on the SetModulation RPC + ListModulations + Python; serialised in MODS + mods.toml; the mixer "Add LFO" prompt gains a Sync field. GloopyTests::Lfo proves the sync-vs-free phase (and caught a wrong triangle expectation); smoke proves a synced LFO modulates the render and sync_beats round-trips.
   • [x] Phase offset + unipolar landed (commit): Mod.phase (0..1 cycle offset) and Mod.unipolar (value ranges center..center+depth instead of center +/- depth). The folding math is a new lfoUnit(shape, phase, offset, unipolar) in (unit-tested in ). / on SetModulation + ListModulations

Wave 5 — Analysis, plugins, diagnostics (offline / headless-friendly)

12. Offline loudness + render reports. S/M Ardour #14. After a render, report peak, true-peak, RMS, and LUFS (EBU R128); feed gloopy validate (Wave 2 #4) and a render report. Transient/onset detection later (for slicing). Done when: RenderToFile optionally returns/writes a report; the numbers match a known test signal within tolerance.

    • [x] Landed (Source/Loudness.cpp, commit): apiAnalyzeFile computes peak (dBFS), true-peak (dBTP, 4x windowed-sinc oversampled), RMS (dBFS), and integrated LUFS (ITU-R BS.1770 — hand-rolled RBJ K-weighting biquads correct at any rate + gated 400ms/100ms blocks, abs −70 / rel −10). No juce_dsp, no audio-thread. AnalyzeFile RPC + gloopy analyze <wav> CLI (JSON) + Python analyze_file. Verified against a −20 dBFS 1 kHz tone (peak −20.0, RMS −23.01, LUFS −23.26 vs ~−23.6 ref) and the smoke render.
    • [x] Wired into RenderToFile's return (commit): RenderRequest.report (bool) makes RenderToFile return a RenderResult { ok, error, LoudnessReport report } — one call for "render + measure" in CI/scripts. RenderResult is wire-compatible with Ack on fields 1/2, so existing clients are unaffected; Python render(..., report=True) returns the loudness dict. smoke.sh asserts the inline report equals a standalone AnalyzeFile and is non-silent.
    • [x] Momentary + short-term LUFS + LRA landed (commit): extends apiAnalyzeFile with max momentary (400 ms window — reuses the integrated-LUFS block loop), max short-term (3 s / 1 s-hop windows), and LRA (EBU Tech 3342: abs gate −70, relative gate −20 LU, P95−P10 of gated short-term). Added to LoudnessReport + the proto + the AnalyzeFile/RenderToFile-report paths + gloopy analyze JSON + Python. smoke proves on a 6 s steady sine that momentary ≥ integrated, short-term ≈ integrated, and LRA is bounded. Not yet: gloopy validate render+analyze wiring of these. Transient/onset detection landed as Source/Onsets.h (detectOnsets), consumed by the audio-clip "Slice at transients" op (Wave 3 #6).