audio-nodes

# Skill: AudioNodes Golden references: `GainNode.h/.cpp` (effect node), `OscillatorNode.h/.cpp` (scheduled source). Mirror their structure for any new node. See [gainnode-example.md](gainnode-example.md) for an annotated header + .cpp. **If spec defaults or parameter ranges are unclear → fetch https://webaudio.github.io/web-audio-api/ before writing any constructor code.** --- ## Directory Structure ``` common/cpp/audioapi/core/ ├── AudioNode.h / .cpp # Base class for all nodes ├── AudioParam.h / .cpp # Automatable parameter ├── BaseAudioContext.h / .cpp # Engine + node factory ├── AudioContext.h / .cpp # Real-time context ├── OfflineAudioContext.h / .cpp # Offline rendering context ├── sources/ │ ├── AudioScheduledSourceNode.h # Base for start/stop sources (INTERNAL) │ ├── AudioBufferBaseSourceNode.h # Base for buffer playback (INTERNAL) │ ├── OscillatorNode.h / .cpp │ ├── AudioBufferSourceNode.h / .cpp │ ├── AudioBufferQueueSourceNode.h / .cpp │ ├── ConstantSourceNode.h / .cpp │ ├── WorkletSourceNode.h / .cpp │ └── RecorderAdapterNode.h / .cpp ├── effects/ │ ├── GainNode.h / .cpp │ ├── BiquadFilterNode.h / .cpp │ ├── DelayNode.h / .cpp │ ├── IIRFilterNode.h / .cpp │ ├── StereoPannerNode.h / .cpp │ ├── WaveShaperNode.h / .cpp │ ├── ConvolverNode.h / .cpp │ ├── WorkletNode.h / .cpp │ ├── channel_merger/ # ChannelMerger internal input/output nodes (composite) │ ├── channel_splitter/ # ChannelSplitter internal input/output nodes (composite) │ └── PeriodicWave.h / .cpp # Wave table (not a node) ├── analysis/ │ └── AnalyserNode.h / .cpp ├── destinations/ │ └── AudioDestinationNode.h / .cpp ├── inputs/ │ └── AudioRecorder.h / .cpp └── utils/ └── AudioGraphManager.h / .cpp ``` --- ## The Audio Thread Contract `processNode()` runs on the **audio thread** — the real-time rendering thread driven by the native audio driver (Oboe on Android, CoreAudio on iOS). This thread has strict requirements: **MUST NOT in `processNode()`:** - Allocate or free memory (`new`, `delete`, `malloc`, `free`, `std::vector::push_back` that grows, etc.) - Acquire any mutex or lock (`std::mutex`, `std::lock_guard`, etc.) - Make any blocking syscall (file I/O, socket, `sleep`, `wait`) - Call into JavaScript — no JSI calls, no `callInvoker_->invokeSync()` - Throw exceptions (or rely on exception unwinding paths that allocate) **Preallocate everything in the constructor:** ```cpp // Constructor — JS thread, allocations OK GainNode::GainNode(const std::shared_ptr<BaseAudioContext> &context, const GainOptions &options) : AudioNode(context, options) { // Preallocate the AudioBuffer used during processing audioBuffer_ = std::make_shared<AudioBuffer>(channelCount_, context->getBufferSize()); // Preallocate params — they own their internal AudioBuffer too gainParam_ = std::make_shared<AudioParam>( options.gain, -3.4028234663852886e+38f, 3.4028234663852886e+38f, context); } // processNode — audio thread, NO allocations std::shared_ptr<AudioBuffer> GainNode::processNode( const std::shared_ptr<AudioBuffer> &processingBuffer, int framesToProcess) { // Already-allocated buffer reused each render quantum auto gainValues = gainParam_->processARateParam(framesToProcess, time); for (size_t i = 0; i < processingBuffer->getNumberOfChannels(); i++) { processingBuffer->getChannel(i)->multiply(*gainValues->getChannel(0), framesToProcess); } return processingBuffer; } ``` --- ## Processable State (reverse-topo pull) Which nodes run each render quantum is decided by `AudioGraph::settleProcessableState()`, run inside `Graph::process()` after toposort/compaction and before the forward `iter()` pass. It is an **audio-thread-only** concern — never derived from HostGraph adjacency (that mutates on the JS thread under `nodesMutex_`). `GraphObject::PROCESSABLE_STATE`: - `ALWAYS_PROCESSABLE` — seed / pull root. Set in ctors of `AudioDestinationNode` and `AnalyserNode`. Never reset by settle. - `CONDITIONAL_PROCESSABLE` — on this quantum because something processable downstream pulls it. Recomputed from scratch every quantum. - `NOT_PROCESSABLE` — idle / disconnected / default. Settle algorithm (allocation-free): 1. **Reverse pull**: walk the topo-sorted node array sinks → sources; for every `ALWAYS_`/`CONDITIONAL_PROCESSABLE` node, mark its inputs (and processable-links) `CONDITIONAL_PROCESSABLE`. Iterates to a fixpoint for processable-links. 2. **End-of-quantum demotion**: after `processInputs()`, each node that was `CONDITIONAL_PROCESSABLE` flips back to `NOT_PROCESSABLE` in `GraphObject::process()`. That replaces a global reset at the start of settle — nodes that ran last quantum are already idle when the next pull begins. Key invariants: - **Pull from `processableState_`, never `AudioNode::isProcessable()`.** A tail-bearing node (Delay/Convolver/Biquad) overrides `isProcessable()` to stay `true` while its tail drains after a disconnect; using that for the pull would wrongly re-activate its whole upstream cone. The tail node stays scheduled via that override; its `processableState_` is `NOT_PROCESSABLE`, so it correctly does not pull upstream. - **`disable()` is sticky.** `AudioNode::disable()` sets `NOT_PROCESSABLE` **and** `alwaysNotProcessable_ = true`, so a finished source still wired to a live consumer is not re-activated by the every-quantum pull. Sources call `disable()` from the audio thread when playback finishes. - **DelayReader → DelayWriter** have no audio edge (they share a ring buffer). `Graph::linkNodes(reader, writer)` records a processable-link, mirrored onto `AudioGraph::Node::link_head`. Settle follows links so pulling the reader also pulls the writer and the writer's inputs. Links are NOT part of the topological sort (that would create a cycle for feedback delays). --- ## Class Hierarchy ```mermaid classDiagram direction TD class AudioScheduledSourceNode { <<internal base>> start(when) stop(when) } class AudioBufferBaseSourceNode { <<internal base>> playbackRate AudioParam detune AudioParam } AudioNode <|-- AudioScheduledSourceNode AudioNode <|-- GainNode AudioNode <|-- BiquadFilterNode AudioNode <|-- DelayNode AudioNode <|-- IIRFilterNode AudioNode <|-- StereoPannerNode AudioNode <|-- WaveShaperNode AudioNode <|-- ConvolverNode AudioNode <|-- WorkletNode AudioNode <|-- AnalyserNode AudioNode <|-- AudioDestinationNode AudioScheduledSourceNode <|-- AudioBufferBaseSourceNode AudioScheduledSourceNode <|-- OscillatorNode AudioScheduledSourceNode <|-- ConstantSourceNode AudioScheduledSourceNode <|-- WorkletSourceNode AudioBufferBaseSourceNode <|-- AudioBufferSourceNode AudioBufferBaseSourceNode <|-- AudioBufferQueueSourceNode ``` ### AudioRecorder (not an AudioNode) `core/inputs/AudioRecorder` is a standalone base class, not part of the `AudioNode` hierarchy — it feeds the graph through a `RecorderAdapterNode` instead of being processed by it. The split between it and `IOSAudioRecorder` / `AndroidAudioRecorder` is: the base owns everything that happens to recorded frames (file writer, JS callback, adapter node — `enableFileOutput`, `setupFileWriter`, `setOnAudioReadyCallback`, `connect`, `detachSideEffects`/`finalizeSideEffects`), the subclasses own only the platform input stream. The one thing the base needs from the platform is `resolveStreamFormat()`, returning sample rate, channel count and max frames per buffer; iOS reads it from `NativeAudioRecorder` on every call (a route change invalidates it), Android returns values cached when the Oboe stream opened. Add shared recorder behavior to the base, not to one platform; state only one platform touches (Android's `streamMutex_`, iOS's `inputChannelCount_`) lives in that subclass. State a consumer already owns stays with the consumer: the adapter node's ring layout (`RecorderAdapterNode::writeFrames`) and the session's file paths (returned by `AudioFileWriter::closeFile()`) are not mirrored in the recorder. Pitfall: never redeclare a base member (`streamSampleRate_`, `fileWriter_`, `lastCallbackFrameCount_`) in a platform recorder. The shadowing copy compiles fine, but the base's audio-thread fan-out reads its own member and silently drops that output. --- ### AudioScheduledSourceNode (internal only — not exposed to JS directly) Base class for source nodes that have a scheduled start and stop time. **Not instantiated directly.** ```cpp // Playback state machine enum class PlaybackState { UNSCHEDULED, // before start() called SCHEDULED, // start() called, waiting for startTime_ PLAYING, // actively producing audio STOP_SCHEDULED, // stop() called, waiting for stopTime_ FINISHED // done, node will be disabled }; ``` Subclasses call `updatePlaybackInfo(currentTime, framesToProcess)` at the top of `processNode()` to transition the state machine and handle sample-accurate start/stop. When the node finishes, `AudioScheduledSourceNode::disable()` fires `ENDED` through its `EventCaller` (`dispatchEmptyFromAudioThread`); a node that will never render (stop-before-start) defers it with `EventCaller::deferEmpty` instead — see `thread-safety-itc`. Pitfalls: - `start(double)` in this base class is a separate overload from `AudioBufferSourceNode::start(when, offset, duration)`. Calling `node->start(when)` on an ABSN skips the derived logic; the HostObject always passes three arguments, and so must tests. - `AudioBufferBaseSourceNode::processNode` is `final` and returns early on `isEmpty()`. Whether a started-but-empty source ends there (`ended` fires) is `endsWhenStartedEmpty()`: the base default is true (spec: a null buffer forces `stop = currentTime`); `AudioBufferQueueSourceNode` overrides it to false because an empty queue is waiting for the next enqueue. The check runs per quantum on the audio thread on purpose, because the buffer setter legitimately acquires content assigned after `start()` but before the first render. --- ## processNode() Signature ```cpp protected: /// @note Audio Thread only virtual std::shared_ptr<AudioBuffer> processNode( const std::shared_ptr<AudioBuffer> &processingBuffer, int framesToProcess) = 0; ``` - `processingBuffer` — already contains the mixed input from all connected input nodes. Modify in-place and return it. - `framesToProcess` — number of samples per channel to process, typically 128 (RENDER_QUANTUM_SIZE). - Called by `AudioNode::processAudio()` which handles input mixing, channel count modes, and deduplication (via `lastRenderedFrame_`). --- ## Thread Annotations in Header Files **Annotate every method with the thread it is safe to call from.** Use comments in the header: ```cpp class MyNode : public AudioNode { public: /// @note JS Thread only void setSomething(float value); float getSomething() const; protected: /// @note Audio Thread only std::shared_ptr<AudioBuffer> processNode( const std::shared_ptr<AudioBuffer> &processingBuffer, int framesToProcess) override; }; ``` In `AudioParam.h` the pattern is: ```cpp /// @note JS Thread only [[nodiscard]] inline float getValue() const noexcept { ... } void setValue(float value); void setValueAtTime(float value, double startTime); /// Audio-Thread only methods (idempotent per quantum — see below) std::shared_ptr<DSPAudioBuffer> processARateParam(int framesToProcess, double time); float processKRateParam(double time); // k-rate is quantum-wide ``` --- ## AudioParam — Automatable Parameters Every automatable property (frequency, gain, detune, Q, etc.) is an `AudioParam`. ```cpp gainParam_ = std::make_shared<AudioParam>( defaultValue, minValue, maxValue, context ); ``` ### A-rate vs K-rate - **A-rate (audio-rate)**: one value per sample — use when the parameter can change significantly within a render quantum (e.g. frequency modulation) ```cpp // Call processARateParam() for per-sample values — returns AudioBuffer, no allocation auto gainValues = gainParam_->processARateParam(framesToProcess, time); float *values = gainValues->getChannel(0)->getData(); // values[i] is the gain for frame i ``` - **K-rate (control-rate)**: one value per render quantum — use when the parameter changes slowly ```cpp // Call processKRateParam() for a single quantum-wide value float gain = gainParam_->processKRateParam(time); // Single value for the whole block ``` ### Param class hierarchy & idempotency `AudioParam` and `CompositeAudioParam<CombineFunction>` both derive from the abstract `GeneralizedAudioParam` base (`core/GeneralizedAudioParam.h`), which owns the nominal range, the a-rate `outputBuffer_`, and the per-quantum memoization state, and centralizes clamping via `finalizeKRate` / `finalizeARate`. - **`AudioParam`** — the only JS-connectable param; owns `inputBuffer_` (BridgeNode modulation). - **`CompositeAudioParam<Fn>`** (`core/CompositeAudioParam.hpp`) — represents a spec `computedValue` (e.g. `computedOscFrequency`). `Fn` is a pure, captureless free function (defined in the owning node's header, next to the composite member) taking float children and returning float; its arity is deduced. It processes each child, folds `Fn` over them, and clamps to its own nominal range. No `inputBuffer_`. `processKRateParam(time)` / `processARateParam(frames, time)` are **idempotent**: a repeat call with the same arguments returns the cached result and does **not** re-consume modulation. This is why a composite and a node can both read the same child param in one quantum. It also means `processNode()` should read `context->getCurrentTime()` **once** and thread that same `double` into every param call so the cache keys match (the context clock is constant within a quantum). Consequently, unit tests that re-process the same node must advance the clock (e.g. `context->processGraph(buffer.get(), frames)`) between renders. **Clamping (§ 1.6.3):** automation intrinsic values are computed **without** clamping (see `getValueAtTimeUnmodulated` / `ParamRenderQueue`). Clip only in `finalizeKRate` / `finalizeARate` after adding modulation — never on the intrinsic alone before modulation. **Automation timing model (scheduled times, snapped on demand):** a `RenderParamEvent` stores only its scheduled `startTime`/`endTime`, and `calculateValueAtTime` interpolates on them unmodified (the spec defines the formulas on real times; a ramp between two times inside one frame must still interpolate on those times — WPT `audioparam-close.html`). Effect boundaries live on the frame grid instead: `ParamRenderQueue::computeValueAtTime` calls `snapToSampleFrameTime` (`std::round(t * sampleRate) / sampleRate`) on the scheduled times whenever it decides which event is in effect, matching Blink and the WPT reference's `timeToSampleFrame`. Nothing stores the snapped values, so resolving neighbours (`resolveEventValues`, `cancelAndHoldAtTime`) only ever touches the scheduled times. A queued event supersedes the current one as soon as its snapped start is due, even mid-ramp; ramps stay active until their scheduled end, other finite events end on their snapped end. On the evaluation side, `processARateParam` derives each sample's time as `(quantumStartFrame + i) / sampleRate` — never accumulate `time += 1/sampleRate`, the ULP drift lands boundaries one frame late. **Never snap an automation time through an integer type.** `snapToSampleFrameTime` rounds in `double` instead of reusing `dsp::timeToSampleFrame`, which returns a frame index as `size_t`. Automation times are unvalidated user input: nothing in `AudioParam` or its host object rejects negative or absurd values, and WPT schedules ramps at `1e300` (`audioparam-large-endtime.html`). A `double → size_t → int` round trip on that is undefined behaviour, and in practice produced a small *negative* time that sorted the event ahead of everything legitimately scheduled. The same unguarded cast is still reachable from `AudioScheduledSourceNode`'s `startTime_`/`stopTime_` conversions. **Param queue capacity:** both param event queues (`ParamRenderQueue` on `AudioParam`, `ParamControlQueue` on the host object) are bounded by `AUDIO_PARAM_MAX_QUEUED_EVENTS` and **silently drop** events past capacity (`BoundedPriorityQueue::push` returns false, nobody checks). Automation scheduled far ahead must fit entirely; the WPT audioparam suites queue 100 events per file. Symptom of overflow: automation freezes at the last accepted event. ### JS → Audio Thread parameter updates `CrossThreadEventScheduler<T>` is a lock-free SPSC channel. When JS calls `param.setValueAtTime(...)`, it enqueues a lambda on the scheduler. The audio thread drains the queue at the start of each `processARateParam` / `processKRateParam` call. ```cpp // JS-thread (in AudioParam): void AudioParam::setValueAtTime(float value, double startTime) { eventScheduler_.scheduleEvent([value, startTime](AudioParam &param) {