Expert guide for building AI-powered applications with the Glove framework. Use when working with glove-core, glove-react, glove-next, tools, display stack, model adapters, stores, or any Glove example project.
Expert guide for building AI-powered applications with the Glove framework. Use when working with glove-core, glove-react, glove-next, tools, display stack, model adapters, stores, or any Glove example project.
Glove Framework — Development Guide
You are an expert on the Glove framework. Use this knowledge when writing, debugging, or reviewing Glove code.
What Glove Is
Glove is an open-source TypeScript framework for building AI-powered applications. Users describe what they want in conversation, and an AI decides which capabilities (tools) to invoke. Developers define tools and renderers; Glove handles the agent loop.
renderData — Client-only data returned from do() that is NOT sent to the AI model. Used by renderResult for history rendering.
Adapter — Pluggable interfaces for Model, Store, DisplayManager, and Subscriber. Swap providers without changing app code.
Context Compaction — Auto-summarizes long conversations to stay within context window limits. The store preserves full message history (so frontends can display the entire chat), while Context.getMessages() splits at the last compaction summary so the model only sees post-compaction context. Summary messages are marked with is_compaction: true.
interfaceToolResultData {
status: "success" | "error";
data: unknown; // Sent to the AI modelmessage?: string; // Error message (for status: "error")renderData?: unknown; // Client-only — NOT sent to model, used by renderResult
}
Important: Model adapters explicitly strip renderData before sending to the AI. This makes it safe to store sensitive client-only data (e.g., email addresses, UI state) in renderData.
<Render> Component
Headless render component that replaces manual timeline rendering:
Token handlers: createVoiceTokenHandler (already in glove-next, no separate import)
Included in glove-next
Architecture
Mic → VAD → STTAdapter → glove.processRequest() → TTSAdapter → Speaker
GloveVoice wraps a Glove instance with a full-duplex voice pipeline. Glove remains the intelligence layer — all tools, display stack, and context management work normally. STT and TTS are swappable adapters. Text tokens stream through a SentenceBuffer into TTS in real-time.
Quick Start (Next.js + ElevenLabs)
Step 1: Token routes — server-side handlers that exchange your API key for short-lived tokens
const { runnable } = useGlove({ tools, sessionId });
const voice = useGloveVoice({ runnable, voice: { stt, createTTS, vad } });
// voice.mode, voice.isActive, voice.isMuted, voice.error, voice.transcript// voice.start(), voice.stop(), voice.interrupt(), voice.commitTurn()// voice.mute(), voice.unmute() — gate mic audio to STT/VAD// voice.narrate("text") — speak text via TTS without model (returns Promise)
Turn Modes
Mode
Behavior
Use for
"vad" (default)
Auto speech detection + barge-in
Hands-free, voice-first apps
"manual"
Push-to-talk, explicit commitTurn()
Noisy environments, precise control
Narration + Mic Control
voice.narrate(text) — Speak arbitrary text through TTS without the model. Resolves when audio finishes. Auto-mutes mic during narration. Abortable via interrupt(). Safe to call from pushAndWait tool handlers.
voice.mute() / voice.unmute() — Gate mic audio forwarding to STT/VAD. audio_chunk events still fire when muted (for visualization).
audio_chunk event — Raw Int16Array PCM from the mic, emitted even when muted. Use for waveform/level visualization.
Compaction silence — Voice automatically ignores text_delta during context compaction so the summary is never narrated.
Voice-First Tool Design
Use pushAndForget instead of pushAndWait — blocking tools that wait for clicks are unusable in voice mode
Return descriptive text in data — the LLM reads it to formulate spoken responses
Add a voice-specific system prompt — instruct the agent to narrate results concisely
Use narrate() for slot narration — read display content aloud from within tool handlers
Supported Voice Providers
Provider
Token Handler Config
Env Variable
ElevenLabs
{ provider: "elevenlabs", type: "stt" | "tts" }
ELEVENLABS_API_KEY
Deepgram
{ provider: "deepgram" }
DEEPGRAM_API_KEY
Cartesia
{ provider: "cartesia" }
CARTESIA_API_KEY
Supporting Files
For detailed API reference, see api-reference.md.
For example patterns from real implementations, see examples.md.
Common Gotchas
model_response_complete vs model_response: Streaming adapters emit model_response_complete, not model_response. Subscribers must handle both.
Closure capture in React hooks: When re-keying sessions, use mutable let currentKey = key to avoid stale closures.
React useEffect timing: State updates don't take effect in the same render cycle — guard with early returns.
Browser-safe imports: glove-core barrel exports include native deps (better-sqlite3). For browser code, import from subpaths: glove-core/core, glove-core/glove, glove-core/display-manager, glove-core/tools/task-tool.
Displaymanager casing: The concrete class is Displaymanager (lowercase 'm'), not DisplayManager. Import it as: import { Displaymanager } from "glove-core/display-manager".
createAdapter stream default: stream defaults to true, not false. Pass stream: false explicitly if you want synchronous responses.
Tool return values: The do function should return ToolResultData with { status, data, renderData? }. data goes to the AI; renderData stays client-only.
Zod .describe(): Always add .describe() to schema fields — the AI reads these descriptions to understand what to provide.
displayPropsSchema is optional but recommended: defineTool's displayPropsSchema is optional, but recommended for tools with display UI — tools without display should use raw ToolConfig instead.
renderData is stripped by model adapters: Model adapters explicitly exclude renderData when formatting tool results for the AI, so it's safe for client-only data.
SileroVAD must use dynamic import: Never import glove-voice/silero-vad at module level in Next.js/SSR. Use await import("glove-voice/silero-vad") to avoid pulling WASM into the server bundle.
Next.js transpilePackages: Add "glove-voice" to transpilePackages in next.config.ts so Next.js processes the ES module.
createTTS must be a factory: GloveVoice calls it once per turn to get a fresh TTS adapter. Pass () => new ElevenLabsTTSAdapter(...), not a single instance.
Barge-in protection requires unAbortable: A pushAndWait resolver suppresses voice barge-in at the trigger level (GloveVoice skips interrupt() when resolverStore.size > 0). But that alone doesn't protect the tool — if interrupt() is called by other means, only unAbortable: true on the tool guarantees it runs to completion despite the abort signal. Use both together for mutation-critical tools like checkout. Use pushAndForget for voice-first tools.
Empty committed transcripts: ElevenLabs Scribe may return empty committed transcripts for short utterances. The adapter auto-falls back to the last partial transcript.
TTS idle timeout: ElevenLabs TTS WebSocket disconnects after ~20s idle. GloveVoice handles this by closing TTS after each model_response_complete and opening a fresh session on next text_delta.
onnxruntime-web build warnings: Critical dependency: require function is used in a way... warnings from onnxruntime-web are expected and harmless.
Audio sample rate: All adapters must agree on 16kHz mono PCM (the default). Don't change unless your provider explicitly requires something different.
narrate() auto-mutes mic: voice.narrate() automatically mutes the mic during playback to prevent TTS audio from feeding back into STT/VAD. It restores the previous mute state when done.
narrate() needs a started pipeline: Calling narrate() before voice.start() throws. The TTS factory and AudioPlayer must be initialized.
Voice auto-silences during compaction: When context compaction is triggered, the voice pipeline ignores all text_delta events between compaction_start and compaction_end. The compaction summary is never narrated.
isCompacting for React UI feedback: GloveState.isCompacting is true while compaction is in progress. Use it to show a loading indicator or disable input during compaction.
