free4chat-agent-media

name	free4chat-agent-media
description	free4chat Agent media plane (Go runtime, in-process Pion) architecture quick reference and staged debugging discipline — multi-MID attribution, signaling direction self-inspection, lazy OnTrack, daemon/env traps. Use when debugging Meeting Notes audio, Voice Reply, image/text attachments, unresponsive agents, or non-flowing RTP.

free4Chat Agent media plane quick reference (Go runtime, in-process Pion)

Canonical runtime: agent/ (self-contained Go binary). Media: agent/internal/media/ — media.Controller + media.Bridge with Pion compiled into the same executable. There is no child process, no sidecar engine bridge, no separately provisioned engine binary.

Architecture in one diagram

Human browsers ──WebRTC──> Cloudflare SFU <──in-process Pion── free4chat-agent (Go)
                                                                 │ resident daemon
                                  media.Controller (meetingNotes grant)
                                  → per-MID Opus frames → Doubao STT → transcript
                                  → Voice Reply: TTS PCM → Opus → publish

• The Go runtime owns all communication: MCP join/heartbeat/grant polling, every /api/sfu/* call, and credentials. Pion is in the same binary.
• ONE shared media session serves both Meeting Notes (Human audio ingress) and Voice Reply (Agent audio egress); bootstrap is receive-only and the outbound track is armed only at voice-grant activation.
• The meetingNotes grant is room state owned by RoomSession: every Agent media operation re-checks the current grant, and Stop is a server-side track close (do/realtimeMedia.ts), never a UI toggle.
• Text attachments: read_attachment returns a JSON envelope {attachment,data,text} for text/* files; the runtime inlines them as . Images ride MCP ImageContent.