ai-core-chat-experience

# Chat Experience This skill builds on ai-core. Read it first for critical rules. ## Setup — Minimal Chat App ### Server: API Route (TanStack Start) ```typescript ignore // src/routes/api.chat.ts import { createFileRoute } from '@tanstack/react-router' import { chat, toServerSentEventsResponse } from '@tanstack/ai' import { openaiText } from '@tanstack/ai-openai' export const Route = createFileRoute('/api/chat')({ server: { handlers: { POST: async ({ request }) => { const abortController = new AbortController() const body = await request.json() const { messages } = body const stream = chat({ adapter: openaiText('gpt-5.5'), messages, systemPrompts: ['You are a helpful assistant.'], abortController, }) return toServerSentEventsResponse(stream, { abortController }) }, }, }, }) ``` ### Client: React Component ```tsx // src/routes/index.tsx import { useState } from 'react' import { useChat, fetchServerSentEvents } from '@tanstack/ai-react' import type { UIMessage } from '@tanstack/ai-react' function ChatPage() { const [input, setInput] = useState('') const { messages, sendMessage, isLoading, error, stop } = useChat({ connection: fetchServerSentEvents('/api/chat'), }) const handleSubmit = () => { if (!input.trim()) return sendMessage(input.trim()) setInput('') } return ( <div> <div> {messages.map((message: UIMessage) => ( <div key={message.id}> <strong>{message.role}:</strong> {message.parts.map((part, i) => { if (part.type === 'text') { return <p key={i}>{part.content}</p> } return null })} </div> ))} </div> {error && <div>Error: {error.message}</div>} <div> <input value={input} onChange={(e) => setInput(e.target.value)} onKeyDown={(e) => { if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault() handleSubmit() } }} disabled={isLoading} placeholder="Type a message..." /> {isLoading ? ( <button onClick={stop}>Stop</button> ) : ( <button onClick={handleSubmit} disabled={!input.trim()}> Send </button> )} </div> </div> ) } ``` Vue/Solid/Svelte/Preact have identical patterns with different hook imports (e.g., `import { useChat } from '@tanstack/ai-solid'`). ## Core Patterns ### 1. Streaming Chat with SSE Server returns a streaming SSE Response; client parses it automatically. **Server:** ```typescript import { chat, toServerSentEventsResponse } from '@tanstack/ai' import { anthropicText } from '@tanstack/ai-anthropic' export async function POST(request: Request) { const { messages } = await request.json() const abortController = new AbortController() const stream = chat({ adapter: anthropicText('claude-opus-5'), messages, modelOptions: { temperature: 0.7, max_tokens: 2000, // Anthropic-native key }, systemPrompts: ['You are a helpful assistant.'], abortController, }) return toServerSentEventsResponse(stream, { abortController }) } ``` To make the SSE response resumable (reconnect after a drop/refresh without re-running the provider), pass a delivery-durability adapter: `toServerSentEventsResponse(stream, { durability: { adapter: memoryStream(request) } })` (`memoryStream` from `@tanstack/ai` is process-local, for dev/tests) or `durableStream(request, { server })` from `@tanstack/ai-durable-stream` (Durable Streams protocol, production). Each SSE event gets an opaque adapter-owned `id:`; `fetchServerSentEvents` auto-reconnects with `Last-Event-ID` and exposes `joinRun(runId)` to replay a run from the start. See `docs/resumable-streams/overview.md`. **Client:** ```typescript import { useChat, fetchServerSentEvents } from '@tanstack/ai-react' const { messages, sendMessage, isLoading, error, stop, status } = useChat({ connection: fetchServerSentEvents('/api/chat'), body: { provider: 'anthropic', model: 'claude-opus-5' }, onFinish: (message) => { console.log('Response complete:', message.id) }, onError: (err) => { console.error('Stream error:', err) }, }) ``` The `body` field is merged into the POST request body alongside `messages`, letting the server read `data.provider`, `data.model`, etc. The `status` field tracks the chat lifecycle: `'ready'` | `'submitted'` | `'streaming'` | `'error'`. ### 2. Rendering Thinking/Reasoning Content Models with extended thinking (Claude, Gemini) emit `ThinkingPart` in the message parts array. ```tsx import type { UIMessage } from '@tanstack/ai-react' function MessageRenderer({ message }: { message: UIMessage }) { return ( <div> {message.parts.map((part, i) => { if (part.type === 'thinking') { const isComplete = message.parts .slice(i + 1) .some((p) => p.type === 'text') return ( <details key={i} open={!isComplete}> <summary> {isComplete ? 'Thought process' : 'Thinking...'} </summary> <pre>{part.content}</pre> </details> ) } if (part.type === 'text' && part.content) { return <p key={i}>{part.content}</p> } if (part.type === 'tool-call') { return ( <div key={part.id}> Tool call: {part.name} ({part.state}) </div> ) } return null })} </div> ) } ``` Server-side, turn thinking on with the `reasoning` option on `chat()`: ```typescript import { chat, toServerSentEventsResponse } from '@tanstack/ai' import { geminiText } from '@tanstack/ai-gemini' export async function POST(request: Request) { const { messages } = await request.json() const stream = chat({ adapter: geminiText('gemini-3.8-flash'), messages, reasoning: 'high', // Gemini 3.x: thinkingLevel; Gemini 2.x: thinkingBudget }) return toServerSentEventsResponse(stream) } ``` ### 3. Sending Multimodal Content (Images) Use `sendMessage` with a `MultimodalContent` object instead of a plain string. ```typescript import { useChat, fetchServerSentEvents } from '@tanstack/ai-react' import type { ContentPart } from '@tanstack/ai' const { sendMessage } = useChat({ connection: fetchServerSentEvents('/api/chat'), }) function sendImageMessage(text: string, imageBase64: string, mimeType: string) { const contentParts: Array<ContentPart> = [ { type: 'text', content: text }, { type: 'image', source: { type: 'data', value: imageBase64, mimeType }, }, ] sendMessage({ content: contentParts }) } function sendImageUrl(text: string, imageUrl: string) { const contentParts: Array<ContentPart> = [ { type: 'text', content: text }, { type: 'image', source: { type: 'url', value: imageUrl }, }, ] sendMessage({ content: contentParts }) } ``` Render image parts in received messages: ```tsx import type { UIMessage } from '@tanstack/ai-react' function ImagePart({ part }: { part: UIMessage['parts'][number] }) { if (part.type !== 'image') return null // A provider file handle is an opaque id, so the browser cannot load it. if (part.source.type === 'file') return null const src = part.source.type === 'url' ? part.source.value : `data:${part.source.mimeType};base64,${part.source.value}` return <img src={src} alt="Attached image" /> } ``` For media reused across turns, upload once via a provider Files adapter (`openaiFiles()`, `anthropicFiles()`, `geminiFiles()`, `grokFiles()`, `falFiles()`) and send a `{ type: 'file' }` source built with `fileSourceFromHandle(handle)` instead of re-sending base64 each request. The source is `{ type: 'file', value, provider }`: an opaque handle and the adapter that issued it. A different provider (or one without Files API support at all) rejects it with a clear error before any request is sent. The source crosses the chat wire, so the browser can put it straight into the `sendMessage` content. Import `fileSourceFromHandle` from the browser-safe `@tanstack/ai/client` entry. See `ai-core/adapter-configuration/SKILL.md` §7 and `docs/advanced/files-api.md`. ### 4. Sending Audio Messages (Browser Recording) Use `useAudioRecorder` from `@tanstack/ai-react` (or `createAudioRecorder` in Svelte) to capture audio in the browser. The resolved `AudioRecording` includes a ready-to-use `part` that slots directly into `sendMessage`. ```typescript import { useAudioRecorder, useChat, fetchServerSentEvents, } from '@tanstack/ai-react' const { isRecording, isSupported, start, stop } = useAudioRecorder() const { sendMessage } = useChat({ connection: fetchServerSentEvents('/api/chat'), }) async function toggle() { if (!isRecording) { await start() return } const recording = await stop() await sendMessage({ content: [recording.part] }) } ``` `recording.part` is `{ type: 'audio', source: { type: 'data', value: base64, mimeType } }`. Returns the recorder's native format (`audio/webm` or `audio/mp4`) with no transcoding. ### 5. HTTP Stream Format (Alternative to SSE) Use `toHttpResponse` + `fetchHttpStream` for newline-delimited JSON instead of SSE. **Server:** ```typescript import { chat, toHttpResponse } from '@tanstack/ai' import { openaiText } from '@tanstack/ai-openai' export async function POST(request: Request) { const { messages } = await request.json() const abortController = new AbortController() const stream = chat({ adapter: openaiText('gpt-5.6'), messages, abortController, }) return toHttpResponse(stream, { abortController }) } ``` **Client:** ```typescript import { useChat, fetchHttpStream } from '@tanstack/ai-react' const { messages, sendMessage } = useChat({ connection: fetchHttpStream('/api/chat'), }) ``` The only difference is swapping `toServerSentEventsResponse` / `fetchServerSentEvents` for `toHttpResponse` / `fetchHttpStream`. Everything else stays identical. This includes resumability: pass the same `durability` adapter to `toHttpResponse(stream, { durability: { adapter: memoryStream(request) } })` and each NDJSON line becomes an `{ id, chunk }` envelope. `fetchHttpStream` auto-reconnects with `Last-Event-ID`, de-dupes the replayed prefix, and exposes `joinRun(runId)` — the same guarantees as resumable SSE. The XHR adapters (`xhrServerSentEvents` / `xhrHttpStream`) are resumable too. ### 6. MCP Tool Discovery via `chat({ mcp })` Pass `mcp` to let `chat()` own discovery **and** lifecycle for one or more MCP clients. Useful when you want minimal boilerplate and don't need to reuse the clients across calls. `createMCPClient` tries spec `2026-07-28` first. If the server does not support that spec, the client uses the 2025 initialize handshake. ```typescript // Prop shape: // chat({ // ..., // mcp: { // clients: Array<MCPClient | MCPClients>, // connection?: 'close' | 'keep-alive', // default: 'close' // lazyTools?: boolean, // onDiscoveryError?: (error: unknown, source) => void, // } // }) ```