06-appkit-serving-wiring

# Wire Serving Endpoint into AppKit Connect a Databricks Model Serving or Agent endpoint to an AppKit app using the Serving plugin. Build streaming chat UIs, single-shot query interfaces, or server-side proxy routes that post-process agent responses. ## When to Use - Wiring a deployed Agent or Model Serving endpoint into an AppKit app - Building a streaming chat interface for a conversational agent - Adding a single-shot inference call to a page (classification, summarization, etc.) - Post-processing agent responses server-side before sending to the frontend **Not for registering the plugin.** Use `04-appkit-plugin-add` with [references/plugin-serving.md](../04-appkit-plugin-add/references/plugin-serving.md) to install and configure the Serving plugin first. **Not for deploying.** Use `03-appkit-deploy` after wiring is complete. **Not for deploying the agent itself.** The agent endpoint must already exist on Databricks Model Serving. --- ## Before You Begin **Prerequisites — verify these before proceeding:** 1. Agent endpoint is deployed on Databricks Model Serving and in `READY` state 2. The Serving plugin is registered in `server/server.ts` (via `04-appkit-plugin-add`) 3. Endpoint added as an **app resource** with `CAN_QUERY` permission (via Databricks Apps UI or `app.yaml` resources) 4. `app.yaml` has `DATABRICKS_SERVING_ENDPOINT_NAME` with `valueFrom: serving-endpoint` 5. `npm run build` passes with the Serving plugin imported 6. `serving` export is available in your installed AppKit version (see [04-appkit-plugin-add/SKILL.md](../04-appkit-plugin-add/SKILL.md) Step 1b). If `typeof require('@databricks/appkit').serving === "undefined"`, stop here — read [references/custom-proxy-fallback.md](references/custom-proxy-fallback.md) and skip Step 3. **Upstream docs (always check for latest):** ```bash npx @databricks/appkit docs "serving" ``` > **Key architectural difference from Lakebase wiring:** The Serving plugin auto-registers HTTP routes (`/api/serving/:alias/invoke` and `/api/serving/:alias/stream`) via the plugin lifecycle — like the Genie plugin. You do NOT need `server.extend()` to create routes for basic invoke/stream calls. Use `server.extend()` only when you need a custom server-side proxy route that post-processes the agent's response (Step 6). ### Working in Genie Code (client routing) All the **code** in this skill — plugin config, hooks, proxy routes — is written the same way on both clients. Only the toolchain commands and the deployed-app verification differ. `$APP_NAME` / `$PROFILE` / endpoint name resolve from `.vibecoding-state.md` when a prior phase wrote it (don't re-derive). Apply these substitutions: | IDE/CLI (as written) | Genie Code substitution | |----------------------|--------------------------| | `databricks serving-endpoints get/query … --profile $PROFILE` (Step 1) | run via `runDatabricksCli` (read-tier, pre-approved), **omit `--profile`** | | `npm run build` gates (Steps 3, 8) — "**You MUST run `npm run build`**" | **IDE-only** convenience — no local Node toolchain. Skip; the platform builds **server-side** on deploy and surfaces errors in `databricks apps logs <name>` | | `npm run dev` | not available (and blocked pre-deploy: `serving()` needs platform-injected env) — verify on the deployed app | | `npx @databricks/appkit docs "serving"` | npx absent (P9) — WebFetch https://databricks.github.io/appkit/docs/plugins/serving | | `databricks auth token` + `curl -H "Authorization: Bearer …"` (Steps 9b/9c) | hard-blocked / raw Bearer rejected by the Apps OAuth gate → use **browser** (open the app URL, test chat) **or** the 3-hop OAuth `requests.Session()` test in `03-appkit-deploy` | | `databricks apps deploy …` (Step 9a) | see the `03-appkit-deploy` deploy-routing contract (`runDatabricksCli`, else SDK `w.apps.deploy(... SNAPSHOT)`) | Paths are relative to `apps_lakebase/$APP_NAME` — inside your git-cloned workshop project (`artifact_root`) on Genie Code, never the read-only `.assistant/skills` copy and never `/tmp`. See `skills/genie-code-environment` for the full manifest. --- ## Decision Defaults When multiple approaches are valid, use these defaults. Override only if the use case demands it. | Decision | Default | Rationale | |----------|---------|-----------| | Streaming or invoke? | `useServingStream` for agent chat; `useServingInvoke` for single-shot | Agents are typically slow — streaming gives progressive UX feedback | | Single or multiple endpoints? | Single (reads `DATABRICKS_SERVING_ENDPOINT_NAME`) | Multi-endpoint via `endpoints: {}` config is an advanced pattern | | Chat UI pattern | Scrollable message list + input box + streaming indicator | Most agent UIs are conversational | | Conversation state | Client-side `useState<Message[]>` array | `useServingStream` is stateless; the app must manage history | | Agent response mapping | Show raw text by default | Not all agents return structured data; see [references/agent-response-mapping.md](references/agent-response-mapping.md) for structured patterns | | OBO auth | Always (AppKit default) | Per-user `CAN_QUERY` enforcement on the serving endpoint | | Timeout | 120000ms (2 minutes) | Agent endpoints may be slow; configurable via `serving({ timeout })` | | `npm run dev` before deploy? | No — `npm run build` only | Serving plugin may throw `ConfigurationError` when env vars are not set | | Check endpoint schema first? | Yes — verify streaming support | Endpoints without an OpenAPI streaming schema produce `chunk: unknown` | --- ## Step 1: Verify Endpoint Access Confirm the serving endpoint is reachable and in `READY` state before writing any code. ### 1a. Check Endpoint Status ```bash databricks serving-endpoints get <endpoint-name> --profile $PROFILE --output json | jq '.state' ``` Expected output includes `"ready": "READY"`. If the endpoint is not ready, wait for it to finish provisioning before proceeding. ### 1b. Test with a Sample Query ```bash databricks serving-endpoints query <endpoint-name> \ --profile $PROFILE \ --input '{"messages": [{"role": "user", "content": "Hello"}]}' ``` If this returns a valid response, the endpoint is accessible and the agent is functional. ### 1c. Automated Test Script ```bash bash apps_lakebase/skills/06-appkit-serving-wiring/scripts/test-serving-endpoint.sh \ --endpoint-name <endpoint-name> --profile $PROFILE ``` **Gate:** The endpoint responds with `READY` status and returns a non-empty response to the sample query. --- ## Step 2: Configure app.yaml ### 2a. Add the Serving Endpoint as an App Resource The endpoint must be registered as a resource in the Databricks Apps UI or in the `app.yaml` resources section: - **Resource key:** `serving-endpoint` (default) - **Permission:** `CAN_QUERY` - **Requirement:** Endpoint must be in `READY` state If using the Databricks Apps UI: navigate to your app's settings, add a Model Serving Endpoint resource, select the endpoint, and grant `CAN_QUERY`. ### 2b. Add the Environment Variable ```yaml env: - name: DATABRICKS_SERVING_ENDPOINT_NAME valueFrom: serving-endpoint ``` > **Critical — env var name mismatch:** The Databricks Apps platform injects `SERVING_ENDPOINT=<name>` via the resource binding, but the AppKit Serving plugin reads `DATABRICKS_SERVING_ENDPOINT_NAME`. You MUST explicitly declare the env var in `app.yaml` with the name `DATABRICKS_SERVING_ENDPOINT_NAME` and use `valueFrom: serving-endpoint` so the platform maps the resource value to the name the plugin expects. ### 2c. Local Development (.env) For local dev, set the env var directly: ```env DATABRICKS_SERVING_ENDPOINT_NAME=<your-endpoint-name> ``` Local dev also requires Databricks authentication (CLI profile or `DATABRICKS_HOST` + `DATABRICKS_TOKEN` in `.env`). **Gate:** `app.yaml` contains the `DATABRICKS_SERVING_ENDPOINT_NAME` env var with `valueFrom: serving-endpoint`. --- ## Step 3: Register in server/server.ts > **If `serving` is undefined in your installed AppKit version**, stop here and read [references/custom-proxy-fallback.md](references/custom-proxy-fallback.md) instead of proceeding with Step 3. The bundler will silently accept a nonexistent `serving` import and fail at client build or runtime. ### 3a. Single Endpoint (Default) ```typescript import { createApp, server, serving } from "@databricks/appkit"; await createApp({ plugins: [ server(), serving(), ], }); ``` With no configuration, the plugin reads `DATABRICKS_SERVING_ENDPOINT_NAME` from the environment and registers it under the `default` alias. Routes are auto-registered at `/api/serving/invoke` and `/api/serving/stream`. ### 3b. Multiple Endpoints with Aliases ```typescript await createApp({ plugins: [ server(), serving({ endpoints: { agent: { env: "DATABRICKS_SERVING_ENDPOINT_NAME" }, classifier: { env: "DATABRICKS_CLASSIFIER_ENDPOINT_NAME" }, }, timeout: 120000, }), ], }); ``` Each alias gets its own route pair: `/api/serving/agent/invoke`, `/api/serving/agent/stream`, etc. ### 3c. Targeting a Specific Model If the endpoint serves multiple models, bypass traffic routing: ```typescript serving({ endpoints: { agent: { env: "DATABRICKS_SERVING_ENDPOINT_NAME", servedModel: "my-agent-v2", }, }, }); ``` > **Unlike Lakebase wiring, no `server.extend()` routes are needed here.** The plugin auto-registers the HTTP endpoints. The `server.extend()` pattern is only needed if you want a custom proxy route (Step 6). **Gate:** `npm run build` passes with the `serving()` plugin imported and configured. --- ## Step 4: Wire Frontend — Streaming Chat **You MUST read [references/chat-ui-patterns.md](references/chat-ui-patterns.md)** for the full conversation state management pattern before building a chat UI. ### 4a. Basic Streaming Hook ```tsx import { useServingStream } from "@databricks/appkit-ui/react"; function AgentChat() { const { stream, chunks, streaming, error, reset } = useServingStream( { messages: [{ role: "user", content: userInput }] }, { alias: "agent", onComplete: (finalChunks) => { console.log("Agent done:", finalChunks.length, "chunks"); }, }, ); return ( <> <button onClick={stream} disabled={streaming}>Ask Agent</button> <button onClick={reset}>Clear</button> {chunks.map((c, i) => <pre key={i}>{JSON.stringify(c)}</pre>)} {error && <p className="text-red-600">{error}</p>} </> ); } ``` ### 4b. Conversation State Management `useServingStream` is **stateless** — it doesn't track history across calls. For multi-turn chat, manage messages in `useState` and pass the full array on each request: ```tsx const [messages, setMessages] = useState<{ role: string; content: string }[]>([]); const { stream, chunks, streaming, error, reset } = useServingStream( { messages: [ ...messages, { role: "user", content: currentInput }, ], }, { alias: "agent", onComplete: (finalChunks) => { const text = finalChunks .map((c: any) => c.choices?.[0]?.delta?.content ?? "") .join(""); setMessages((prev) => [ ...prev, { role: "user", content: currentInput }, { role: "assistant", content: text }, ]); reset(); }, }, ); ``` Key rules: - Pass the **full conversation history** on every call so the agent has context - Append the assistant message **in `onComplete`**, not during streaming - Call `reset()` after capturing the response to clear chunks for the next turn - For long conversations, consider a sliding window to stay within token limits > **Request body timing:** The first argument to `useServingStream` is the request body sent when `stream()` is called. Verify the hook re-reads this argument at call time, not at mount time. If the messages array appears stale (always sends the initial empty array), use a `useRef` to hold current messages — see [references/chat-ui-patterns.md](references/chat-ui-patterns.md) for the `useRef` fallback pattern. Test multi-turn early: send two messages and confirm the second request includes the first exchange. ### 4c. Streaming Chunk Format Databricks emits streaming chunks in **two different shapes** depending on how the endpoint was deployed: - **Databricks Responses API** (agents deployed via `databricks.agents.deploy()` with `ResponsesAgent`): `{ type: "response.output_text.delta", delta: "..." }` - **OpenAI Chat Completion** (custom Model Serving, OpenAI-compatible pyfunc): `chunk.choices[0].delta.content` A parser that reads only `choices[0].delta.content` will silently produce empty output against a Responses-API endpoint. Before writing the parser, **read [references/sse-format-patterns.md](references/sse-format-patterns.md)** for the `curl` pre-deploy format test and a dual-format parser you can copy directly. Minimal dual-format extractor (use against both endpoint types): ```typescript function extractDelta(chunk: any): string { if (chunk.type === "response.output_text.delta") return chunk.delta ?? ""; return chunk.choices?.[0]?.delta?.content ?? ""; } ``` For the full SSE reader (buffering, `[DONE]` handling, unknown-chunk warnings, and the `curl` pre-deploy format test), read [references/sse-format-patterns.md](references/sse-format-patterns.md).