building-chatgpt-apps

name

building-chatgpt-apps

description

Guides creation of ChatGPT Apps with interactive widgets using the Apps SDK and MCP servers. Use when building ChatGPT custom apps with visual UI components, embedded widgets, or rich interactive experiences. Covers widget architecture, MCP server setup with FastMCP, response metadata, and Developer Mode configuration. NOT when building standard MCP servers without widgets (use building-mcp-servers skill instead).

Apps SDK Development Guide

Overview

Create ChatGPT Apps with interactive widgets that render rich UI inside ChatGPT conversations. The Apps SDK combines MCP servers (providing tools) with embedded HTML widgets that communicate via the window.openai API.

Official Documentation: https://developers.openai.com/apps-sdk/ Examples Repository: https://github.com/openai/openai-apps-sdk-examples

Three-Layer Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        ChatGPT UI                                │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │                    Widget (iframe)                          ││
│  │   React/Vanilla JS + CSS                                    ││
│  │   window.openai.* APIs for host communication               ││
│  └─────────────────────────────────────────────────────────────┘│
│                              │                                   │
│                              ▼                                   │
│                     ChatGPT Backend                              │
│                              │                                   │
│                              ▼                                   │
│              MCP Server (HTTP/SSE)                               │
│              - Tools: exposed actions                            │
│              - Resources: widget HTML (text/html+skybridge)      │
│              - Response: structuredContent + _meta               │
└─────────────────────────────────────────────────────────────────┘

Flow: User prompt → Model invokes tool → Server returns structured data → Widget renders → Model narrates result

---

window.openai API Reference

Complete API surface for widget-host communication:

State & Data Access

Property	Purpose	Example
toolInput	Arguments passed when tool was invoked	window.openai.toolInput
toolOutput	Structured content from server response	window.openai.toolOutput
toolResponseMetadata	Server _meta hidden from model	window.openai.toolResponseMetadata
widgetState	Persisted UI state snapshot	window.openai.widgetState

Runtime Actions

Method	Purpose	Example
callTool(name, args)	Invoke another MCP tool	await window.openai.callTool("refresh_data", {city})
sendFollowUpMessage({prompt})	Insert conversational message	await window.openai.sendFollowUpMessage({prompt: "Summarize"})
setWidgetState(state)	Persist state synchronously	window.openai.setWidgetState({favorites: []})
uploadFile(file)	Upload user file (PNG/JPEG/WebP)	const {fileId} = await window.openai.uploadFile(file)
getFileDownloadUrl({fileId})	Get temporary download URL	const {downloadUrl} = await window.openai.getFileDownloadUrl({fileId})

Layout Control

Method	Purpose
requestDisplayMode({mode})	Request layout: "inline", "pip", "fullscreen"
requestModal(...)	Spawn ChatGPT-owned modal
notifyIntrinsicHeight(...)	Report dynamic height to avoid clipping
openExternal({href})	Open vetted external link
requestClose()	Close widget from UI

Context Properties (Read-Only)

window.openai.theme        // "light" | "dark"
window.openai.displayMode  // "inline" | "pip" | "fullscreen"
window.openai.maxHeight    // container height in px
window.openai.safeArea     // viewport constraints
window.openai.userAgent    // browser identifier
window.openai.locale       // "en-US", "es-ES", etc.

Code Examples

sendFollowUpMessage (Best for Action Buttons):

async function suggestAction(prompt) {
  if (window.openai?.sendFollowUpMessage) {
    await window.openai.sendFollowUpMessage({ prompt });
  }
}
// Usage: suggestAction('Summarize this chapter');

callTool (For Tool Chaining):

async function refreshData(city) {
  if (window.openai?.callTool) {
    const result = await window.openai.callTool("refresh_list", { city });
    // result contains fresh structuredContent
  }
}

Note: callTool requires tool metadata "openai/widgetAccessible": true.

---

Tool Definition with Metadata

Tools require proper metadata to enable widget rendering:

TypeScript/Node.js Pattern

server.registerTool(
  "kanban-board",
  {
    title: "Show Kanban Board",
    inputSchema: { workspace: z.string() },
    annotations: {
      readOnlyHint: true,      // Skip confirmation for read operations
      destructiveHint: false,  // Set true for delete/modify actions
      openWorldHint: false     // Set true if publishing externally
    },
    _meta: {
      "openai/outputTemplate": "ui://widget/kanban.html",  // Required
      "openai/widgetAccessible": true,  // Enable callTool from widget
      "openai/visibility": "public",    // "private" hides from model
      "openai/toolInvocation/invoking": "Loading board…",
      "openai/toolInvocation/invoked": "Board ready."
    }
  },
  async ({ workspace }) => {
    const tasks = await db.fetchTasks(workspace);
    return {
      // Data for model narration (keep concise)
      structuredContent: {
        columns: ["todo", "in-progress", "done"].map(status => ({
          id: status,
          : tasks.( t. === status)
        }))
      },
      
      : [{ : , :  }],
      
      : {
        : .(tasks.( [t., t])),
        :  ().()
      }
    };
  }
);

Python/FastMCP Pattern

from mcp.server.fastmcp import FastMCP
import mcp.types as types

mcp = FastMCP("My App")

@mcp.tool(
    annotations={
        "title": "Show Dashboard",
        "readOnlyHint": True,
        "openWorldHint": False,
    },
    _meta={
        "openai/outputTemplate": "ui://widget/dashboard.html",
        "openai/widgetAccessible": True,
    },
)
def show_dashboard(user_id: str) -> types.CallToolResult:
    data = fetch_user_data(user_id)
    return types.CallToolResult(
        content=[types.TextContent(type="text", text="Dashboard loaded.")],
        structuredContent={"summary": data.summary},
        _meta={"fullData": data.dict(), "timestamp": datetime.now().isoformat()}
    )

Tool Metadata Reference

Key	Type	Purpose
openai/outputTemplate	string (URI)	Required. Resource URI for widget HTML
openai/widgetAccessible	boolean	Enable window.openai.callTool from widget
openai/visibility	"public" or "private"	Hide tool from model but keep widget-callable
openai/toolInvocation/invoking	string (≤64 chars)	Status text while executing
openai/toolInvocation/invoked	string (≤64 chars)	Status text when complete
openai/fileParams	string[]	Input fields accepting file objects

Response Payload Structure

Field	Visibility	Purpose
structuredContent	Model + Widget	Concise JSON for model narration
content	Model + Widget	Optional markdown/plaintext
_meta	Widget Only	Sensitive/large data hidden from model

Widget Resource Registration

Resources define widget HTML with proper MIME type:

TypeScript Pattern

server.registerResource(
  "kanban-widget",
  "ui://widget/kanban.html",
  {},
  async () => ({
    contents: [{
      uri: "ui://widget/kanban.html",
      mimeType: "text/html+skybridge",  // Required for widget rendering
      text: WIDGET_HTML,
      _meta: {
        "openai/widgetPrefersBorder": true,
        "openai/widgetDomain": "https://chatgpt.com",
        "openai/widgetCSP": {
          connect_domains: ["https://api.example.com"],
          resource_domains: ["https://*.oaistatic.com"],
          frame_domains: []
        }
      }
    }]
  })
);

Python Pattern

@mcp.resource(
    uri="ui://widget/{widget_name}.html",
    name="Widget Resource",
    mime_type="text/html+skybridge"
)
def widget_resource(widget_name: str) -> str:
    return WIDGETS[widget_name]["html"]

Widget Resource Metadata

Key	Purpose
openai/widgetPrefersBorder	Visual border preference
openai/widgetDomain	Dedicated origin for API allowlisting
openai/widgetCSP	Security boundaries (connect, resource, frame domains)
openai/widgetDescription	Summary shown when widget loads

---

React Hooks for Widgets

Official patterns for React-based widgets:

useOpenAiGlobal (Reactive State Subscription)

import { useSyncExternalStore } from "react";

export function useOpenAiGlobal<K extends keyof OpenAiGlobals>(
  key: K
): OpenAiGlobals[K] {
  return useSyncExternalStore(
    (onChange) => {
      const handle = (e: CustomEvent) => {
        if (e.detail.globals[key] !== undefined) onChange();
      };
      window.addEventListener("SET_GLOBALS", handle, { passive: true });
      return () => window.removeEventListener("SET_GLOBALS", handle);
    },
    () => window.openai?.[key]
  );
}

useWidgetState (Persistent Component State)

export function useWidgetState<T>(defaultState?: T | (() => T)) {
  const widgetStateFromWindow = useOpenAiGlobal("widgetState") as T;

  const [state, _setState] = useState<T | null>(() =>
    widgetStateFromWindow ?? (typeof defaultState === "function"
      ? defaultState()
      : defaultState ?? null)
  );

  useEffect(() => {
    _setState(widgetStateFromWindow);
  }, [widgetStateFromWindow]);

  const setState = useCallback((newState: T | ((prev: T) => T)) => {
    _setState((prev) => {
      const next = typeof newState === "function" ? newState(prev) : newState;
      window.openai?.setWidgetState(next);
      return next;
    });
  }, []);

  return [state, setState] as const;
}

Helper Hooks

export function useToolInput() {
  return useOpenAiGlobal("toolInput");
}

export function useToolOutput() {
  return useOpenAiGlobal("toolOutput");
}

export function useToolResponseMetadata() {
  return useOpenAiGlobal("toolResponseMetadata");
}

---

Quick Start

1. Create MCP server with tools and widget resources
2. Define widget HTML with window.openai communication
3. Set tool metadata with openai/outputTemplate pointing to widget
4. Return structured responses with structuredContent + _meta
5. Expose via ngrok for ChatGPT access
6. Register in ChatGPT Developer Mode settings

---

Widget HTML Requirements

Basic Widget Template

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>My Widget</title>
  <style>
    * { margin: 0; padding: 0; box-sizing: border-box; }
    body {
      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
      background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
      min-height: 100vh;
      padding: 24px;
      color: white;
    }
    .container { max-width: 600px; margin: 0 auto; }
    .card {
      background: rgba(255,255,255,0.95);
      : ;
      : ;
      : ;
      :    (,,,);
    }
     {
      : ;
      : white;
      : none;
      :  ;
      : ;
      : pointer;
      : ;
    }
     { : ; }
  


  
    
      Widget Title
      Widget content here
      Click Me
    
  
  

Key Widget Rules

1. Always check window.openai.toolOutput before calling
2. Use inline styles - external CSS may not load reliably
3. Keep widgets self-contained - all HTML/CSS/JS in one file
4. Test with actual ChatGPT - browser preview won't have window.openai

---

MCP Server Setup (FastMCP Python)

Project Structure

my_chatgpt_app/
├── main.py              # FastMCP server with widgets
├── requirements.txt     # Dependencies
└── .env                 # Environment variables

requirements.txt

mcp[cli]>=1.9.2
uvicorn>=0.32.0
httpx>=0.28.0
python-dotenv>=1.0.0

main.py Template

import mcp.types as types
from mcp.server.fastmcp import FastMCP

# Widget MIME type for ChatGPT
MIME_TYPE = "text/html+skybridge"

# Define your widget HTML
MY_WIDGET = '''<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <style>
    body { font-family: sans-serif; padding: 20px; }
    .container { max-width: 500px; margin: 0 auto; }
  </style>
</head>
<body>
  <div class="container">
    <h1>Hello from Widget!</h1>
    <p>This content renders inside ChatGPT.</p>
  </div>
</body>
</html>'''

# Widget registry
WIDGETS = {
    "main-widget": {
        "uri": "ui://widget/main.html",
        "html": MY_WIDGET,
        "title": "My Widget",
    },
}

# Create FastMCP server
mcp = FastMCP("My ChatGPT App")


@mcp.resource(
    uri="ui://widget/{widget_name}.html",
    name="Widget Resource",
    mime_type=MIME_TYPE
)
def widget_resource(widget_name: str) -> str:
    """Serve widget HTML."""
    widget_key = f"{widget_name}"
    if widget_key in WIDGETS:
        return WIDGETS[widget_key]["html"]
    return WIDGETS["main-widget"]["html"]


def _embedded_widget_resource(widget_id: str) -> types.EmbeddedResource:
    """Create embedded widget resource for tool response."""
    widget = WIDGETS[widget_id]
     types.EmbeddedResource(
        =,
        resource=types.TextResourceContents(
            uri=widget[],
            mimeType=MIME_TYPE,
            text=widget[],
            title=widget[],
        ),
    )


 () -> :
    
     {
        : {
            : WIDGETS[][],
            : WIDGETS[][]
        }
    }


 () -> :
    
     {
        : _embedded_widget_resource()
    }



 () -> types.CallToolResult:
    
     types.CallToolResult(
        content=[
            types.TextContent(
                =,
                text=
            )
        ],
        structuredContent={
            : ,
            : 
        },
        _meta=response_meta(),
    )


 __name__ == :
     uvicorn
    ()
    ()
    uvicorn.run(
        ,
        host=,
        port=,
        reload=
    )

---

Response Metadata Format

Critical: _meta["openai.com/widget"]

Tool responses MUST include widget metadata:

types.CallToolResult(
    content=[types.TextContent(type="text", text="...")],
    structuredContent={"key": "value"},  # Data for widget
    _meta={
        "openai.com/widget": types.EmbeddedResource(
            type="resource",
            resource=types.TextResourceContents(
                uri="ui://widget/my-widget.html",
                mimeType="text/html+skybridge",
                text=WIDGET_HTML,
                title="My Widget",
            ),
        )
    },
)

structuredContent

Data passed to the widget. The widget can access this via window.openai APIs.

---

Development Setup

1. Start Local Server

cd my_chatgpt_app
python main.py
# Server runs on http://localhost:8001

2. Start ngrok Tunnel

ngrok http 8001
# Get URL like: https://abc123.ngrok-free.app

3. Register in ChatGPT

1. Go to https://chatgpt.com/apps
2. Click Settings (gear icon)
3. Enable Developer mode
4. Click Create app
5. Fill in:
   • Name: Your App Name
   • MCP Server URL: https://abc123.ngrok-free.app/mcp
   • Authentication: No Auth (for development)
6. Check "I understand and want to continue"
7. Click Create

4. Test the App

1. Start a new chat in ChatGPT
2. Type @ to see available apps
3. Select your app
4. Ask it to use your tool

---

OAuth 2.1 Authentication

For apps requiring user authentication:

Protected Resource Metadata

Host at /.well-known/oauth-protected-resource:

{
  "resource": "https://your-mcp.example.com",
  "authorization_servers": ["https://auth.yourcompany.com"],
  "scopes_supported": ["files:read", "files:write"]
}

Tool Security Schemes

securitySchemes: [
  { type: "noauth" },  // Public access
  { type: "oauth2", scopes: ["docs.read"] }  // Authenticated access
]

Token Validation

Servers must:

• Validate signature/issuer via authorization server's JWKS
• Reject expired tokens (exp/nbf claims)
• Confirm audience matches (aud claim)
• Return 401 with WWW-Authenticate header on failure

Error Response (Triggers Auth UI)

{
  "_meta": {
    "mcp/www_authenticate": [
      "Bearer resource_metadata=\"https://.../.well-known/oauth-protected-resource\", error=\"insufficient_scope\""
    ]
  },
  "isError": true
}

---

Common Issues and Solutions

Issue	Cause	Solution
Widget shows "Loading..."	HTML not delivered correctly	Check _meta with openai/outputTemplate, verify MIME type
Widget not updating	Aggressive caching	Delete app, restart ngrok with new URL, create new app
JavaScript errors	window.openai unavailable	Always use optional chaining: window.openai?.methodName
Tool not in @mentions	MCP server disconnected	Verify ngrok URL, check server logs for ListToolsRequest
callTool not working	Widget access disabled	Add "openai/widgetAccessible": true to tool metadata

---

Decision Logic

Situation	Pattern
Simple display widget	Vanilla HTML + CSS + JS
Complex interactive UI	React + hooks (useWidgetState)
Multi-tool workflow	callTool from widget with widgetAccessible: true
User suggestions	sendFollowUpMessage (most reliable)
Persistent UI state	setWidgetState + widgetState
Large data payloads	Send via _meta (hidden from model)
User authentication	OAuth 2.1 with security schemes
Display mode changes	requestDisplayMode (inline/pip/fullscreen)

---

Safety

NEVER

• Embed API keys, tokens, or secrets in structuredContent, content, or _meta
• Rely on userAgent or locale hints for authorization decisions
• Expose destructive operations without user intent verification

ALWAYS

• Validate tokens server-side (ChatGPT assumes tokens are untrusted)
• Use environment variables for secrets
• Design handlers as idempotent (model may retry)
• Check window.openai existence before calling methods

---

References

• Official Docs
• Examples Repo
• Complete Template - Ready-to-use server + widget
• Widget Patterns - HTML/CSS/JS examples
• Response Structure - Metadata format details
• Debugging Guide - Troubleshooting common issues