add-integration

name	add-integration
description	Use when adding a new external API integration to Switchboard, scaffolding an integration adapter, or deciding between SDK vs raw HTTP for a new service. Not for modifying existing integrations or fixing bugs in current adapters.
metadata	{"author":"switchboard","version":"1.0"}

Add Integration

Full lifecycle for adding a new integration adapter to Switchboard. See AGENTS.md for interface contracts, project structure, and conventions referenced below.

1. Research the Target API

Before writing code, answer these questions:

• Auth model: API key, OAuth (which grant type?), session tokens, other?
• API shape: REST, GraphQL, or mixed?
• Go SDK: Does a maintained, well-typed Go SDK exist? Does it cover the endpoints you need?
• Rate limits: Documented? Per-endpoint or global? Headers for remaining quota?
• Pagination: Cursor-based, offset, or link-header? Consistent across endpoints?
• Error format: Structured JSON errors or plain text? Status code conventions?
• Scope: Which API resources/operations are needed? Group by domain (e.g., issues, projects, users)
• Parent types: Do entities have polymorphic parent relationships? Do different parent types require different mutation commands? (e.g., Notion block parents use listAfter, collection parents use setParent)

Undocumented APIs: When working with internal or undocumented APIs (no OpenAPI spec, no public docs), intercept the first-party client's network traffic to discover expected payload shapes. For web apps: monkey-patch window.fetch in Chrome DevTools to log request bodies. For mobile apps: use a MITM proxy. This is often the only way to discover commands, required fields, and transaction formats.

2. Design Decisions

SDK vs Raw HTTP

Criteria	Use Typed SDK	Use Raw HTTP
Go SDK exists and maintained	Yes	-
SDK covers needed endpoints	Yes	-
No Go SDK available	-	Yes
SDK exists but poorly typed or incomplete	-	Yes
API is GraphQL	-	Yes (hand-rolled queries)

Existing precedent: GitHub, Datadog, Slack use typed SDKs. Linear, Sentry, Metabase use raw HTTP.

Tool Granularity and File Organization

One tool per API operation. Follow naming and dispatch conventions in AGENTS.md > Conventions and Patterns.

Tool count	Structure
< 30	1-2 handler files + tools.go + <name>.go + <name>_test.go
30-60	3-5 handler files split by domain (see sentry/)
60+	5+ handler files (see github/, datadog/)

Tool Description Quality (Search Discoverability)

Tool descriptions are scored by a TF-IDF search engine with synonym expansion. They're the ONLY text an LLM sees when deciding which tool to use. Write them for discoverability, not just accuracy.

Three-tier pattern (from the GitHub adapter — the gold standard):

1. Entry points — tools users search for first:

   "List error and exception issues for a project. Start here for error tracking, debugging, and finding unresolved bugs or crashes."
   

2. Drill-down tools — used after entry points:

   "Get details of a specific error issue, including stacktrace and debugging context. Use after list_issues."
   

3. Action tools — mutations with chaining hints:

   "Update an error issue (resolve, assign, triage). Use after list_issues or get_issue."
   

Rules:

• Include domain keywords users would search for. "List issues" is invisible to someone searching "find bugs" or "error tracking". Add the words: "List error and exception issues... bugs, crashes, unresolved problems."
• Include synonym-group words in descriptions. The search engine expands "ticket" to {"ticket","issue","task","bug"}, but the tool still needs at least one variant in its description to score. Don't rely on synonyms alone.
• Scope descriptions to prevent false positives. HomeAssistant tools should say "smart home" not just "state". CI/CD tools should say "CI/CD pipeline" not just "run" or "logs". Generic words cause tools to surface as noise in unrelated queries.
• Add workflow hints. "Start here for..." on entry points. "Use after X..." on drill-down tools. "Preferred over X because..." when tools overlap.
• Don't pad with stop words. Words like "a", "the", "to", "for" are filtered by the search engine. Every word in the description should carry meaning.
• Include both singular and plural forms. The search engine does exact token matching with NO stemming. "errors" ≠ "error". If users might search for either form, include both: "List errors and exceptions" covers both "sentry errors" and "sentry error". Check the synonym groups in server/search.go — common plurals are covered there, but new domain words need explicit plural coverage either in the description or as a synonym group.

Entry-point guidance (MUST): Every integration's first/primary tool MUST include "Start here" in its description. This is wayfinding — models use it to orient when browsing an unfamiliar integration. Examples:

• "List all schemas in the database. Start here for schema discovery."
• "List all projects in the PostHog organization. Start here to discover projects."

Naming deviation callouts: When the integration uses non-standard verb prefixes (e.g., Notion uses retrieve_* where others use get_*), add the standard verb in parentheses: "Retrieve (get) a page's metadata". This helps both search scoring and models that skip search.

Noun synonym awareness: The search engine has noun synonym groups (table/tables, label/labels/tag/tags, diff/patch/changes, database/databases/db, etc.) in server/search.go. When adding tools with domain-specific nouns, check if your nouns are covered. If not, either add a synonym group or include both forms in the description.

IDF dilution warning: Avoid creating synonym groups where the union covers too many tools (>60). High-union groups dilute IDF scores and can cause regressions. Verb groups are safe because MAX-per-word scoring means rare synonyms carry the score. Noun groups need more care — test with /search-benchmark before and after.

Anti-patterns:

• "List issues for a project" — too generic, no domain keywords
• "Get a specific message by ID" — no "email", "mail", or "read"
• "List events with optional filters" — what kind of events? For what purpose?
• Missing "Start here" on entry-point tool — model has no wayfinding signal

Verify with benchmark: After adding tools, run /search-benchmark to check that your tools surface for natural-language queries users would actually type.

Auth Pattern

Auth type	Pattern	Example adapter
API key / token	Header in doRequest	metabase/ (x-api-key), sentry/ (Bearer)
OAuth token via SDK	SDK transport/config	github/ (oauth2), datadog/ (context keys)
Session token + cookie	Custom http.RoundTripper	slack/ (cookieTransport)
OAuth setup flow	Separate oauth.go file	github/, linear/, sentry/, slack/

Add an OAuth flow when the API supports it and you want guided credential setup in the Web UI. Get basic token auth working first. Grant type depends on the API: Device Flow for headless, PKCE for browser-redirect. Add a corresponding setup page in web/templates/pages/.

3. Implementation

Reference AGENTS.md > Adding a New Integration for the 7-step mechanical checklist. Focus here on judgment calls:

Configure as Defensive Validation Boundary

Configure() is where you reject invalid state. Validate eagerly, fail on missing credentials — never let an unconfigured adapter reach Execute().

func (x *myapi) Configure(creds mcp.Credentials) error {
    x.apiKey = creds["api_key"]
    if x.apiKey == "" {
        return fmt.Errorf("myapi: api_key is required")
    }
    // For services with a fixed base URL, hardcode a default (see sentry/)
    // For services where URL varies, require it (see metabase/)
    if v := creds["base_url"]; v != "" {
        x.baseURL = strings.TrimRight(v, "/")
    }
    return nil
}

Healthy() Check

Implement a lightweight API call that verifies credentials work (e.g., "get current user" or "list with limit=1"). Must handle the case where Configure() hasn't been called yet (nil client) — return false, don't panic.

Error Handling

Follow AGENTS.md > Error Handling. Key judgment: surface errors to the caller — never swallow them, never add fallback defaults.

When to Add Custom Helpers

Add integration-specific helpers when a pattern repeats 3+ times within an adapter:

• Org/workspace slug injection (see integrations/sentry/org())
• Entity ID resolution by name (see integrations/linear/resolveTeamID())
• Query string building from optional params (see integrations/sentry/queryEncode())

Note: arg helpers are shared from args.go — use mcp.NewArgs(args) reader for bulk extraction or standalone mcp.ArgStr/mcp.ArgInt/etc. for conditional fields. NEVER define local argStr/argInt in adapters. Use r.OptInt("page", 1) for pagination defaults. See docs/go-anti-patterns.md for extraction pitfalls. Result constructors (mcp.JSONResult, mcp.RawResult, mcp.ErrResult) are shared from the root package. Some adapters wrap mcp.ErrResult in a local errResult to inject retry semantics.

4. Testing Requirements

Every adapter must have these test categories (see existing *_test.go files):

• Constructor: New() returns valid integration, Name() matches
• Configure success: Valid credentials accepted
• Configure failures: One test per required credential, verifying error message
• Tools metadata: All have Name + Description, prefix matches Name(), no duplicates. Descriptions follow the three-tier pattern (see "Tool Description Quality" above)
• Search discoverability: Run /search-benchmark — new tools surface for natural-language queries. Check synonym coverage with existing synonymGroups in server/search.go
• Dispatch parity (non-negotiable):
  • TestDispatchMap_AllToolsCovered — every Tools() entry has a dispatch handler
  • TestDispatchMap_NoOrphanHandlers — every dispatch key has a ToolDefinition
• Execute unknown tool: Returns IsError: true, "unknown tool" in Data
• HTTP helpers: httptest.NewServer for success, API errors (>=400), 204 no-content
• Arg extraction: Uses shared mcp.NewArgs(args) reader with r.Err() check — type coercion is tested in root args_test.go. TestNewArgs_ErrCheckParity automatically covers new adapters

5. Wiring and Verification

Follow AGENTS.md > Adding a New Integration steps 6-7 (register + config defaults), then verify:

1. go build ./... && go test ./... && go vet ./... && go tool golangci-lint run
2. Smoke test: start server, call search for new integration tools, execute one

6. Field Compaction

New adapters should implement FieldCompactionIntegration to keep list/search responses compact.

Contract: implement CompactSpec(toolName ToolName) ([]CompactField, bool) on the adapter struct. The shared compact package loads specs from an embedded compact.yaml next to your adapter Go file — there's no per-adapter loader to write.

Optional: implement MaxBytes(toolName ToolName) (int, bool) if you want per-tool response size caps declared in the same YAML.

Token Budget Principle

Optimize specs for fewest total tokens across the entire task workflow, not smallest single response. A field that prevents an N+1 drill-down saves ~5KB per item even if it costs 50 bytes in the compacted list. Distribution of tokens across 1 or N calls doesn't matter as long as N is small enough that network latency doesn't dominate timing. The goal is a finite minimum token budget for any given workflow — get as close to it as possible.

Example: requested_reviewers[].login adds ~80 bytes per PR to the compacted list, but without it "which PRs need review?" requires a separate list_requested_reviewers call per PR (~3KB each). For 20 open PRs: +1.6KB in compacted list vs. +60KB in drill-down calls.

Checklist

• Create integrations/<name>/compact.yaml. Header comment + version: 1 + tools.<tool_name>.spec: [<dot-notation paths>]. Copy integrations/linear/compact.yaml as the reference shape.
• In the adapter Go file, wire the loader:

  //go:embed compact.yaml
  var compactYAML []byte
  
  var compactResult = compact.MustLoadWithOverlay("<name>", compactYAML, compact.Options{Strict: false})
  var fieldCompactionSpecs = compactResult.Specs
  var maxBytesByTool = compactResult.MaxBytes
  

• Design field compaction specs using the spec design questions below
• Add specs for all read tools (list, search, AND single-record get) — keep identifiers, names, states, dates, counts, URLs; drop nested full objects, permissions, avatars, node_ids, CRDT noise
• Implement CompactSpec(toolName ToolName) ([]CompactField, bool) on the adapter struct (returns fieldCompactionSpecs[toolName])
• Implement MaxBytes(toolName ToolName) (int, bool) on the adapter struct (returns maxBytesByTool[toolName])
• Add compile-time assertions:

  var (
      _ mcp.FieldCompactionIntegration = (*myapi)(nil)
      _ mcp.ToolMaxBytesIntegration    = (*myapi)(nil)
  )
  

• Add TestFieldCompactionSpecs_NoOrphanSpecs — every spec key must exist in dispatch
• Unwrap SDK list responses to the inner slice (e.g., resp.Items not resp) so field compaction operates on the array directly
• Mutation tools (create/update/delete) should NOT have field compaction specs — return full confirmation responses

Spec Design Questions

For each tool's spec, verify against these questions before finalizing:

1. Routing sufficiency: Can the LLM decide which item to drill into from the compacted list alone? If it must open every item to answer "which PR broke the build?", the spec is missing a field.
2. Workflow gaps: Trace common workflows (triage, review, debug CI). Does each workflow have the fields to complete without per-item get calls? Missing a field like requested_reviewers means "which PRs need review?" requires N extra calls.
3. Dead weight: Would you only look at this field after already deciding to open the full record? If yes, drop it — it's noise in a list context. Also watch for phantom fields — fields that exist in SDK structs but are only populated by Get endpoints (e.g., additions/deletions on GitHub's List PRs API return 0/null).
4. Field dependencies: Do included fields make sense alone? status without conclusion in CI runs is incomplete. additions without deletions in PRs is half the story. Include paired fields together or not at all.
5. Follow-up keys: Does the LLM have the identifiers it needs to make follow-up API calls? Verify that id, number, or sha — whatever the get tool requires — is included.

Canonical example: integrations/linear/compact.yaml (schema + header conventions) and integrations/linear/linear.go (loader wiring + interface assertions).

Multi-View Tools

When to declare views (gated): One tool, two or more useful projections of the same fetch. Notion's get_page_content is the canonical case — same handler output, TOC view (~1KB) for navigation and full view (10-50KB) for reading. If the handler code would be identical for each projection, it's a view. If you'd write two handlers, write two tools.

When NOT to use views: Single-shape tools (most tools). A flat spec: is correct. Adding views: for a single shape inflates YAML without preventing any failure.

What failure this prevents: Without views, every caller pays the cost of the largest useful projection on every call. With views, the LLM picks the shape it needs — default is the smallest useful, and the response embeds discovery of larger views via the _more envelope.

Schema (only when 2+ projections of the same fetch are useful):

<tool_name>:
  views:
    <view_name>:
      spec: [<paths>]
      hint: "<one-line description shown in _more envelope>"
      formats: [json]                       # or [json, markdown]
    <other_view>:
      spec: [<paths>]
      hint: "..."
      formats: [json, markdown]
  default:
    view: <view_name>
    format: json

Rules (each prevents a specific failure):

• spec: and views: are mutually exclusive per tool — declaring both is rejected at load. Prevents: two sources of truth for one tool.
• default.view must be a key in views:. Prevents: silent fallback to an unknown projection.
• default.format must be in default.view's formats:. Prevents: default combo that produces an error envelope.
• Conservative default: pick the smallest useful view + JSON. Prevents: every caller paying for the largest shape.
• Inverted default is legitimate when the heavy view is what the LLM almost always wants next (e.g., retrieve_data_source precedes query_data_source, so schema-on-by-default is correct). Make the full view the default and expose the slim view as an escape hatch. Mixing inverted and conservative defaults in one file is informative — it shows readers when each applies. Prevents: forcing follow-up calls for the common case.
• Hint text is shown to the LLM in the _more envelope. Write it as a usage signal ("Use for navigation", "Use when you need page content"), not a description.

Adapter wiring: add the ToolViewsIntegration interface and pass Renderers if a view declares a format the framework defaults can't handle (e.g., domain-specific markdown):

var compactRenderers = map[compact.RenderKey]compact.Renderer{
    {Tool: "<tool_name>", View: "<view>", Format: compact.FormatMarkdown}: customRenderer,
}

var compactResult = compact.MustLoadWithOverlay("<name>", compactYAML, compact.Options{
    Strict:    false,
    Renderers: compactRenderers,
})

var viewSets = compactResult.Views

var _ compact.ToolViewsIntegration = (*myapi)(nil)

func (x *myapi) Views(toolName mcp.ToolName) (compact.ViewSet, bool) {
    vs, ok := viewSets[toolName]
    return vs, ok
}

JSON is always free. Markdown for object-shaped responses is also free — the framework's generic markdown formatter produces definition lists and tables. Provide a custom renderer only when the generic output isn't good enough for the data shape (nested block trees, threaded comments). Declaring formats: [text] without a custom renderer fails at load.

Legacy markdown bridge pattern: when an existing RenderMarkdown function (renderXMD([]byte) (Markdown, bool)) already produces good output for a tool moving to views, wrap it as a typed renderer instead of rewriting:

func renderFullXMD(projected any) ([]byte, error) {
    data, err := json.Marshal(projected)
    if err != nil { return nil, fmt.Errorf("...: %w", err) }
    md, ok := renderXMD(data)
    if !ok { return nil, fmt.Errorf("...: declined to render") }
    return []byte(md), nil
}

Pipeline ownership is exclusive: a tool with views: bypasses MarkdownIntegration at runtime (views pipeline runs first). Remove the tool from markdownRenderers when it moves to views — leaving both registered creates two paths producing potentially different output. Prevents: silent path divergence after a renderer update.

Verify spec shapes against real data: unit tests with hand-built fixtures pass because the inputs are perfect. Live data exposes gaps — Notion v3 search, for instance, populates properties.title only when the block is resolved in recordMap; results without resolved blocks have only highlight.title. Build the binary, hit the tool with real arguments, and confirm the projected shape carries enough information before committing.

Test coverage required: every declared (view, format) combo needs a unit test. The pattern: register views in YAML → assert Views(toolName) returns the expected combos → assert each renderer produces non-empty output for a representative input. See integrations/notion/compact_specs_test.go for the canonical shape (TestPageContent_ViewsRegistered, TestPageContent_RenderersResolved, TestPageContent_FullMarkdownRendererBridge, TestSearch_TitlesMarkdownRenderer).

Anti-Patterns

Mistake	Correct approach
Defaulting missing credentials	Return error from Configure()
Returning Go error for API failures	Use ToolResult{IsError: true}, nil Go error
Skipping dispatch parity tests	Non-negotiable — tests catch tool/handler drift
Pre-building helpers before duplication	Wait for 3+ uses, then extract
Duplicating AGENTS.md content in handlers	Read AGENTS.md for conventions
Adding OAuth before basic auth works	Get token-based auth working first, add OAuth flow after
Returning full SDK wrapper for lists	Unwrap to inner slice, declare compaction specs
No default page size on list tools	Use sensible defaults (10-50 items)