api-utils

## Overview Utility exports from `@cyanheads/mcp-ts-core/utils`. Utilities with complex APIs have dedicated reference files; simpler utilities are documented inline below. **Tier 3** = optional peer dependency. Install as needed (e.g., `bun add js-yaml`). All Tier 3 methods are **async** (lazy-load deps on first call). **Context parameters.** Every helper below that takes a `context` accepts the handler `Context` as well as a `RequestContext` bag — pass `ctx` straight through, no slicing. ## References | Reference | Path | Covers | |:----------|:-----|:-------| | Formatting | `references/formatting.md` | `markdown()`, `MarkdownBuilder`, `diffFormatter`, `tableFormatter`, `treeFormatter` — builder patterns, option types, style variants, usage examples | | Parsing | `references/parsing.md` | `yamlParser`, `xmlParser`, `csvParser`, `jsonParser`, `pdfParser`, `dateParser`, `frontmatterParser` — method signatures, option types, peer deps, `Allow` flags, PDF workflows | | Security | `references/security.md` | `sanitization`, `RateLimiter`, `IdGenerator` — config types, method details, sensitive fields, usage examples | --- ## `@cyanheads/mcp-ts-core/utils` — network | Export | API | Notes | |:-------|:----|:------| | `fetchWithTimeout` | `(url, timeoutMs, context, options?: FetchWithTimeoutOptions) -> Promise<Response>` | Wraps `fetch` with `AbortController` timeout. `timeoutMs` bounds the **whole exchange**: on a 2xx carrying a body the returned `Response` is a passthrough wrapper that keeps the deadline armed until the body closes, errors, or is cancelled, so a stalled stream rejects the caller's `.text()`/`.json()` with the same `Timeout` error the header phase raises. `status`, `statusText`, `headers`, `url`, `redirected`, and `type` carry across the wrapper; the original body is locked by it, and bodyless/null-body responses (HEAD, 204/205/304) come back untouched. `FetchWithTimeoutOptions` extends `RequestInit` (minus `signal`) and adds `rejectPrivateIPs?: boolean`, `expectedStatuses?: number[]` (listed non-2xx statuses logged at `debug` not `error`, still thrown), `errorBodyLimit?: number` (bytes of a non-2xx body kept, default `500`), `errorHeaders?: string[]` (response headers copied onto `error.data.headers` on a non-2xx — same selector as `httpErrorFromResponse` below; `location` is selectable under `redirect: 'manual'` but does **not** compose with `rejectPrivateIPs`, whose per-hop branch consumes the 3xx before the throw path sees it), and `signal?: AbortSignal` (external cancellation — an abort on it throws `RequestCancelled` (-32011), logged at `info` and outside `withRetry`'s transient set, since the caller is gone and no retry can reach them). On a non-2xx, `error.data` carries `status`/`body` plus the legacy `statusCode`/`responseBody` aliases (identical values; consolidating in a future major); a body over `errorBodyLimit` is captured from both ends — 40% head, 60% tail, joined by `…[N bytes elided]…` — so a diagnostic behind a boilerplate preamble survives the cap, while a body still streaming at the 16 KiB scan ceiling stays head-only with a trailing `…`. SSRF guard (best-effort, not hard isolation): blocks RFC 1918, loopback, link-local, CGNAT, cloud metadata. DNS validation on Node, Bun, and Cloudflare Workers under `nodejs_compat`; hostname-only fallback otherwise. **Both resolvers are queried** — `resolve4`/`resolve6` (c-ares) and `lookup` (the system resolver, which is what reads `/etc/hosts`, split DNS, and NSS modules) — and a non-global answer from either rejects. Runtimes differ in which resolver the connection uses (Bun 1.4 moved `net.connect()` on Linux to `getaddrinfo` while leaving `dns.resolve*()` on c-ares), so checking one alone leaves a name the other can see unguarded; each probe settles independently, so a resolver absent from the runtime is skipped rather than fatal. Manual redirect following (max 5) with per-hop SSRF check. **DNS rebinding / TOCTOU gap** — the validation lookup and `fetch`'s own resolution are independent; pair with egress controls or a DNS-pinning fetch proxy for strong isolation. **Error/log redaction:** URLs written into thrown errors and log lines are reduced to `origin + pathname` — the query string (where API keys commonly ride: `?api-key=…`, `?api_key=…`) never reaches the client or the logs. The actual request still uses the full URL. | | `withRetry` | `<T>(fn: (attempt: RetryAttempt) => Promise<T>, options?: RetryOptions) -> Promise<T>` | Executes `fn` with exponential backoff. Retries on transient errors (`ServiceUnavailable`, `Timeout`, `RateLimited`); non-transient errors fail immediately. Honors an upstream `Retry-After` on `data.retryAfter` (delta-seconds or HTTP-date) over exponential backoff, capped at `maxDelayMs`; a requested wait beyond the cap fails fast rather than sleeping. On exhaustion, enriches the final error with attempt count in message and `data.retryAttempts`. **Place the retry boundary around the full pipeline** (fetch + parse), not just the network call. `RetryOptions`: `maxRetries` (default `3`), `baseDelayMs` (default `1000`), `maxDelayMs` (default `30000`), `jitter` (default `0.25`), `operation` (log label), `context` (RequestContext), `signal` (AbortSignal), `isTransient` (custom predicate), `deadlineMs` (total wall-clock budget — see below). | | `RetryAttempt` | `{ readonly signal: AbortSignal; readonly remainingMs: number }` | What `fn` receives each attempt. `signal` is `AbortSignal.any` over the `deadlineMs` clock and `options.signal`; `remainingMs` is what is left of the total budget as the attempt starts, never negative and `Number.POSITIVE_INFINITY` when no deadline is set — so `Math.min(perAttemptMs, remainingMs)` is correct either way. A zero-argument `fn` stays assignable, so existing callers compile unchanged. | | `deadlineMs` | `RetryOptions` field | One wall-clock budget across every attempt, backoff, and honored `Retry-After` — the bound `maxRetries` plus a per-attempt timeout cannot express. Four 30s attempts outlast a client's 60s request timeout, so the caller gets a transport timeout instead of the server's classified error. **Thread `attempt.signal` into the attempt's I/O** (`fetchWithTimeout(url, Math.min(30_000, remainingMs), ctx, { signal })`) or the deadline overshoots by one in-flight request. Clock is `AbortController` + `setTimeout` (never `AbortSignal.timeout()`, per the Bun realm mismatch), cleared on return — no timer outlives the call. Expiry rejects with `Timeout` (-32004) carrying `data: { reason: 'retry_deadline_exceeded', deadlineMs, elapsedMs, retryAttempts }` and the last attempt's error as `cause`; **one shape for every expiry**, including the `RequestCancelled` that an external-signal abort raises inside `fetchWithTimeout` and the raw abort reason a mid-backoff expiry would otherwise surface. No `retryable` flag (a narrower call can still succeed) and no `attempt` index (`retryAttempts` carries it). A backoff that would outlast the remaining budget fails fast with the expiry instead of sleeping into a certain timeout; an honored `Retry-After` that would outlast it takes the `maxDelayMs` exit instead — the attempt's error unchanged, `data.retryAfter` intact, since "wait the window the upstream named" is still the caller's action. **Three clocks stay distinct:** a caller abort on `options.signal` keeps precedence and rethrows unchanged (stamped `RequestCancelled` by the handler factory), a single attempt's timeout is `Timeout` with `errorSource: 'FetchTimeout'` and no `reason`, and the expiry is `Timeout` with the `reason`. Unset, behavior is identical to before — attempt counts, delays, log lines, and the exhausted-error shape untouched. Bounds **one** ladder: a tool making three upstream calls threads its own remaining budget into each. | | `defaultIsTransient` | `(error: unknown) -> boolean` | The predicate `withRetry` uses when `isTransient` is omitted: an `McpError` with a transient code (`ServiceUnavailable`, `Timeout`, `RateLimited`) unless it carries `data.retryable === false` or `data.reason === 'pacer_shed'`; any non-`McpError` throw is assumed transient. Exported so `isTransient` — which **replaces** the default outright — can compose instead of mirroring the transient set, which drifts silently when the framework's classification changes: `isTransient: (error) => !isMyBudgetRefusal(error) && defaultIsTransient(error)`, or the inverse `defaultIsTransient(error) \|\| isMyRetryableShape(error)`. The transient code set itself stays private (a module-level `Set` an exported binding could be mutated into framework-wide retry behavior). | | `httpErrorFromResponse` | `(response: Response, options?: HttpErrorFromResponseOptions) -> Promise<McpError>` | Maps an HTTP `Response` to a properly classified `McpError` — full status table including 401/403/408/422/429/5xx, body capture (truncated), `retry-after` header, optional `cause`. `error.data` carries `status`/`body` plus the legacy `statusCode`/`responseBody` aliases (identical values), so a consumer can classify either helper's error without knowing which raised it. Use this instead of hand-rolling `if (status === 429) ...` ladders. Reads the response body — `clone()` first if you need it elsewhere. **`error.data` is client-facing** — the framework forwards it verbatim as `structuredContent.error.data` — so the full upstream URL is **omitted by default**: a request URL routinely carries user input, internal identifiers, or an API key in its query string. `includeUrl: true` opts into `data.url` carrying the full `response.url`; with an empty `response.url` no key is added either way, and the message still names the host. Response headers are opt-in on the same footing: `errorHeaders: ['x-ratelimit-remaining-usd', 'x-request-id']` copies the named headers onto `data.headers` under **lowercase** keys — selection is case-insensitive and entries differing only in case collapse to one key, presence follows `Headers.has()` (an empty value is captured as `''`, an absent header adds no key), and a multi-valued field is captured comma-joined as `Headers.get()` returns it. Omitted, empty, or matching nothing, no `headers` key is emitted. `set-cookie` is **never** captured whatever the selector says: it is credential-bearing and `Headers.get()` joins its values into a string that is not a valid reconstruction. Every selected value reaches the client, so never name a header that carries a credential — and a selected `Location` can itself carry a sensitive path, query, or token. `HttpErrorFromResponseOptions`: `service?` (logical name in message, e.g. `'NCBI'`), `captureBody?` (default `true`), `bodyLimit?` (default `500`), `includeUrl?` (default `false`), `errorHeaders?` (default none), `data?` (extra fields merged into `error.data`, overriding defaults on key collision — a caller's own `url` or `headers` still reaches the wire), `cause?`, `codeOverride?` (per-status mapping override). Pairs naturally with `withRetry` — both classify codes the same way. A 501 also carries `data.retryable: false`, so retry fails it fast instead of re-asking for a method the upstream does not implement. | | `createPacer` | `(options: PacerOptions) -> Pacer` | FIFO queue in front of one rate-limited upstream — the outbound counterpart to `RateLimiter` (`utils/security`), which is inbound, per-caller, and reject-only, so it cannot queue work against an upstream budget. `pacer.run(task, { signal?, maxWaitMs? })` holds `task` until every `limits` window, `minStartGapMs`, `maxConcurrent`, and the cooldown gate allow it, then calls it with the caller's signal. `PacerOptions`: `name` (author-set telemetry label), `limits` (`{ requests, perMs }[]` — each a sliding window over recorded **start** times, so a slow response never widens the rate the upstream sees; all must allow a start), `minStartGapMs` (**not** expressible through `limits`: `{ requests: 10, perMs: 1000 }` permits ten starts in the same millisecond), `maxConcurrent`, `maxQueueDepth` (absolute backpressure for callers passing no `maxWaitMs`; rejects without arming a timer), `cooldown` (`{ baseMs, maxMs }`). **Shed:** `maxWaitMs` bounds queue time only, never the task. The projected wait is exact over the windows and the gap but a lower bound once `maxConcurrent` binds (a slot frees on an unknowable completion), so enqueue rejects only when that lower bound already exceeds `maxWaitMs` — no false sheds — and a still-queued entry rejects when `maxWaitMs` elapses. The shed error is `rateLimited` (-32003) with `data: { reason: 'pacer_shed', retryAfter, queueDepth }` and **no `retryable: false`** — to the calling agent a shed is an ordinary rate limit (wait `retryAfter`, call again) and that flag would say the opposite; `defaultIsTransient` reads the `reason` instead, so an enclosing `withRetry` fails fast rather than sleeping past the deadline the shed enforces. **Cooldown gate:** a `RateLimited` thrown by the task closes the gate for every queued caller until an absolute instant, `min(max(baseMs · 2^(consecutive−1), retryAfter), maxMs)` — `maxMs` caps both the doubling and an honored `Retry-After`, so a pathological upstream value cannot park the queue. Absent or unparseable `retryAfter` leaves the doubling; any other error leaves the gate open; the first success resets the count. **Composition:** `withRetry(({ signal }) => pacer.run(fn, { signal }), { signal, deadlineMs })` — retry outside, pacer inside, so each attempt re-queues and is re-paced. Because the gate is an absolute instant rather than a duration counted from dequeue, retry's `Retry-After` sleep and the gate overlap in wall-clock instead of summing: the window is waited once, not twice. **Lifecycle:** timers and `AbortSignal` only, process-local; the dispatch timer is `unref()`'d where supported; `dispose()` / `[Symbol.dispose]()` clears it and rejects queued waiters with `RequestCancelled` (in-flight tasks are left to finish) — wire it through `createApp({ teardown })`. On Workers state is per-isolate so the limits bind per isolate, OTel is off so the metrics are inert, and `createWorkerHandler` accepts no `teardown`. Metrics: `mcp.pacer.queue_depth`, `mcp.pacer.wait`, `mcp.pacer.sheds`, `mcp.pacer.cooldowns`, attributed by `mcp.pacer.name` only — see `api-telemetry`. | | `httpStatusToErrorCode` | `(status: number) -> JsonRpcErrorCode \| undefined` | Sync status → code lookup. Returns `undefined` for 1xx/2xx. A 3xx maps to `InvalidRequest` — it reaches error mapping under `redirect: 'manual'`, where the request as sent cannot be served at this URL, and that code is outside `withRetry`'s transient set since re-issuing returns the same redirect. Use when you need just the code without a `Response` object handy. No status maps to `InternalError` — that code means *this* server failed, which a remote status cannot establish; every 5xx is `ServiceUnavailable` (or `Timeout` for 504) and so picks up `withRetry`'s default transient policy. | --- ## `@cyanheads/mcp-ts-core/utils` — pagination | Export | API | Notes | |:-------|:----|:------| | `extractCursor` | `(params?) -> string \| undefined` | Extracts opaque cursor string from MCP request params. Checks `params.cursor` then `params._meta.cursor`. Returns `undefined` when no cursor is present. Does not decode. | | `paginateArray` | `<T>(items, cursorStr, defaultPageSize, maxPageSize, context: RequestContext) -> PaginatedResult<T>` | Decodes cursor, slices array, returns `{ items, nextCursor?, totalCount }`. `nextCursor` omitted on last page. Throws `McpError(InvalidParams)` on invalid cursor. | | `encodeCursor` | `(state: PaginationState) -> string` | Encodes `{ offset, limit, ...extra }` to opaque base64url string. | | `decodeCursor` | `(cursor, context: RequestContext) -> PaginationState` | Decodes opaque base64url cursor. Throws `McpError(InvalidParams)` if malformed. | --- ## `@cyanheads/mcp-ts-core/utils` — runtime | Export | API | Notes | |:-------|:----|:------| | `runtimeCaps` | `RuntimeCapabilities` object | Snapshot at import time. Fields: `isNode`, `isBun`, `isWorkerLike`, `isBrowserLike`, `hasProcess`, `hasBuffer`, `hasTextEncoder`, `hasPerformanceNow`. All booleans. Never throws. | --- ## `@cyanheads/mcp-ts-core/utils` — scheduling | Export | API | Notes | |:-------|:----|:------|