api-utils

## Overview Utility exports from `@cyanheads/mcp-ts-core/utils`. Utilities with complex APIs have dedicated reference files; simpler utilities are documented inline below. **Tier 3** = optional peer dependency. Install as needed (e.g., `bun add js-yaml`). All Tier 3 methods are **async** (lazy-load deps on first call). **Context parameters.** Every helper below that takes a `context` accepts the handler `Context` as well as a `RequestContext` bag — pass `ctx` straight through, no slicing. ## References | Reference | Path | Covers | |:----------|:-----|:-------| | Formatting | `references/formatting.md` | `markdown()`, `MarkdownBuilder`, `diffFormatter`, `tableFormatter`, `treeFormatter` — builder patterns, option types, style variants, usage examples | | Parsing | `references/parsing.md` | `yamlParser`, `xmlParser`, `csvParser`, `jsonParser`, `pdfParser`, `dateParser`, `frontmatterParser` — method signatures, option types, peer deps, `Allow` flags, PDF workflows | | Security | `references/security.md` | `sanitization`, `RateLimiter`, `IdGenerator` — config types, method details, sensitive fields, usage examples | --- ## `@cyanheads/mcp-ts-core/utils` — network | Export | API | Notes | |:-------|:----|:------| | `fetchWithTimeout` | `(url, timeoutMs, context, options?: FetchWithTimeoutOptions) -> Promise<Response>` | Wraps `fetch` with `AbortController` timeout. `timeoutMs` bounds the **whole exchange**: on a 2xx carrying a body the returned `Response` is a passthrough wrapper that keeps the deadline armed until the body closes, errors, or is cancelled, so a stalled stream rejects the caller's `.text()`/`.json()` with the same `Timeout` error the header phase raises. `status`, `statusText`, `headers`, `url`, `redirected`, and `type` carry across the wrapper; the original body is locked by it, and bodyless/null-body responses (HEAD, 204/205/304) come back untouched. `FetchWithTimeoutOptions` extends `RequestInit` (minus `signal`) and adds `rejectPrivateIPs?: boolean`, `expectedStatuses?: number[]` (listed non-2xx statuses logged at `debug` not `error`, still thrown), `errorBodyLimit?: number` (bytes of a non-2xx body kept, default `500`), `errorHeaders?: string[]` (response headers copied onto `error.data.headers` on a non-2xx — same selector as `httpErrorFromResponse` below; `location` is selectable under `redirect: 'manual'` but does **not** compose with `rejectPrivateIPs`, whose per-hop branch follows every 3xx carrying a `Location` before the throw path sees it), and `signal?: AbortSignal` (external cancellation — an abort on it throws `RequestCancelled` (-32011), logged at `info` and outside `withRetry`'s transient set, since the caller is gone and no retry can reach them; an abort whose reason is a `TimeoutError` — `AbortSignal.timeout()`, or an `AbortSignal.any` whose timeout member fired — is a deadline instead, and throws `Timeout` (-32004) with `data.errorSource: 'FetchSignalTimeout'`, distinct from the helper's own `timeoutMs` expiry, `'FetchTimeout'`, and likewise outside `withRetry`'s default transient set, since every retry would reuse the fired signal). Validation rejections carry `data.reason` and a `recovery.hint`: `invalid_url` (not an absolute `http:`/`https:` URL, including a redirect target), `private_address_blocked` (the SSRF guard refused the host by name, literal IP, or DNS answer), `too_many_redirects` (past the 5-hop cap, with `data.maxRedirects`). A redirect hop's rejection is thrown with no record from `fetchWithTimeout`, as on the initial URL, so the caller's record — at a tool's declared `severity` — is the only one. On a non-2xx, `error.data` carries `status`/`body` plus the legacy `statusCode`/`responseBody` aliases (identical values; consolidating in a future major); a body over `errorBodyLimit` is captured from both ends — 40% head, 60% tail, joined by `…[N bytes elided]…` — so a diagnostic behind a boilerplate preamble survives the cap, while a body still streaming at the 16 KiB scan ceiling stays head-only with a trailing `…`. SSRF guard (best-effort, not hard isolation): blocks RFC 1918, loopback, link-local, CGNAT, cloud metadata. DNS validation on Node, Bun, and Cloudflare Workers under `nodejs_compat`; hostname-only fallback otherwise. **Both resolvers are queried** — `resolve4`/`resolve6` (c-ares) and `lookup` (the system resolver, which is what reads `/etc/hosts`, split DNS, and NSS modules) — and a non-global answer from either rejects. Runtimes differ in which resolver the connection uses (Bun 1.4 moved `net.connect()` on Linux to `getaddrinfo` while leaving `dns.resolve*()` on c-ares), so checking one alone leaves a name the other can see unguarded; each probe settles independently, so a resolver absent from the runtime is skipped rather than fatal. Manual redirect following (max 5) with per-hop SSRF check; a 3xx without `Location` (304 included) is not followed and fails as a non-2xx (`InvalidRequest`, not retried), as it does without the option. **DNS rebinding / TOCTOU gap** — the validation lookup and `fetch`'s own resolution are independent; pair with egress controls or a DNS-pinning fetch proxy for strong isolation. **Error/log redaction:** every thrown message and log record names the request by its origin plus elision markers — `https://api.example.com/sk-key/reverse?lat=1` reads `https://api.example.com/…?…`, and a root URL keeps its bare origin — so a credential in the path (`/bot<token>/…`, webhook URLs) or the query (`?api-key=…`, `?api_key=…`) never reaches the client or the logs. A URL the runtime quotes in its own rejection message is reduced the same way. The actual request still uses the full URL. To tell endpoints apart in the logs, label the call through its context: `fetchWithTimeout(url, ms, withExtra(ctx, { endpoint: 'reverse' }))` (`withExtra` from `/utils`) puts `endpoint` on every record for the call. The label is a field, not part of the message, so calls to one origin share one budget under the logger's per-message rate limit (`MCP_LOG_RATE_LIMIT_THRESHOLD` records per `MCP_LOG_RATE_LIMIT_WINDOW_MS`, default 10 a minute) whatever endpoint they name; repeats past it are dropped and reported later as a `Suppressed N` line. A network-level failure — an unparseable redirect `Location` included — throws `ServiceUnavailable` (`data.errorSource: 'FetchNetworkErrorWrapper'`) with the runtime's rejection as `cause`, so its transport code survives — on the rejection itself under Bun (`ConnectionRefused`), on its `cause` under Node (`ECONNREFUSED`) — and the failure record carries the same `causeChain` field as `withRetry`'s retry record. | | `withRetry` | `<T>(fn: (attempt: RetryAttempt) => Promise<T>, options?: RetryOptions) -> Promise<T>` | Executes `fn` with exponential backoff. Retries on transient errors (`ServiceUnavailable`, `Timeout`, `RateLimited`); non-transient errors fail immediately. Honors an upstream `Retry-After` on `data.retryAfter` (delta-seconds or HTTP-date) over exponential backoff, capped at `maxDelayMs`; a requested wait beyond the cap fails fast rather than sleeping. On exhaustion, enriches the final error with attempt count in message and `data.retryAttempts`. Each `Retry N/M for <operation>: <message> — waiting Xms` debug record adds `causeChain` to the context's `extra` when the retried error has a cause or a string `code` — `{ name, message, code? }` per node, the error itself first, so a transport code (`ECONNRESET`, `ConnectionRefused`) survives a retry that later succeeds; projections only, never a raw `Error`, a stack, or `McpError.data`. An error with neither logs exactly as before. `<message>`, and the exhausted error's message and name, are the error's own read as text: a `ZodError`'s message its one-line summary (`<path>: <message> (+N more)`), a Symbol reads `Symbol(…)`, a number its digits, and one that throws on read or is an object — or a thrown value that is an object but not an `Error` — `[Unreadable]`, so what was thrown never fails the retry. **Place the retry boundary around the full pipeline** (fetch + parse), not just the network call. `RetryOptions`: `maxRetries` (default `3`), `baseDelayMs` (default `1000`), `maxDelayMs` (default `30000`), `jitter` (default `0.25`), `operation` (log label), `context` (RequestContext), `signal` (AbortSignal), `isTransient` (custom predicate), `deadlineMs` (total wall-clock budget — see below). | | `RetryAttempt` | `{ readonly signal: AbortSignal; readonly remainingMs: number }` | What `fn` receives each attempt. `signal` is `AbortSignal.any` over the `deadlineMs` clock and `options.signal`; `remainingMs` is what is left of the total budget as the attempt starts, never negative and `Number.POSITIVE_INFINITY` when no deadline is set — so `Math.min(perAttemptMs, remainingMs)` is correct either way. A zero-argument `fn` stays assignable, so existing callers compile unchanged. | | `deadlineMs` | `RetryOptions` field | One wall-clock budget across every attempt, backoff, and honored `Retry-After` — the bound `maxRetries` plus a per-attempt timeout cannot express. Four 30s attempts outlast a client's 60s request timeout, so the caller gets a transport timeout instead of the server's classified error. **Thread `attempt.signal` into the attempt's I/O** (`fetchWithTimeout(url, Math.min(30_000, remainingMs), ctx, { signal })`) or the deadline overshoots by one in-flight request. Clock is `AbortController` + `setTimeout` (never `AbortSignal.timeout()`, per the Bun realm mismatch), cleared on return — no timer outlives the call. Expiry rejects with `Timeout` (-32004) carrying `data: { reason: 'retry_deadline_exceeded', deadlineMs, elapsedMs, retryAttempts }` and the last attempt's error as `cause`; **one shape for every expiry**, including the per-attempt `Timeout` (`errorSource: 'FetchSignalTimeout'`) the clock's abort raises inside `fetchWithTimeout` and the raw abort reason a mid-backoff expiry would otherwise surface. No `retryable` flag (a narrower call can still succeed) and no `attempt` index (`retryAttempts` carries it). A backoff that would outlast the remaining budget fails fast with the expiry instead of sleeping into a certain timeout; an honored `Retry-After` that would outlast it takes the `maxDelayMs` exit instead — the attempt's error unchanged, `data.retryAfter` intact, since "wait the window the upstream named" is still the caller's action. **Three clocks stay distinct:** a caller abort on `options.signal` keeps precedence — mid-attempt it rethrows the attempt's error unchanged, mid-backoff it rejects with `signal.reason` itself (an `AbortError` `DOMException` for a reason-less `abort()`), and the handler factory reports either as `RequestCancelled` when the request signal is the one that fired — a single attempt's timeout is `Timeout` with `errorSource: 'FetchTimeout'` and no `reason`, and the expiry is `Timeout` with the `reason`. Unset, behavior is identical to before — attempt counts, delays, log lines, and the exhausted-error shape untouched. Bounds **one** ladder: a tool making three upstream calls threads its own remaining budget into each. | | `defaultIsTransient` | `(error: unknown) -> boolean` | The predicate `withRetry` uses when `isTransient` is omitted: an `McpError` with a transient code (`ServiceUnavailable`, `Timeout`, `RateLimited`) unless it carries `data.retryable === false`, `data.reason === 'pacer_shed'`, or `data.errorSource === 'FetchSignalTimeout'` (a caller-side deadline that already fired); any non-`McpError` throw is assumed transient. Exported so `isTransient` — which **replaces** the default outright — can compose instead of mirroring the transient set, which drifts silently when the framework's classification changes: `isTransient: (error) => !isMyBudgetRefusal(error) && defaultIsTransient(error)`, or the inverse `defaultIsTransient(error) \|\| isMyRetryableShape(error)`. The transient code set itself stays private (a module-level `Set` an exported binding could be mutated into framework-wide retry behavior). | | `httpErrorFromResponse` | `(response: Response, options?: HttpErrorFromResponseOptions) -> Promise<McpError>` | Maps an HTTP `Response` to a properly classified `McpError` — full status table including 401/403/408/422/429/5xx, body capture (truncated), `retry-after` header, optional `cause`. `error.data` carries `status`/`body` plus the legacy `statusCode`/`responseBody` aliases (identical values), so a consumer can classify either helper's error without knowing which raised it. Use this instead of hand-rolling `if (status === 429) ...` ladders. Reads the response body — `clone()` first if you need it elsewhere. **`error.data` is client-facing** — the framework forwards it verbatim as `structuredContent.error.data` — so the full upstream URL is **omitted by default**: a request URL routinely carries user input, internal identifiers, or an API key in its query string. `includeUrl: true` opts into `data.url` carrying the full `response.url`; with an empty `response.url` no key is added either way, and the message still names the host. Response headers are opt-in on the same footing: `errorHeaders: ['x-ratelimit-remaining-usd', 'x-request-id']` copies the named headers onto `data.headers` under **lowercase** keys — selection is case-insensitive and entries differing only in case collapse to one key, presence follows `Headers.has()` (an empty value is captured as `''`, an absent header adds no key), and a multi-valued field is captured comma-joined as `Headers.get()` returns it. Omitted, empty, or matching nothing, no `headers` key is emitted. `set-cookie` is **never** captured whatever the selector says: it is credential-bearing and `Headers.get()` joins its values into a string that is not a valid reconstruction. Every selected value reaches the client, so never name a header that carries a credential — and a selected `Location` can itself carry a sensitive path, query, or token. `HttpErrorFromResponseOptions`: `service?` (logical name in message, e.g. `'NCBI'`), `captureBody?` (default `true`), `bodyLimit?` (default `500`), `includeUrl?` (default `false`), `errorHeaders?` (default none), `data?` (extra fields merged into `error.data`, overriding defaults on key collision — a caller's own `url` or `headers` still reaches the wire), `cause?`, `codeOverride?` (per-status mapping override). Pairs naturally with `withRetry` — both classify codes the same way. A 501 also carries `data.retryable: false`, so retry fails it fast instead of re-asking for a method the upstream does not implement. |