openapi-from-traffic

name

openapi-from-traffic

description

Reverse-engineer an OpenAPI 3.1 spec from observed HTTP traffic. Use `@ir-kit/openapi-recon` (a runtime-agnostic lib that accepts standard Web Fetch `Request` / `Response` pairs and folds them into a JSON Schema 2020-12 document) when the user has live traffic but no spec, wants to discover a third-party API's shape, capture an internal API from real usage, or generate a starting `openapi.yaml` from a HAR/proxy log. Ships a `fromHAR()` helper that one-lines HAR-file ingestion — works with HARs exported from k6 Studio Recorder, browser DevTools, mitmproxy, Charles, Postman, anything. For interactive capture from a browser, point them at `@ir-kit/glean` (the Chrome DevTools extension built on `openapi-recon`). Triggers on "reverse engineer API", "discover spec from traffic", "OpenAPI from HAR", "k6 Studio HAR to spec", "capture API in DevTools", "mitmproxy to OpenAPI", "infer schema from requests", "spec from observed HTTP", "generate spec from network tab", `fromHAR`. Do NOT use when the user already has an OpenAPI spec — see openapi-sdk or k6-loadtest instead.

OpenAPI from observed HTTP — @ir-kit/openapi-recon

Programmatic spec inference from real traffic. Feed standard Web Fetch Request / Response pairs, get back an OpenAPIV3_1.Document with templated paths, JSON Schema 2020-12 bodies, per-status response schemas, and detected auth schemes.

Runtime-agnostic: works in browsers, Node, Deno, Bun, Cloudflare Workers — anywhere Request / Response exist. Pure functions, no global state, tree-shakable.

When to reach for this

• User wants to document an existing API but has no openapi.yaml.
• User has a HAR file, a proxy log, or service-worker observations they want to convert to a spec.
• User is integrating with a third-party API and needs to capture its shape from real calls.
• User wants live spec generation from a service mesh / observability pipeline.

For interactive browser-tab capture, suggest the Chrome DevTools extension @ir-kit/glean instead of writing the lib glue yourself — it ships with the UI for picking origins, filtering noise, and exporting YAML/JSON.

Library use (Node / browser / Workers)

import { createRecon } from "@ir-kit/openapi-recon";

const recon = createRecon({
  title: "Petstore",
  version: "0.1.0",
  maxExamples: 3,                      // examples kept per shape (default 3)
  refDedupeThreshold: 2,               // when to extract repeated schemas as $ref (default 2)
  redactHeaders: ["x-secret-header"],  // header names to scrub from examples
});

// Anywhere you observe traffic — fetch wrapper, service-worker, proxy hook:
await recon.observe(request, response);

// Snapshot whenever:
const document = recon.toOpenAPI();   // → OpenAPIV3_1.Document

// Per-origin snapshot (when feeding traffic from multiple backends):
const justBackend = recon.toOpenAPI({ origin: "https://api.example.com" });

Input requirements

• request / response are standard Web Fetch types — same as fetch() accepts/returns.
• Bodies must already be readable. Call .clone() upstream if you also need to forward the response — recon.observe consumes the body.
• Non-JSON bodies are skipped silently. Form data, binary, multipart — currently ignored.

What it infers

Layer	Behavior
Path templating	/pets/42 + /pets/8 collapse into /pets/{petId} with { petId: "string" }. Pure-numeric segments become integer. Disable with pathTemplating: false.
Request / response bodies	JSON Schema 2020-12, merged across samples (union types, optional fields, enum candidates).
Per-status responses	200 and 404 keep distinct shapes — not collapsed into one.
Auth schemes	Authorization: Bearer … → bearerAuth; X-API-Key: … → apiKeyAuth; HTTP Basic too. Detected per-operation, deduped at components.securitySchemes.
Per-origin grouping	Feed traffic from many backends, snapshot one origin at a time via toOpenAPI({ origin }).

Common integration patterns

Fetch wrapper (intercept at the call site)

import { createRecon } from "@ir-kit/openapi-recon";

const recon = createRecon({ title: "MyAPI" });

const originalFetch = globalThis.fetch;
globalThis.fetch = async (input, init) => {
  const req = new Request(input, init);
  const res = await originalFetch(req.clone());
  // Clone so the caller still gets a readable body
  await recon.observe(req, res.clone());
  return res;
};

Cloudflare Worker / edge proxy

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const res = await fetch(req.clone());
    await env.RECON_DO.observe(req, res.clone());  // store via Durable Object
    return res;
  },
};

HAR file replay (offline analysis)

Two equivalent paths — pick the one that fits.

CLI (Node only, fastest for one-shot conversion):

# File → file
openapi-recon ./traffic.har --out spec.json --title "Captured API"

# File → stdout, pipe to yq for YAML
openapi-recon ./traffic.har | yq -P > spec.yaml

# Stdin (pipe from any HAR-emitting tool)
cat traffic.har | openapi-recon -

Flags: --out, --title, --version, --origin, --max-examples, --no-path-templating, --help.

Programmatic (runtime-agnostic — Node, browsers, Workers, Deno):

import { readFile } from "node:fs/promises";
import { fromHAR } from "@ir-kit/openapi-recon";

// HAR JSON content (any runtime). For Node, read the file yourself:
const recon = await fromHAR(
  await readFile("./traffic.har", "utf8"),
  { title: "Captured" },
);

const document = recon.toOpenAPI();

Pre-parsed HarFile objects also work — useful when traffic comes from a tool that already deserialized:

import { fromHAR, type HarFile } from "@ir-kit/openapi-recon";

const har: HarFile = /* … from k6 Studio Recorder, browser DevTools, mitmproxy, etc. */;
const recon = await fromHAR(har, { title: "Studio capture" });

Both paths handle HTTP/2 pseudo-headers (:authority etc.), body-on-GET edge cases, and malformed-URL entries by skipping silently. For custom HAR-entry filtering (drop hosts, redact tokens, date-range slicing), use the underlying recon.observe(req, res) and iterate the HAR yourself.

Glean — the Chrome extension

For ad-hoc capture from a browser, install @ir-kit/glean (or the Chrome Web Store listing once shipped). It adds a DevTools panel that:

1. Captures network traffic from the inspected page.
2. Folds it through openapi-recon in real time.
3. Lets the user pick an origin, redact headers, and export YAML/JSON.

No code required — useful when the workflow is "browse the app, dump a spec at the end".

Common pitfalls

• Bodies are consumed. If your hook also forwards the response, clone first: recon.observe(req, res.clone()).
• JSON only. Form-encoded / multipart / binary bodies don't fold into the spec yet. Skip the request rather than failing.
• Auth detection is heuristic — based on header presence + format. If the API uses a non-standard auth scheme, post-process components.securitySchemes manually.
• Numeric path segments like /items/v2/foo won't break path templating — the v2 stays literal because it's not pure-numeric.
• Sample count matters. With maxExamples: 1, optional fields look required. Feed more traffic for accurate optionality.

HAR as the universal capture format

Any tool that exports HAR feeds straight into fromHAR: k6 Studio Recorder, browser DevTools "Save all as HAR with content", mitmproxy (mitmweb → Export → HAR), Charles, Postman, Wireshark plugins. The HAR file is the integration seam — no per-tool integration needed.

That's why this package doesn't ship "k6 Studio integration" or "DevTools plugin" — it consumes the standard format every recorder already emits.

Workflow: capture → spec → SDK

Recon emits a spec; pair with one of the SDK generators to round-trip:

import { createRecon } from "@ir-kit/openapi-recon";
import { generate } from "@ir-kit/openapi-typescript";

// ... feed traffic, then:
const spec = recon.toOpenAPI();

await generate({
  input: spec,            // pre-parsed object — no fetch/parse step
  output: "./client",
});

This is the entire "spec discovery → typed client" pipeline for unknown APIs.

How AI agents should use this

1. If the user wants to capture interactively from a browser → suggest the Glean extension first.
2. If the user wants programmatic capture (Node script, Worker, proxy) → show createRecon + the observe(req, res) loop.
3. If the user has a HAR file (from k6 Studio, DevTools, mitmproxy, Charles, etc.) → show fromHAR(harJsonContent, config?) — it's the one-line replacement for the manual observe loop. Mention that HAR is the universal integration format, so we don't need per-tool plugins.
4. If they want to pipe the output to an SDK generator → show the recon → openapi-typescript (or any openapi-{lang}) handoff.
5. Remind them to .clone() request/response when also forwarding.