v2-api-conventions

# v2 API Conventions The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** ``` success (single) { "data": {...} } success (collection) { "data": [...], "nextCursor": "..." | null } failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } ``` Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. Each rule below guards a caller-visible failure: a non-integer `limit` reaching SQL, a non-envelope error body, an HTML 404 for unknown paths, an unapplied `limit` stripped by non-strict Zod, and an untyped bound parameter into an overloaded SQL function. ## Where the machinery lives | Concern | File | |---|---| | Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | | Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | | Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | | Contracts | `apps/sim/lib/api/contracts/v2/**` | | Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | ## Rule 1 — the envelope is produced by helpers, never by hand `v2Data`, `v2Error`, and the typed error helpers in `response.ts` (`v2ValidationError`, `v2RateLimitError`, `v2HttpError`, `v2InsufficientScope`, `v2HeadNoEffect`, `v2UploadDataPlaneError`) are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. **The envelope must hold for every failure mode, including the ones that happen before your handler runs.** Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. ## Rule 2 — status codes mean specific things | Status | `code` | Meaning | |---|---|---| | 200 / 201 | — | Success. 201 only for a created resource. | | 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | | 401 | `UNAUTHORIZED` | No/!valid API key. | | 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Where the cause is one a caller can act on it is named in `details.code`, from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). A few domain refusals still reach the wire without one. | | 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** an unknown path. | | 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | | 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes` — **and** a collection the response must materialize that is over *its* ceiling. Fourteen bodyless `GET`/`DELETE` operations publish it for the folder-tree cap (`FolderCollectionLimitExceededError`) or the rendered-artifact cap. | | 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | | 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | Two of these carry real design weight: **404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` answers with the same body on purpose. **500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. Treat any well-formed request that produces a 500 as the highest-severity defect on this surface: non-integer numerics, a method guard rejecting the `HEAD`-on-`GET` pairing, and untyped placeholders into SQL functions are the known shapes. **Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. **Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: - An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. - An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. **A 403 a caller can act on names its cause in `error.details.code` — but not every 403 does yet.** One status covers several different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan, delete a resource to get under a quota — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. A handful of domain refusals still throw a bare `OrchestrationError('forbidden', …)` and reach the wire with no code, so **write client code that treats `details.code` as optional**, and read `openapi/shared.ts`'s `FORBIDDEN_DESCRIPTION` for the current position rather than assuming the sweep is finished. For code you are *writing*, the rule below is unconditional. The vocabulary is the closed tuple `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts` (each member TSDoc-documented), published to OpenAPI as the `V2ForbiddenDetailCode` enum via `v2ForbiddenDetailCodeSchema`. A new code needs a documented entry there and is warranted only when the remedy differs. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, which hand-assembled lists drop. **HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. **A `GET` with side effects must declare `headSafe: false`.** The aliasing above is only sound because RFC 9110 §9.2.1 defines `HEAD` as safe. A `GET` that writes a row or opens an outbound connection is not, and an uptime monitor or link checker walking the documented URL list would drive those effects on every probe — `GET /files/{fileId}` records a `FILE_DOWNLOADED` audit event, so without `headSafe: false` a `HEAD` would fabricate a download that never happened. Both the JSON and binary builders take `headSafe`: a route that sets it `false` still authenticates and rate-limits a `HEAD`, then answers `v2HeadNoEffect()` — a bodiless 200 — before parsing or executing. Nothing observable is lost, because `HEAD` carries no body either way. Audit this whenever a read acquires an audit projection or an outbound call. ## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. Build the query slice from the shared helper, never by hand: ```ts ...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) ``` That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Never re-declare `limit: z.coerce.number()...` inline; inline schemas drift from `.int()`. Three cursor schemes exist. Two are the shared codecs in `response.ts`, both opaque base64-JSON, and which of them you use is decided by what the read can express, not by taste: - **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort AND the filters are stamped into the cursor and re-checked on replay, so changing `sortBy` or any filter mid-pagination is a 400, not a silently skipped page. - **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results. Both take the same two stamps: `cursorSortKey(sortBy, sortOrder)` for the ordering, and `cursorScopeKey(cursorRoute(contract, pathParams), { ... })` for every param that filters the sequence — the route identity is the first argument, the filter parts the second. **`limit` is never a stamp** — it selects how much of the sequence to return, not what the sequence is, and binding it strands every cursor the moment a caller changes page size. Params that only shape the response body are out for the same reason. The third is **per-domain**: a list whose read predates the shared codecs, or whose page boundary is not expressible as one, mints its own — a bare `encodeCursor({ version })` on `GET /workflows/{id}/versions` and `encodeCursor({ email })` on the workspace member list, the local codecs in `lib/audit-logs/query.ts`, `lib/logs/list-logs.ts`, and `lib/table/rows/cursor.ts`, and a usage-event id passed straight through by `GET /billing/logs`. Those tokens stay opaque and untouched, but a domain-minted cursor on a list a caller can re-filter is wrapped at the surface with `encodeScopedCursor(cursorScopeKey(cursorRoute(contract, pathParams), {...}), token)` and unwrapped with `readScopedCursor`, so it carries the same binding as the shared schemes. **A new list picks one of the two shared schemes.** Do not add a fourth. Every paged list's binding is declared in `lib/api/contracts/v2/list-pagination.test.ts` and checked against what the contract actually accepts, in both directions. A new list, or a new filter on an existing one, fails that test until its binding is declared or the param is explicitly recorded as unable to change the sequence. **A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. **Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Nearly every paged list takes the pair; `CURSOR_BINDINGS` in `contracts/v2/list-pagination.test.ts` is the authoritative set. Exactly one — `GET /workflows/{workflowId}/runs` — has a single sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That is the *only* sanctioned deviation, and it is documented in its contract. A new list picks the pair. Do not "fix" it by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency. Before documenting a second `order`-style exception, check every other endpoint on the same collection: if one of them already sorts those rows more than one way, the "exactly one sortable column" premise is false — fix the premise rather than documenting the exception. **A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`; it coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`. Reusing an internal `.shape.x` inherits the internal spelling (often a `z.enum(['true','false'])`); re-declare instead when the internal one is not the v2 convention. ## Rule 4 — reject what you do not implement **Every contract declares a `query`, even when the endpoint takes none** — `query: noInputSchema` (`z.object({}).strict()`), never omission. `parseRequest` validates the query slice only when the contract declares one, so an omitted `query` means "never look at the query string", not "takes no query params". An omitted `query` silently accepts any query string; `query-declaration.test.ts` fails a contract that omits it. Unknown query params are a 400. That is safe for first-party callers — the two SDKs send only `includeOutput`/`selectedOutputs`, both declared; the UI makes no v2 calls; `requestJson` appends nothing implicitly and there is no v2 cache buster — and it matches the already-strict body slice. Third-party callers appending a tracking tag or cache buster do break, which is why `api-reference/getting-started.mdx` documents the behavior rather than leaving it to be discovered from a 400. Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. Error messages name the field and, where there is one, the escape hatch: ``` limit must be a whole number limit cannot exceed 100 search cannot be empty sortBy: expected one of "name" | "createdAt" | "updatedAt" Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export ``` That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. ## Rule 5 — contract first, then use case, then route Order matters because each layer is checked against the one before it. 1. **Contract** in `lib/api/contracts/v2/<domain>.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. 2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives the use-case result **and the parsed request**, so a presenter reads request params (the active `sortBy`/`sortOrder` and filters it stamps into a cursor) straight from `query`/`params` rather than making the use case carry an HTTP concern back out. 3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. 4. **OpenAPI description** in `lib/api/contracts/v2/openapi/<domain>.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. ### Public descriptions Use the [API description conventions](../../../apps/sim/lib/api/contracts/v2/openapi/README.md) when writing or auditing endpoint and field descriptions. Keep a short action-and-resource summary; use the description for behavior that changes the caller's choice, input, interpretation, or next action. Ordinary operations usually need one to three sentences, with no mandatory minimum. Keep archive versus permanent-delete behavior, replacement versus partial-update semantics, partial success, retry safety, redaction, and asynchronous completion explicit. Verify these claims against the implementation. Describe observable behavior without exposing storage formats, locking mechanisms, internal identifiers, deployment architecture, or implementation history unless that detail changes how the caller must use the API. Reuse wording across resource families when behavior matches: “Omitted fields remain unchanged,” “Archive,” and “permanently delete.” Prefer “during the request” or “asynchronously” to “settled inline.” Preserve real semantic differences; do not standardize them away. Put field-specific rules in the source schema and reuse shared authentication and pagination wording. Shared schema descriptions also feed CLI help, so refer to related operation names rather than HTTP paths. Regenerate OpenAPI, CLI metadata, and CLI docs after changing their source descriptions; never hand-edit generated output. ## Rule 6 — a transient failure says when to come back A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: | Status | Source of the value | Where | |---|---|---| | 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | | 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. **Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. Carry it the whole way: `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`; a mapping that drops it turns a concurrency denial into a bare 429 with no `Retry-After` even though the policy named the wait. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. **A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; none of Sim's 413s are temporary — they are fixed ceilings, on the request body and on the collections a response must materialize — so it correctly sends none. ## Deliberate non-adoptions Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. | Practice | Verdict | Why |