backend-api-design

# Backend API Design ## Purpose Design consistent, production-grade REST or GraphQL APIs. Every endpoint must follow the same conventions for URLs, requests, responses, errors, and pagination. ## Agent Protocol ### Trigger Exact user phrases: "API design", "REST API", "GraphQL schema", "endpoint design", "API conventions", "URL structure", "HTTP methods", "response format", "API versioning", "pagination", "design an endpoint", "API contract". ### Input Context Before activating, verify: - The resource or feature being designed is known. - The tech-spec for the feature exists or the user has described the resource. - The chosen API style (REST/GraphQL) is known. If not, ask: "REST or GraphQL?" ### Output Artifact No file output unless the user requests it. Produces endpoint specifications as text. ### Response Format For each endpoint: ``` {method} {path} Auth: {required/optional/none} Request: {schema reference} Response 2xx: {schema reference} Errors: {list of error codes} ``` For a full API design, group by resource: ``` ## {resource} {list of endpoints} ``` No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output. No explanations of REST principles. ### Completion Criteria - [ ] All resources follow the naming conventions below. - [ ] Every endpoint has: method, path, auth requirement, request schema, response schema, error codes. - [ ] List endpoints are paginated. - [ ] Response envelope is consistent across all endpoints. - [ ] Error responses follow the standard format. - [ ] No verbs in URL paths. ### Max Response Length Per endpoint: 6 lines. Per resource: unlimited. ## Architecture Decision Trees ### REST vs GraphQL Decision Tree ``` What is the primary client type? ├── Public API with many consumers │ ├── REST — broadest compatibility, cache-friendly, CDN-friendly │ └── GraphQL only if clients need flexible data shapes ├── Single-page application (SPA) │ ├── REST + BFF — simpler, more performant, better caching │ └── GraphQL — if client needs to compose multiple resources ├── Mobile application │ ├── REST + BFF — smaller payloads, server-controlled shapes │ └── GraphQL — if network round-trips are the bottleneck └── Service-to-service (internal) ├── REST — simple, familiar, good enough └── gRPC — if latency-critical or streaming needed ``` ### Resource Modeling Decision Tree ``` Is it a thing (noun)? ├── Yes → Is it a top-level resource? │ ├── Yes → /v1/{resources} │ └── No → Is it always accessed through a parent? │ ├── Yes → /v1/{parents}/{parentId}/{children} │ └── No → Top-level resource with query filter └── No → Is it an action (verb)? ├── Yes → POST /v1/{resources}/{id}/{action} └── No → Neither — reconsider modeling ``` ### Pagination Strategy Decision Tree ``` Is the data real-time (new items inserted frequently)? ├── Yes → Cursor-based pagination — stable across insertions └── No → Is the dataset small (<10K rows)? ├── Yes → Offset-based pagination — simpler, skip+limit └── No → Do users need random page access? ├── Yes → Offset-based with keyset pagination fallback └── No → Cursor-based pagination — best performance ``` ### Consistency Model Decision Tree ``` Is immediate consistency required? ├── Yes → Synchronous writes, read-after-write guaranteed │ └── Use REST with strong consistency patterns └── No → Can the client tolerate stale data? ├── Yes → Eventual consistency, cache-friendly │ └── Use GraphQL with stale-while-revalidate └── No → Strong consistency within bounded context └── Use REST with version vectors / ETags ``` ## Workflow ### Step 1: Choose API Style REST: default for CRUD-heavy services with clear resources. GraphQL: when clients need flexible data shapes or multiple resources in one request. gRPC: when performance-critical, streaming, or polyglot service mesh. ### Step 2: Design REST Resources ``` GET /v1/{resources} -> list (paginated) GET /v1/{resources}/{id} -> single POST /v1/{resources} -> create PUT /v1/{resources}/{id} -> full replace PATCH /v1/{resources}/{id} -> partial update DELETE /v1/{resources}/{id} -> delete // Sub-resources GET /v1/{parents}/{parentId}/{children} // Actions (when CRUD does not fit) POST /v1/{resources}/{id}/{action} Example: POST /v1/orders/{id}/cancel // Search POST /v1/{resources}/search GET /v1/search?q={query}&type={resourceType} ``` ### Step 3: Naming Rules - Plural nouns: /users, /orders, /products. Never singular: /user. - kebab-case for multi-word: /order-items, /shipping-addresses. - No verbs in CRUD paths: /users not /getUsers. - Query parameters for filtering: ?status=active&sort=-createdAt. - Version prefix: /v1/, /v2/. Never remove a version once released. ### Step 4: Response Envelope Every response uses this exact structure: ```json { "data": { "id": "uuid", ... }, "meta": { "requestId": "uuid", "timestamp": "2026-05-14T10:30:00Z" }, "error": null } ``` Error case: ```json { "data": null, "error": { "code": "RESOURCE_NOT_FOUND", "message": "User with id abc-123 not found", "details": [ { "field": "id", "reason": "not_found", "message": "No user matches this id" } ] } } ``` ### Step 5: Pagination Every list endpoint uses cursor or offset pagination: ```json { "data": [...], "meta": { "pagination": { "page": 1, "limit": 20, "total": 150, "totalPages": 8, "hasNext": true, "hasPrev": false }, "requestId": "uuid" } } ``` ### Step 6: Error Codes | HTTP Status | Error Code | When | |-------------|------------|------| | 400 | VALIDATION_ERROR | Input validation failed | | 400 | MALFORMED_REQUEST | Invalid JSON, bad encoding | | 401 | UNAUTHORIZED | Missing or invalid authentication | | 403 | FORBIDDEN | Authenticated but no permission | | 404 | NOT_FOUND | Resource does not exist | | 409 | CONFLICT | State conflict (duplicate, stale version) | | 422 | UNPROCESSABLE_ENTITY | Semantic validation failure | | 429 | RATE_LIMITED | Too many requests | | 500 | INTERNAL_ERROR | Unexpected server error | ## API Governance Framework ### Governance Levels ```yaml governance_levels: level_0_no_governance: description: "Each team designs APIs independently" characteristics: ["Inconsistent conventions across teams", "No API review process"] when_appropriate: "Single team, early stage startup, internal-only APIs" level_1_conventions: description: "Shared conventions documented but not enforced" characteristics: ["API style guide exists", "Naming conventions documented", "Error format agreed"] enforcement: "Manual review in PR — architect reviews API changes" when_appropriate: "2-3 teams, moderate API surface" level_2_spec_governance: description: "API specs validated automatically in CI" characteristics: ["OpenAPI/GraphQL spec mandatory", "Automated linting on spec", "Breaking change detection"] enforcement: "CI pipeline rejects PRs that violate conventions or break compatibility" tools: ["spectral (OpenAPI linting)", "openapi-diff (breaking change detection)", "graphql-inspector"] when_appropriate: "3-8 teams, growing API surface, external consumers" level_3_api_platform: description: "Centralized API management with developer portal" characteristics: ["API gateway with unified auth, rate limiting, analytics", "Developer portal for discovery", "API version lifecycle management"] enforcement: "Automated + manual — ARB reviews cross-cutting API changes" when_appropriate: "8+ teams, public APIs, B2B integrations" ``` ### API Review Checklist ```yaml api_review_checklist: naming_and_structure: - "Resource names are plural nouns, lowercase, kebab-case" - "No verbs in CRUD paths (use /orders not /getOrders)" - "Nested resources follow /parents/:parentId/children" - "Query parameters for filtering, sorting, pagination" - "Consistent date format (ISO 8601 — RFC 3339)" request_contract: - "Request body schema validated (JSON Schema / OpenAPI)" - "Required vs optional fields explicitly marked" - "Idempotency key accepted for mutation endpoints" - "Content-Type and Accept headers validated" - "Rate limit headers documented and enforced" response_contract: - "Consistent envelope — data, error, meta structure" - "Error codes are UPPER_SNAKE_CASE strings" - "Pagination meta included on all list endpoints" - "Response includes requestId for tracing" - "No internal implementation details exposed" backwards_compatibility: - "No removal or rename of existing fields" - "No change to existing field types" - "No narrowing of existing field constraints" - "Adding new fields is safe (clients should ignore unknown fields)" - "Adding new optional request fields is safe" - "Making required fields optional is safe" - "Making optional fields required is BREAKING" security: - "Authentication method defined (API key, JWT, OAuth2, mTLS)" - "Authorization model (RBAC, ABAC, scope-based)" - "Sensitive data not exposed in URLs or responses" - "Rate limiting configured with graduated tiers" - "HTTPS enforced — no cleartext endpoints" ``` ## API Versioning Decision Tree ```yaml versioning_decision_tree: "Are consumers external (third-party developers, partners)?": yes: "URI path versioning (v1, v2) — clear, obvious, easy to communicate" no_internal: "Can you coordinate deployments with all consumers?": yes: "No versioning — evolve API with backwards compatibility, coordinate changes" no: "Header versioning — consumers opt-in to new version without path changes" "Do mobile clients consume this API?": yes: "URI path versioning — app store versions can't change API paths dynamically" "Support last 2 major versions — force upgrade when dropping v1" no: "Header versioning or no versioning (backwards compatible evolution)" "Is the API in rapid iteration mode (<1 year old, <10 consumers)?": yes: "No versioning — additive changes only, feature flags for experiments" no: "Formal versioning strategy with deprecation policy (18-month minimum)" ``` ## REST API Implementation Patterns ### Pattern: Consistent Error Responses ```typescript // TypeScript error handler middleware import { Request, Response, NextFunction } from 'express'; interface ApiError { code: string; message: string; details?: Array<{ field: string; reason: string; message: string }>; } class ApiResponse<T> { constructor( public data: T | null, public error: ApiError | null, public meta: { requestId: string; timestamp: string } ) {} static success<T>(data: T, requestId: string): ApiResponse<T> { return new ApiResponse(data, null, { requestId, timestamp: new Date().toISOString(), }); } static error(code: string, message: string, requestId: string, details?: ApiError['details']): ApiResponse<null> { return new ApiResponse(null, { code, message, details }, { requestId, timestamp: new Date().toISOString(), }); } } function errorHandler(err: Error, req: Request, res: Response, _next: NextFunction): void { const requestId = req.headers['x-request-id'] as string || crypto.randomUUID(); if (err instanceof ValidationError) { res.status(400).json(ApiResponse.error('VALIDATION_ERROR', err.message, requestId, err.details)); } else if (err instanceof NotFoundError) { res.status(404).json(ApiResponse.error('NOT_FOUND', err.message, requestId)); } else if (err instanceof ConflictError) { res.status(409).json(ApiResponse.error('CONFLICT', err.message, requestId)); } else { console.error('Unhandled error:', err); res.status(500).json(ApiResponse.error('INTERNAL_ERROR', 'An unexpected error occurred', requestId)); } } ``` ### Pattern: Idempotent POST ```typescript // Idempotency-Key support for mutation endpoints import { Request, Response, NextFunction } from 'express'; interface IdempotencyRecord { key: string; status: 'pending' | 'completed'; response?: { status: number; body: unknown }; expiresAt: number; } class IdempotencyMiddleware { constructor(private store: { get(key: string): Promise<IdempotencyRecord | null>; set(key: string, value: IdempotencyRecord): Promise<void> }) {} middleware() { return async (req: Request, res: Response, next: NextFunction): Promise<void> => { if (req.method !== 'POST' && req.method !== 'PATCH') { return next(); } const idempotencyKey = req.headers['idempotency-key'] as string; if (!idempotencyKey) { return next(); } const existing = await this.store.get(idempotencyKey); if (existing) { if (existing.status === 'completed') { res.status(existing.response!.status).json(existing.response!.body); return; } if (existing.status === 'pending') { res.status(409).json({ error: { code: 'CONFLICT', message: 'Request with this idempotency key is already being processed' }, }); return; } } await this.store.set(idempotencyKey, { key: idempotencyKey, status: 'pending', expiresAt: Date.now() + 86_400_000 }); const originalJson = res.json.bind(res); res.json = function (body: unknown) { this.store.set(idempotencyKey, { key: idempotencyKey, status: 'completed', response: { status: res.statusCode, body } }); return originalJson(body); }; next(); }; } } ``` ### Pattern: Sparse Fieldsets ```typescript // GraphQL-style field selection for REST endpoints function parseFields(fields: string | undefined): Set<string> | null { if (!fields) return null;