api-validator

# API Validator Audit Bifrost's HTTP API surface by scanning controller/handler route registrations, deriving the actual authentication behavior from middleware wiring, and comparing the result with `docs/openapi/openapi.yaml` and referenced OpenAPI path files. Use this skill when the user asks for any of the following: - "missing APIs" - "incorrect APIs" - "validate APIs" - "check OpenAPI docs against controllers" - "include auth information" - "fix API docs" - "audit route coverage" Default behavior is **audit only**. Do not edit files unless the user explicitly asks for fixes or approves a proposed fix plan. --- ## Usage ```bash /api-validator # Full audit: all handlers + OpenAPI /api-validator management # Audit only /api/*, /health, /ws, /metrics /api-validator inference # Audit /v1/* and provider integration APIs /api-validator auth # Focus on effective OpenAPI security vs middleware auth /api-validator <path-prefix> # Audit one prefix, e.g. /api/governance or /openai /api-validator --fix # Audit, present plan, then fix after approval ``` If scope is unclear, ask: ```text Should I audit all APIs, only management APIs (/api/*), only inference/integration APIs (/v1/* and provider prefixes), or a specific path prefix? ``` --- ## Source of truth ### Code route sources Scan these files/directories as the controller source of truth: | Area | Source | Notes | |---|---|---| | Server route wiring | `transports/bifrost-http/server/server.go` | `RegisterAPIRoutes`, `RegisterInferenceRoutes`, `RegisterUIRoutes`, direct `/metrics`, middleware lists | | HTTP handlers/controllers | `transports/bifrost-http/handlers/*.go` | `RegisterRoutes` methods contain direct route registrations | | SDK/provider integrations | `transports/bifrost-http/integrations/*.go` | `RouteConfig` factories and `GenericRouter.RegisterRoutes` register OpenAI/Anthropic/GenAI/Bedrock/Cohere/LiteLLM/LangChain/PydanticAI/Cursor/Passthrough routes | | Auth middleware | `transports/bifrost-http/handlers/middlewares.go` | `APIMiddleware`, `InferenceMiddleware`, whitelists, realtime auth skips | | Context auth extraction | `transports/bifrost-http/lib/ctx.go` | Virtual key and API key header extraction | | Governance VK parser | `plugins/governance/utils.go` | Accepted virtual key headers for VK self-service endpoints | ### OpenAPI documentation sources Scan these files as the documented API source of truth: | Area | Source | |---|---| | Root spec/path map/security schemes | `docs/openapi/openapi.yaml` | | Bundled output (generated, **do not edit**) | `docs/openapi/openapi.json` — produced by CI (`.github/workflows/openapi-bundle.yml`) on push to `main` | | Inference paths | `docs/openapi/paths/inference/*.yaml` | | Integration paths | `docs/openapi/paths/integrations/**/*.yaml` | | Management paths | `docs/openapi/paths/management/*.yaml` | | Schemas | `docs/openapi/schemas/**/*.yaml` | `docs/openapi/openapi.json` is a build artifact. **Never edit it by hand and never regenerate it into the checked-in path.** The `OpenAPI Bundle` GitHub Actions workflow runs `python bundle.py` on push to `main` and commits the result as a separate `chore: regenerate openapi.json --skip-ci` commit. When validating locally, always bundle to `/tmp`, not into the repo. --- ## Workflow overview 1. **Preflight** -- Confirm scope, inspect git status, avoid edits unless approved. 2. **Extract actual API inventory** -- Scan route registrations and route config factories. 3. **Derive actual auth behavior** -- Trace middleware registration and handler-specific auth checks. 4. **Extract documented OpenAPI inventory** -- Resolve path refs and effective operation security. 5. **Compare inventories** -- Detect missing, stale, method/path, parameter, schema, and auth mismatches. 6. **Report findings** -- Present actionable tables with file/line references and recommended fixes. 7. **Fix with approval** -- If requested, update OpenAPI files or controller code after showing a plan. 8. **Validate** -- Re-bundle OpenAPI and run relevant tests/linters if available. --- ## Step 1: Preflight Always start with: ```bash git status --short ``` If the worktree is dirty, mention it and avoid overwriting unrelated changes. Identify the scope: ```bash # Route registration entry points grep -R "func (.*RegisterRoutes" -n transports/bifrost-http/handlers transports/bifrost-http/integrations --include='*.go' grep -n "func (s \*BifrostHTTPServer) RegisterAPIRoutes\|func (s \*BifrostHTTPServer) RegisterInferenceRoutes\|RegisterUIRoutes" transports/bifrost-http/server/server.go ``` If the user asked for `--fix`, still audit first and present a fix plan before editing. --- ## Step 2: Extract actual API inventory from controllers Create an actual route inventory with this shape: | Method | Path | Handler | Source | Route group | Registration condition | Actual auth class | |---|---|---|---|---|---|---| | `GET` | `/api/config` | `ConfigHandler.getConfig` | `handlers/config.go:76` | Management | Always | Admin/session API | ### 2a. Direct handler route registrations Scan all handler route registration methods: ```bash grep -R "\.GET\|\.POST\|\.PUT\|\.DELETE\|\.PATCH\|\.HEAD\|\.OPTIONS\|r.Handle" \ -n transports/bifrost-http/handlers --include='*.go' ``` For a cleaner first pass: ```bash for f in transports/bifrost-http/handlers/*.go; do if grep -q "RegisterRoutes" "$f"; then echo "--- $f" grep -nE 'func \(h .*RegisterRoutes|\.(GET|POST|PUT|DELETE|PATCH|HEAD|OPTIONS)\(|r\.Handle\(' "$f" fi done ``` Record: - HTTP method - path pattern - handler function - file:line - whether route is registered with `middlewares...`, custom middleware, or no middleware - whether route registration is conditional ### 2b. Server-level and conditional route registration Read `RegisterAPIRoutes`, `RegisterInferenceRoutes`, and the middleware-assembly block carefully. Locate them by symbol so the ranges don't drift as the file grows: ```bash SERVER=transports/bifrost-http/server/server.go # RegisterInferenceRoutes — inference route registrations. # Window is generous (+60) so newly added integration handlers don't fall outside it. INFER=$(grep -n "func (s \*BifrostHTTPServer) RegisterInferenceRoutes" "$SERVER" | head -1 | cut -d: -f1) [ -n "$INFER" ] && sed -n "$((INFER)),$((INFER+60))p" "$SERVER" # RegisterAPIRoutes — management/api route registrations (conditional plugin routes live here) API=$(grep -n "func (s \*BifrostHTTPServer) RegisterAPIRoutes" "$SERVER" | head -1 | cut -d: -f1) [ -n "$API" ] && sed -n "$((API)),$((API+120))p" "$SERVER" # Middleware-assembly block — how apiMiddlewares / inferenceMiddlewares are composed, # where AuthMiddleware.APIMiddleware()/InferenceMiddleware() are appended, and where # TracingMiddleware / TransportInterceptorMiddleware are layered onto inference routes. grep -n "apiMiddlewares\s*=\|inferenceMiddlewares\s*=\|AuthMiddleware\.APIMiddleware\|AuthMiddleware\.InferenceMiddleware\|TransportInterceptorMiddleware\|TracingMiddleware" "$SERVER" ``` Important conditional registrations: | Condition | Routes affected | |---|---| | Governance plugin present | `GovernanceHandler.RegisterRoutes` (`/api/governance/*`) | | Logging plugin present | `LoggingHandler.RegisterRoutes` (`/api/logs*`, `/api/mcp-logs*`) | | Semantic cache plugin present | `CacheHandler.RegisterRoutes` (`/api/cache/*`) | | Prompts plugin present | `PromptsHandler.RegisterRoutes` (`/api/prompt-repo/*`) | | Prometheus plugin present | `/metrics` | | Dev mode only | `/api/dev/pprof*` | | OAuth metadata/per-user/consent | Registered without API auth middleware; public by design | | UI catch-all | `/` and `/{filepath:*}`; do not compare as API docs unless user asks | ### 2c. Inference and SDK integration routes Direct unified inference routes live in `handlers/inference.go`, `handlers/asyncinference.go`, `handlers/mcpinference.go`, and `handlers/mcpserver.go`. Provider/framework integration routes are registered through `integrations.GenericRouter` and route config factories. Scan integration route config factories: ```bash grep -R "func Create.*RouteConfigs\|func OpenAI.*Paths\|RouteConfig{" \ -n transports/bifrost-http/integrations --include='*.go' ``` Key factories/routers: | Integration | Code source | |---|---| | OpenAI | `integrations/openai.go` (`CreateOpenAIRouteConfigs`, list models, batch, files, containers, container files, realtime path helpers) | | Anthropic | `integrations/anthropic.go` | | GenAI/Gemini | `integrations/genai.go` | | Bedrock | `integrations/bedrock.go` | | Cohere | `integrations/cohere.go` | | LiteLLM | `integrations/litellm.go` | | LangChain | `integrations/langchain.go` | | PydanticAI | `integrations/pydanticai.go` | | Cursor | `integrations/cursor.go` | | Passthrough | `integrations/passthrough.go` catch-all prefixes | | Realtime WebSocket/WebRTC/client secrets | `handlers/wsresponses.go`, `handlers/wsrealtime.go`, `handlers/webrtc_realtime.go`, `handlers/realtime_client_secrets.go`, `integrations/openai.go` path helper functions | **Do not rely only on direct `r.GET(...)` grep for integrations.** Many integration routes are built from `RouteConfig{ Path: pathPrefix + path, Method: ... }` inside loops. ### 2d. Handle dynamic/catch-all routes correctly Some routes are intentionally catch-all in Go but represented as concrete operations in OpenAPI. Examples: - OpenAI Azure-style deployments: `pathPrefix + "/openai/deployments/{deploymentPath:*}"` dispatches by suffix in `GetHTTPRequestType`. Expand the suffixes documented in the switch (`chat/completions`, `responses`, `embeddings`, etc.) before comparing. - Passthrough routers register `{path:*}` for provider passthrough prefixes. Treat these as catch-all passthrough support. Do not mark every possible downstream provider URL as missing OpenAPI unless the user explicitly wants passthrough documentation. - UI catch-all `/{filepath:*}` is not an API endpoint. Normalize path parameters for comparison: - Exact path match first. - Then normalized match where `{name}`, `{deployment-id}`, and `{deploymentPath:*}` become `{param}`. - If normalized paths match but parameter names differ, report as a **path parameter naming mismatch**, not a missing route. --- ## Step 3: Derive actual auth behavior Auth correctness is part of this skill. Always inspect actual middleware wiring instead of guessing from path names. ### 3a. Read middleware setup Read the auth and middleware setup. Locate the blocks by symbol so the ranges don't drift as the files grow — auth derivation for every route depends on these functions being read correctly, so a stale range would poison the whole audit: ```bash SERVER=transports/bifrost-http/server/server.go MW_FILE=transports/bifrost-http/handlers/middlewares.go # Middleware-assembly block in server.go — where apiMiddlewares / inferenceMiddlewares # get AuthMiddleware.APIMiddleware()/InferenceMiddleware() / Tracing / # TransportInterceptor appended. MW=$(grep -n "apiMiddlewares := commonMiddlewares" "$SERVER" | head -1 | cut -d: -f1) [ -n "$MW" ] && sed -n "$((MW)),$((MW+65))p" "$SERVER" # InferenceMiddleware — auth wiring for /v1/* and provider integration routes. INF=$(grep -n "func (m \*AuthMiddleware) InferenceMiddleware" "$MW_FILE" | head -1 | cut -d: -f1) [ -n "$INF" ] && sed -n "$((INF)),$((INF+15))p" "$MW_FILE" # APIMiddleware — auth wiring for /api/* management routes (whitelist + prefix # whitelist + WebSocket dashboard handling lives here; function body is large). API_MW=$(grep -n "func (m \*AuthMiddleware) APIMiddleware" "$MW_FILE" | head -1 | cut -d: -f1) [ -n "$API_MW" ] && sed -n "$((API_MW)),$((API_MW+180))p" "$MW_FILE" ``` Key logic: - `apiMiddlewares` includes `AuthMiddleware.APIMiddleware()` in OSS when config store exists and auth middleware initializes. - `inferenceMiddlewares` includes `AuthMiddleware.InferenceMiddleware()` in OSS when config store exists and auth middleware initializes. - `OAuthMetadataHandler`, `PerUserOAuthHandler`, and `ConsentHandler` are registered without auth middleware. - `/api/governance/virtual-keys/quota` is registered without `middlewares...` and performs virtual-key authentication inside the handler. - Realtime transport endpoints have special skip behavior in `isRealtimeTransportEndpoint` and handler-level auth extraction. ### 3b. Actual auth classes Classify every route into one of these actual auth classes: | Auth class | Actual code behavior | OpenAPI expectation | |---|---|---| | Public | No auth middleware, or explicitly whitelisted in `APIMiddleware` | Operation must set `security: []` if root `security` is defined | | Admin/session API | `APIMiddleware` protects route; accepts session Bearer token, Basic admin auth, and session cookie fallback. WebSocket dashboard also accepts `?ticket=`/legacy token/cookie. | `BearerAuth` and `BasicAuth`; mention cookie/ticket in description where relevant. Do **not** include `VirtualKeyAuth` unless handler accepts VK. Do **not** include `ApiKeyAuth` unless `x-api-key` is actually accepted for that route. | | Inference API | `InferenceMiddleware` protects route unless `auth_config.disable_auth_on_inference` is true; context extraction supports virtual keys and direct provider/API key headers for inference flows. | Usually `BearerAuth`, `BasicAuth`, `VirtualKeyAuth`, `ApiKeyAuth` |