api-filters

# v3 API Filter Parsing You are helping the user add or modify AIP-style query-parameter filters on an OpenMeter v3 list endpoint. OpenMeter follows the **Kong AIP filter spec** (NOT Google AIP-160 expression syntax). Filters use the deepObject query-parameter encoding `?filter[field][op]=value`. The implementation is split across three layers: - `api/v3/filters/` — API-layer filter types, `Parse` entry point, and `FromAPI*` converters - `pkg/filter/` — internal predicate model with `Validate()`, `Select(field)`, `ApplyToQuery(...)` helpers - domain service input structs — hold the already-converted `*filter.*` predicates ## Relationship to other skills Filtering straddles two layers that the repo skill set keeps separate: - **TypeSpec / OAS side** — `Common.*FieldFilter` types, `Shared.ResourceFilters`, `deepObject` exposure, label dot-notation. See `../api/rules/aip-160-filtering.md` (the canonical Kong AIP-160 rule for OpenMeter). Use the `/api` skill when you also need to scaffold or modify the TypeSpec operation itself. - **Go implementation side** — what this skill covers: `api/v3/filters.Parse`, the API-layer filter structs, `FromAPI*` helpers, service input wiring, adapter `filter.ApplyToQuery`, gotchas. If you are adding a brand-new filterable endpoint, invoke `/api` first to wire up the TypeSpec + handler shell, then come back here for the conversion + adapter code. If you are only adding/modifying filters on an existing endpoint, this skill is enough on its own. ## Context - **API-layer package:** `api/v3/filters/` — API-shaped filter structs and `FromAPI*` converters - **Internal predicate model:** `pkg/filter/` — implements the `Filter` interface (`Validate`, `Select`, `IsEmpty`, …); used by Ent query builders - **Reference implementation in use:** `api/v3/handlers/customers/list.go` (handler) + `openmeter/customer/adapter/customer.go` (adapter) + `openmeter/customer/customer.go` (service input struct) - **Kong AIP spec for filtering:** `../api/rules/aip-160-filtering.md` ## Architecture: three-layer conversion ``` TypeSpec Common.*FieldFilter │ (make gen-api) ▼ api.Filter* (generated OAS types) ─┐ │ │ handler decode layer api/v3/filters.Filter* (API-layer types) │ calls filters.FromAPIFilter*(...) │ │ pkg/filter.Filter* (predicate model) ─┘ stored on service input struct │ ▼ adapter layer filter.ApplyToQuery(query, input.Field, dbField) ``` Rules: - The **handler** converts `params.Filter.X` (API-shaped) → `*filter.X` (predicate) using `filters.FromAPIFilter*`. - The **service input struct** holds `*filter.FilterString`, `*filter.FilterTime`, `*filter.FilterULID`, etc. — NOT the API-layer types. - The **adapter** calls `filter.ApplyToQuery(query, input.Field, dbField)` to attach the predicate to the Ent query. - `filters.Parse` is called by the **generated** deepObject binding layer in `api/v3/api.gen.go`, not by handlers. Handlers receive `params.Filter` already populated. ## Filter Grammar (Kong AIP) **Encoding:** deepObject query parameters. Two-level brackets identify field and operator: ``` filter[field]=value # shorthand → eq filter[field][eq]=value # exact match filter[field][neq]=value # not equal (also returns NULLs) filter[field][contains]=value # substring match (case-insensitive on strings) filter[field][oeq]=a,b,c # one-of-equal (comma-separated, max 50 items) filter[field][ocontains]=a,b # one-of-contains filter[field][gt]=value # greater than filter[field][gte]=value # greater than or equal filter[field][lt]=value # less than filter[field][lte]=value # less than or equal filter[field] # bare key → exists=true (presence check) filter[field][exists] # explicit existence check filter[field][nexists] # absence check (only for additionalProperties maps like labels) filter[labels.key_1][eq]=val # dot-notation: only the FIRST dot is a delimiter ``` Operator constants live in `api/v3/filters/parse.go` as `OpEq`, `OpNeq`, `OpGt`, `OpGte`, `OpLt`, `OpLte`, `OpContains`, `OpOeq`, `OpOcontains`, `OpExists`, `OpNexists`. ### API-layer filter types (`api/v3/filters/filter.go`) | Go type | Fields | | ------------------- | ------------------------------------------------------------------------------ | | `FilterBoolean` | `Eq` | | `FilterNumeric` | `Eq`, `Neq`, `Oeq`, `Gt`, `Gte`, `Lt`, `Lte` | | `FilterDateTime` | `Eq`, `Gt`, `Gte`, `Lt`, `Lte` (all `*time.Time`; no `Neq`/`Oeq`) | | `FilterString` | `Eq`, `Neq`, `Gt`, `Gte`, `Lt`, `Lte`, `Contains`, `Oeq`, `Ocontains`, `Exists` | | `FilterULID` | `Eq`, `Neq`, `Contains`, `Oeq`, `Ocontains`, `Exists` (no range ops) | | `FilterStringExact` | `Eq`, `Neq`, `Oeq` (no `Exists`, no `Contains`) | | `FilterLabel` | `Eq`, `Neq`, `Contains`, `Oeq`, `Ocontains` (label map value predicates) | | `FilterLabels` | type alias: `map[string]FilterLabel` | The wire operator for `Exists` is plain `exists` (see `OpExists` in `api/v3/filters/parse.go`), matching its `json:"exists,omitempty"` tag — don't confuse it with the unrelated `$`-prefixed Mongo-style tags used by the v1 API (`api/api.gen.go`). **Important:** the API-layer types do NOT have `Validate()` methods. Validation (mutual exclusivity, complexity bounds, format checks) happens on the internal `pkg/filter.*` predicates — typically from the service input struct's own `Validate()`, calling `f.Validate()` on each non-nil filter. ### `pkg/filter` predicates | Predicate | Produced by converter | Notes | | ------------------------ | ----------------------------- | -------------------------------- | | `*filter.FilterString` | `FromAPIFilterString` | Also used by `FromAPIFilterLabel`, `FromAPIFilterStringExact` | | `*filter.FilterULID` | `FromAPIFilterULID` | Embeds `FilterString` | | `*filter.FilterFloat` | `FromAPIFilterNumeric` | (note: not `FilterNumeric`) | | `*filter.FilterTime` | `FromAPIFilterDateTime` | RFC-3339 already parsed to `time.Time` by `Parse` | | `*filter.FilterBoolean` | `FromAPIFilterBoolean` | | | `map[string]filter.FilterString` | `FromAPIFilterLabels` | Label map flatten | The `Filter` interface (`pkg/filter/filter.go:19`) exposes `Validate()`, `ValidateWithComplexity(maxDepth int)`, `Select(field string) func(*sql.Selector)`, `SelectWhereExpr(...)`, and `IsEmpty()`. ### Multi-filter semantics - Multiple `filter[...]` parameters with **different fields** combine with **AND**. - A single field with `oeq` / `ocontains` combines its values with **OR** (`IN (...)` or `OR ILIKE ...`). - A single field with multiple operators (e.g. both `gte` and `lte`) is wrapped by the converter into `And{...}` of single-operator `pkg/filter` nodes. - The bare-key existence shortcut maps to `IS NOT NULL`; `nexists` only works on schemaless maps (`labels`, `metadata`). ### Validation is done by `pkg/filter` Mutual-exclusivity and format rules (e.g. "multiple operators on one node", ULID format, complexity depth) are enforced by `*filter.FilterX.Validate()` — not by the API-layer types. A typical service input `Validate()` looks like: ```go if i.Key != nil { if err := i.Key.Validate(); err != nil { errs = append(errs, models.NewGenericValidationError(fmt.Errorf("invalid key filter: %w", err))) } } ``` ### Hard limits (security, `api/v3/filters/parse.go:16-19`) - **1024 bytes** per single value (`maxFilterValueLength`) - **50 items** per comma-separated list (`maxCommaSeparatedItems`) - **Repeated query params for the same key are rejected** (e.g., `?filter[f][eq]=a&filter[f][eq]=b`) - **Unknown filter fields are rejected** before any other validation (`checkUnknownFilterKeys`) ## Workflow Follow these steps in order. Use the `/api` skill alongside this one when you also need to touch TypeSpec. ### Step 1: Define the filterable fields in TypeSpec In `api/spec/packages/aip/src/<domain>/operations.tsp`, define a named filter model for the list operation and expose it as `filter` with `style: "deepObject", explode: true`. Use the `Common.*FieldFilter` types from `common/parameters.tsp` — **do not hand-roll filter models**. The canonical rule for *which* `Common.*FieldFilter` type to pick, the `Shared.ResourceFilters` spread, label dot-notation, and OAS documentation requirements is `../api/rules/aip-160-filtering.md`. That rule also includes the TypeSpec type ↔ Go `filters.Filter*` mapping. Read it once before picking types — this skill is not the source of truth for the TypeSpec side. The events list endpoint (`api/spec/packages/aip/src/events/operations.tsp`) and the customer list endpoint are the canonical worked examples. After editing TypeSpec, run `make gen-api` so the generated `params.Filter` struct in `api/v3/api.gen.go` picks up the new fields. ### Step 2: Store `pkg/filter` predicates on the service input struct In your domain service input type, add fields typed as **pkg/filter predicates**, not API-layer types. Example from `openmeter/customer/customer.go:296`: ```go type ListCustomersInput struct { Namespace string pagination.Page OrderBy string Order sortx.Order Key *filter.FilterString Name *filter.FilterString PrimaryEmail *filter.FilterString // ... } func (i ListCustomersInput) Validate() error { var errs []error // ... if i.Key != nil { if err := i.Key.Validate(); err != nil { errs = append(errs, models.NewGenericValidationError(fmt.Errorf("invalid key filter: %w", err))) } } // ... return models.NewNillableGenericValidationError(errors.Join(errs...)) } ``` Pick the narrowest predicate: `filter.FilterString` for strings, `filter.FilterULID` for ULID columns, `filter.FilterFloat` for numbers, `filter.FilterTime` for timestamps, `filter.FilterBoolean` for bools. ### Step 3: Convert API filters in the HTTP handler In the handler decoder (the first argument to `httptransport.NewHandlerWithArgs`), call the matching `filters.FromAPIFilter*` helper against the generated `params.Filter.<field>` and assign to the request. The canonical pattern is in `api/v3/handlers/customers/list.go`: ```go import ( "github.com/openmeterio/openmeter/api/v3/apierrors" "github.com/openmeterio/openmeter/api/v3/filters" ) if params.Filter != nil { key, err := filters.FromAPIFilterString(params.Filter.Key) if err != nil { return ListCustomersRequest{}, apierrors.NewBadRequestError(ctx, err, apierrors.InvalidParameters{ {Field: "filter[key]", Reason: err.Error(), Source: apierrors.InvalidParamSourceQuery}, }) } req.Key = key name, err := filters.FromAPIFilterString(params.Filter.Name) if err != nil { return ListCustomersRequest{}, apierrors.NewBadRequestError(ctx, err, apierrors.InvalidParameters{ {Field: "filter[name]", Reason: err.Error(), Source: apierrors.InvalidParamSourceQuery}, }) } req.Name = name } ``` Notes: - Handlers do **not** call `filters.Parse` directly — the generated OAS binding layer does that and surfaces any parse/validation errors as `InvalidParamFormatError` before the handler runs. - Every `FromAPIFilter*` returns `(*filter.X, error)`. The error channel is reserved for helpers that can fail (e.g. future format checks); today most helpers only return `(nil, nil)` on a nil input, but always handle the error for forward-compatibility. - On error, wrap with `apierrors.NewBadRequestError(...)` using `Source: apierrors.InvalidParamSourceQuery` and `Field: "filter[<field>]"`. ### Step 4: Apply to the query in the adapter Adapters use `filter.ApplyToQuery(query, input.Field, dbField)` — a generic helper that: 1. Returns the query unchanged when the predicate is nil. 2. Builds an Ent predicate via `pkg/filter.SelectPredicate[P](...)`. 3. Calls `q.Where(*p)` when the predicate is non-empty. From `openmeter/customer/adapter/customer.go:52`: