ai-models

# AI Models — Research & Update Workflow ## When to Use This Skill - Bumping a text, image, or video model to a newer version - Adding a new image/video generation model or provider (Vercel gateway, Replicate, fal.ai) - Updating pricing data (per-token, per-image flat, per-image tiered, per-second) - Verifying model specs (context window, output limit, cost) against providers - Grounding a model's first broad public release date and source - Auditing hosted usage metering against prepaid organization credit --- ## Key Files | File | Role | | ----------------------------------------------- | ------------------------------------------------------------------------------------------ | | `data/ai/facts.json` | Authored model identities, capabilities, provider bindings, published rates and provenance | | `data/ai/service.json` | Authored Grida membership/lifecycle, preferences, request presets and text tiers | | `data/ai/inputs.json` | Authored operation input schemas and documented `x-grida-*` validation rules | | `data/ai/schemas/` | Structural JSON Schema envelopes for facts and service data | | `data/ai/PROVENANCE.md` | Research qualifications and source notes that do not belong in JSON comments | | `packages/grida-ai-models/scripts/generate.mjs` | Deterministic source generation and Rust bundle drift checks | | `packages/grida-ai-models/src/models.ts` | Generated factual literals plus handwritten TypeScript types and lookup helpers | | `packages/grida-ai-models/src/grida/` | Generated service literals plus handwritten ordering and schema-1 compatibility helpers | | `packages/grida-ai/schemas/` | Generated package-local input schemas and operation projections | | `crates/grida-ai/data/` | Generated embedded Rust assets; never a separate authoring home | | `editor/lib/ai/models.ts` | AI Gateway + BYOK provider seam (service catalog from `@grida/ai-models/grida`) | | `editor/lib/ai/ai.ts` | `toMills()` + Replicate call shapes; re-aggregates the shared catalogue under `ai.*` | | `editor/lib/ai/server.ts` | Product model selection and delegated execution adapters | | `editor/lib/platform/` | Account/credit consumers and retained native execution receipts | | `editor/app/(www)/(ai)/ai/models/page.tsx` | Public models catalog page | | `docs/models/index.md` | User-facing models & pricing documentation | ## Shared authoring and generation Author catalogue data in repository-root `data/ai/`. TypeScript packages, the web API and the Rust CLI consume generated projections of those sources. Follow [the shared data guide](../../../data/ai/README.md) and commit authored and generated changes together. Use `lower_snake_case` for authored domain fields. Preserve exact model/provider IDs and standard JSON Schema keywords. The generator explicitly maps text-card and cost fields to existing camelCase TypeScript API fields; preserve schema-1 wire spelling instead of renaming public fields during a data update. Do not hand-edit `generated:*` literal blocks or package/crate JSON copies. Types, lookup helpers, validation semantics and provider adapters remain handwritten code. Update an explicit ID union or type outside generated blocks when a new card requires it; a JSON binding alone does not implement a provider. From the repository root, after editing JSON: ```sh node packages/grida-ai-models/scripts/generate.mjs pnpm --filter @grida/ai-models build pnpm --filter @grida/ai build node packages/grida-ai-models/scripts/generate.mjs --bundle node packages/grida-ai-models/scripts/generate.mjs --bundle --check ``` The first pass updates TS literals and package-local input schemas. The bundle pass uses freshly built TS consumers to derive operations, snapshots and service views, and writes the checked-in Rust assets. Cargo builds use these embedded assets without Node or network access. Builds/typechecks reject stale source projections; the CLI contract gate also checks the complete generated bundle. ## Tools Script: `.agents/skills/ai-models/scripts/model_info.py` (symlink to `.tools/model_info.py`) ### Model lookup ```sh # Text / language models python .agents/skills/ai-models/scripts/model_info.py <model_id> # Image models python .agents/skills/ai-models/scripts/model_info.py --image <model_id> python .agents/skills/ai-models/scripts/model_info.py --image --all ``` Discovery source: `models.dev/api.json`. Accepts exact IDs (`anthropic/claude-sonnet-4.6`) or substring search (`gpt-5.4`). Its `release_date` is a lead to verify, not authoritative provenance to copy into the catalogue. Note: `models.dev` has per-token costs but not per-image tier breakdowns. For per-image pricing (OpenAI quality tiers, BFL flat rates), consult provider docs directly. ### Provider pricing pages | Provider | URL | | ---------- | ----------------------------------------------------------------------------------------------------------- | | OpenAI | `https://developers.openai.com/api/docs/models/<model_id>` | | Anthropic | `https://docs.anthropic.com/en/docs/about-claude/models` | | Google | `https://ai.google.dev/pricing` | | BFL (Flux) | `https://docs.bfl.ml/pricing` | | fal.ai | `https://fal.ai/models/<endpoint-id>` · pricing API: `https://fal.ai/docs/documentation/model-apis/pricing` | | OpenRouter | `https://openrouter.ai/<vendor>/<model>` | ### Providers & model IDs The same model has different ids — and different availability and pricing — across providers; an id is never portable. Two cataloguing patterns: - **text / audio / image_tools / 3D** — one card = one provider or exact endpoint; `id` is in that provider's format, and the `provider` field (or namespace) fixes the route. - **image** — one intrinsic card carries per-provider bindings, like video. The service view adds a primary provider for older single-provider consumers. - **video** — the ecosystem is fragmented, so a card is **canonical** (`vendor/model`, e.g. `google/veo-3.1`) and carries a `providers` record (keyed by provider) of bindings, each with its own call id + meter. Default-provider choice is deferred (see Video Models). Pick a route with `video.binding(card, provider)`. | Provider | Used in catalogue for | ID format / example | | ----------------- | -------------------------- | ------------------------------------------------------------------------------ | | Vercel AI Gateway | text, image, video binding | `google/veo-3.1-generate-001`, `bytedance/seedance-2.0` | | Replicate | audio, image_tools | `google/lyria-3`, `nightmareai/real-esrgan` | | fal.ai | video binding (+ image) | `fal-ai/veo3.1`, `fal-ai/kling-video/v3/pro/image-to-video`, `fal-ai/flux/dev` | | OpenRouter | video binding | `google/veo-3.1`, `google/veo-3.1-fast`, `google/veo-3.1-lite` | - **Availability + price differ per provider.** **Veo 3.1 Lite** is on OpenRouter/fal.ai but **not** the Vercel gateway — a canonical card just omits the Vercel binding. Veo 3.1 audio-on is `$0.40/s` on both Vercel and fal, but fal also meters silent (`$0.20/s`) and 4K, while **OpenRouter exposes only `$0/MTok` token pricing for video — no usable per-second meter (don't invent one).** - **fal.ai** is the broadest video/image catalogue (pay-per-use); billing unit is per-model — per-image, per-megapixel, or per-second video — retrievable from its Platform pricing API. - Image facts are multi-homed across Vercel, fal, and OpenRouter where verified. Service listing is a separate decision; a listed card does not establish that every provider, installed adapter, or request mode can serve it. --- ## What the catalogue is for Keep one package with two explicit entries: `@grida/ai-models` for facts and `@grida/ai-models/grida` for service policy. Service definitions consume facts; the root entry never imports or re-exports Grida policy. Add verified model facts independently of Grida admission. Manage Grida choices in the service definitions, not source declaration order or provider timestamps. The shared execution SDK (`@grida/ai`) retains its Grida defaults by explicitly importing the service entry where needed. Its catalog store accepts an optional snapshot or refresh URL; callers need not inject a catalog. Keep provider execution and refresh lifecycle in the SDK and the schema-1 codec in the service entry; do not restore agent-local adapters or duplicate membership. Preference discipline: an optional default must be listed and nonlegacy. The independent order is partial; unknown and duplicate IDs are errors. Views sort default first, other active models before legacy, then explicit rank, label and ID. Explicit user selections are not replaced by a recommendation. Native subscription and custom-endpoint choices remain with their own runtime owners. The catalogue states what is true and useful **now**. Its shape must never be a record of how recently someone got round to updating it — a stale entry is a wrong answer, not a conservative one. - **Price the steady state, not the promotion.** When a vendor runs an introductory or time-limited rate, catalogue the price that applies once it ends and record the date and qualification in `data/ai/PROVENANCE.md`. Otherwise the promotion expiring is a silent cost increase. Recheck when that date passes: a promotion can also be made permanent, which changes the fact, not the rule. - **Deprecate a card that is still a real choice; remove one that is not.** Grida `legacy: true` (projected as `deprecated` for existing consumers) is for a model someone might still reasonably pick — same price as its successor, or better at something. Delete the entry when the successor is _strictly dominant_ (never worse on any axis, better on at least one): a card nobody should choose is noise in every picker, and keeping it is not caution. Removing a service member changes admission for consumers of the updated catalogue; it is not merely picker cleanup. It does not require deleting factual identity or imply upstream retirement. TS consumers configured to refresh a published snapshot receive updates through that refresh lifecycle (`docs/wg/platform/hosted-ai.md`). The Rust CLI embeds its catalogue and does not refresh it at runtime: regenerate its assets and ship a new CLI version for updated discovery data. Do not assume a remote catalogue update revokes an installed binary's bundled knowledge. Hosted authorization remains a separate runtime boundary. Preserve schema-1 membership and legacy fields when publishing; installed snapshot clients ignore additive preferences. The v1 snapshot still has broad GG/BYOK membership and per-family fallback behavior; adapter support and authorization remain independent checks. ## Release dates and provenance Every bundled entry carries a `release` object: ```json { "date": "2026-07-09", "basis": "model", "source_url": "https://vendor.example/release-note" } ``` Use a `YYYY-MM-DD` date, or `null` only when an endpoint day is unknown. `basis` is `model` or `provider_endpoint`. The date means the earliest day the exact named model or variant became broadly available. A public preview counts; a closed, invitation-only, or limited preview does not. This is intrinsic model metadata, so adding a provider binding does not change a `basis: "model"` release. Use `basis: "provider_endpoint"` only when the release fact describes a serving route because no exact upstream model launch can be established. An endpoint-shaped card may still use `basis: "model"` when its exact underlying model and launch are documented. Do not substitute any of these: - snapshot `generated_at` - the date Grida added the card - the date one provider added a binding - a later GA date when an exact public preview date exists - an API object's opaque `created` timestamp Source priority for release facts: 1. Vendor release note, changelog, announcement, or model card that names the exact variant. 2. Vendor-maintained repository or official provider documentation for the exact endpoint. 3. First-party vendor social announcement when no durable release page exists. 4. Serving-provider history, only for `basis: "provider_endpoint"` or when the vendor has no usable record. 5. `models.dev` only to discover candidates; verify its date against one of the sources above. If no authoritative source establishes the exact day, keep `date: null` with an HTTPS source showing the endpoint's history. Never infer a day from search-result ordering, repository commit time, or Grida history. Base snapshot types keep the field optional solely for older snapshots and custom models; every bundled card must include it, and tests enforce valid calendar dates, complete provenance, and the narrow `null` rule. ## Text Models Author text facts in `data/ai/facts.json` under `text.catalog`. The generated TypeScript consumer exposes `models.text.catalog: Record<CatalogId, ModelSpec>`. Author Grida tier assignments in `data/ai/service.json` under `tiers`; each must resolve to a listed service member. Authored fields to update per model: - `id` — gateway format: `provider/model-name` - `label` — human-readable name - `release` — grounded date, basis, and first-party source under the contract above - `context_window`, `output_limit` — use `model_info.py` as a discovery lead and verify against provider documentation - `cost` — `input`, `output`, optional `cache_read` and `cache_write`, per 1M tokens; TS projects these to its existing camelCase fields ## Image Models Author image facts in `data/ai/facts.json` under `image.models`, and membership, legacy state, primary-provider choice and request presets in `data/ai/service.json`. The generated service view joins them. Editor consumers reach that joined view via `import { ai } from "@/lib/ai/ai"` (which also adds `ai.toMills` and `ai.server.methods.*`). ### Pricing types Three pricing schemes, modeled as discriminated union `ImageModelPricing`: ``` per_image_tiered — quality x size tiers (e.g. OpenAI) { type: "per_image_tiered", tiers: { "medium/1024x1024": 0.034, ... } } per_image_flat — single price per image (e.g. BFL Flux) { type: "per_image_flat", usd: 0.06 } per_token — charged by token (e.g. Google Gemini) { type: "per_token", input: 0.5, output: 3.0 } ``` ### Fields per model - `pricing` — real provider data, one of the three types above - `avg_cost_usd` — existing fallback billable-cost estimate, not a provider quote. Retained compatibility surface; do not treat it as independently verified pricing or expand it into service routing/billing policy. - `release` — intrinsic model release; do not use a provider-binding date - `min_width`, `max_width`, `min_height`, `max_height`, `sizes` — dimension constraints - Add new factual cards in JSON and update the handwritten `ImageModelId` type union outside the generated block in `src/models.ts` ### New providers Image generation currently routes through the Vercel AI Gateway (`gateway.image(id)`); fal.ai is the main alternative for models the gateway lacks (see Providers & model IDs). For a new provider: - Verify the gateway supports it (or wire a new `provider` label for fal.ai / OpenRouter) - Add to the `Vendor` type if needed - Add a logo component and register in the `Logos` map on the models page ## Video Models