# PublicProviderConf
PublicProviderConf is a TypeScript CLI and library that aggregates the canonical [models.dev catalog](https://models.dev/api.json) together with several custom provider integrations (PPInfra, TokenFlux, Groq, Qiniu-hosted snapshots, and others). The tool normalizes capabilities, fills in missing metadata, and emits standardized JSON payloads that downstream apps can consume without bespoke adapters.
## Highlights
- Unified JSON schema for every provider with consistent capability flags
- Concurrent fetcher pipeline that merges live APIs with maintained templates
- Configurable CLI built with Commander + Vite for both dev (ts-node) and production builds
- Automated GitHub Actions workflow that can publish fresh `dist/` artifacts and sync them to CDN storage
## Data Sources & Coverage
The aggregated dataset starts with the upstream `https://models.dev/api.json`. During each run we overlay:
- Provider overrides from `manual-templates/`
- Live fetchers for operators that are not (yet) covered by models.dev, such as `ppinfra`, `tokenflux`, and `groq`
- Lightweight snapshots for ecosystems like `ollama` and `siliconflow`
After post-processing, the CLI writes the final catalog to `dist/`. Key outputs include:
- `dist/all.json` – aggregated providers, model counts, and capability rollups
- `dist/{provider}.json` – normalized payload for each individual provider
Consumers can point CDN tooling at the `dist/` directory (see GitHub Actions workflow) to publish the latest data for public access.
## Extended Capability Fields
The dataset keeps legacy capability fields unchanged for backward compatibility. Existing fields such as `reasoning`, `reasoning_effort`, and `interleaved` are preserved as-is.
When we need richer model-level metadata, we add it under `extra_capabilities` instead of changing old fields. The first supported extension is `extra_capabilities.reasoning`, which represents a fixed reasoning portrait for the model itself.
### Design Rules
- `extra_capabilities.reasoning` describes the model portrait, not provider-specific parameter exposure
- The same model should have the same reasoning portrait across providers whenever possible
- If a model is not covered by the portrait registry, `extra_capabilities.reasoning` is omitted
- Legacy fields remain the compatibility layer for downstream consumers that already depend on them
- Legacy `reasoning.supported` is the compatibility signal that a model supports reasoning; provider-specific controls such as `effort` may still depend on the current provider surface
- `visibility` is a normalized output-visibility field, not a provider-native parameter name such as Anthropic's `thinking.display`
- `omitted` is distinct from `hidden`: `omitted` tracks Anthropic's omitted thinking display, while `hidden` remains the generic non-exposed visibility bucket used by other providers
- Portrait defaults may intentionally differ from an upstream provider's raw default when we choose a better client-side starting point for a supported reasoning model
### Example
```json
{
"id": "claude-opus-4.7",
"reasoning": {
"supported": true,
"default": false
},
"extra_capabilities": {
"reasoning": {
"supported": true,
"default_enabled": false,
"mode": "effort",
"effort": "high",
"effort_options": ["low", "medium", "high", "xhigh", "max"],
"interleaved": true,
"summaries": true,
"visibility": "omitted",
"continuation": ["thinking_blocks"]
}
}
}
```
### `extra_capabilities.reasoning` Fields
- `supported`: whether the model family supports reasoning
- `default_enabled`: whether reasoning should be enabled by default in the model portrait
- `mode`: one of `budget`, `effort`, `level`, `fixed`, or `mixed`; the primary control, not an exhaustive capability list. An effort-first model can still support an advanced budget control.
- `budget`: token-budget style reasoning controls such as min/max/default/auto/off
- `effort`: default reasoning effort for this portrait, which may be a client-friendly default rather than the upstream provider's raw default
- `effort_options`: distinct model-effective effort levels after aliases or mappings; provider-specific controls remain in legacy `reasoning_options`
- `verbosity`: default reasoning verbosity
- `verbosity_options`: supported verbosity levels
- `level`: default reasoning level for models that use level-based controls
- `level_options`: supported reasoning levels
- `interleaved`: whether interleaved reasoning is part of the model portrait
- `summaries`: whether reasoning summaries are part of the model portrait
- `visibility`: one of `hidden`, `summary`, `full`, `mixed`, or `omitted`; normalized across providers rather than mirroring vendor-native parameter names
- `continuation`: continuation mechanism hints such as `thinking_blocks` or `thought_signatures`
- `notes`: short implementation notes when the model family has important quirks
### Provider Control Constraints
`reasoning_options` describes controls exposed by this provider's API. Each option may
include `exclusive_with`, an array of other option `type` values in the same model's
list. A declaration in either direction means the pair cannot be sent together:
```json
[
{ "type": "effort", "values": ["none", "low", "medium", "xhigh"], "exclusive_with": ["budget_tokens"] },
{ "type": "budget_tokens", "min": 0, "max": 262144 }
]
```
- References use catalog control types, not wire parameter names. Both `budget` and
`budget_tokens` exist in legacy data; adapters translate the declared control into
the endpoint's native parameter (for example, Alibaba's `thinking_budget`).
- An absent constraint means unknown, not permission to combine controls. Do not
infer exclusivity for every model: some Claude models accept effort plus budget.
- Model portraits do not prove a proxy exposes the same controls. Provider-local
declarations take precedence; `custom-provider` is official-source fallback
metadata, not an endpoint capability guarantee.
- For effort-first models with an exclusive budget alternative, omit `budget.default`
so older consumers do not automatically inject a conflicting budget. Keep the
range for an explicit advanced override. Defaults describe a starting point;
they are not instructions to send every default on every request.
- Consumers should retain one active user choice (provider default, effort, or
explicit budget), then resolve it against the provider before serializing. Switching
between exclusive controls must not send a stale value from the inactive control.
"Provider default" sends neither override; it is not equivalent to choosing the
highest effort or copying a numeric budget default.
For `zai` models `glm-5.3`, `glm-5.3-flash`, and `glm-5.3-flashx`, `tool_choice: true`
means the endpoint accepts only `"auto"` for [`tool_choice`](https://docs.z.ai/guides/capabilities/function-calling#core-parameter-description).
### Current Coverage
The initial portrait registry covers these model families:
- OpenAI: `gpt-5`, `gpt-5.1`, `o3`, `o4-mini`, and close variants/aliases
- Anthropic: Claude 3.7, Claude 4.x manual/adaptive thinking, and Claude Fable/Mythos effort portraits
- Google: Gemini 2.5 and Gemini 3 reasoning-capable models
- DeepSeek: DeepSeek V4 Flash and V4 Pro effective effort portraits
- MiniMax: MiniMax M2.7 and MiniMax M2.7 highspeed interleaved reasoning portraits
## Getting Started
### Prerequisites
- Node.js 18+
- pnpm 8+
### Install & Build
```bash
git clone https://github.com/ThinkInAIXYZ/PublicProviderConf.git
cd PublicProviderConf
pnpm install
pnpm build
```
`pnpm build` runs both Vite targets: the library bundle under `build/` and the bundled CLI at `build/cli.js`.
## CLI Usage
### Development Flow
```bash
pnpm run dev # ts-node, defaults to fetch-all
ts-node src/cli.ts fetch-all # explicit command
ts-node src/cli.ts fetch-providers -p ppinfra,tokenflux
```
### Production / Bundled CLI
```bash
pnpm build
pnpm start # equivalent to node build/cli.js fetch-all
node build/cli.js fetch-providers -p ppinfra,tokenflux -o ./dist
```
### Global Installation (optional)
```bash
pnpm install -g .
public-provider-conf fetch-all
public-provider-conf fetch-providers -p ppinfra,tokenflux
```
### Command Reference
```bash
public-provider-conf fetch-all [options]
public-provider-conf fetch-providers -p <providers> [options]
Options:
-p, --providers <providers> Comma-separated provider IDs
-o, --output <dir> Output directory (default: dist)
-h, --help Show command help
```
## Supported Providers
- models.dev catalog (OpenAI, Anthropic, OpenRouter, Google Gemini, Vercel, GitHub Models, DeepSeek, etc.)
- PPInfra (live API)
- TokenFlux (marketplace API)
- Groq (requires `GROQ_API_KEY`)
- AIHubMix (live API)
- Cheaper Inference (live API, no key needed)
- BurnCloud (manual template)
- Ollama (snapshot templates)
- SiliconFlow (snapshot templates)
Adding a new provider usually involves implementing `Provider` under `src/providers/`, adding configuration to `src/config/app-config.ts`, and optionally contributing templates to `manual-templates/`.
AIHubMix is fetched live from `https://aihubmix.com/api/v1/models` so you have a ready-made fallback dataset for models that aren't yet covered by the primary `models.dev` catalog. Keeping the provider enabled ensures `dist/aihubmix.json` stays current without manual snapshots.
BurnCloud rides exclusively on `manual-templates/burncloud.json`, which is regenerated from `models.txt` snapshots so the CLI can ship the latest catalog without hitting proprietary endpoints. Keep that template up to date (and re-run `jq -c` to refresh `dist/burncloud.json`) whenever BurnCloud publishes new models or capability metadata.
The Google entry for `gemini-4-argon` sets `temperature: false` because Google ignores `temperature`, `top_p`, and `top_k`. Its introductory prices are $2 input / $10 output and $0.10 cached input per million tokens through November 30, 2026; from December 1, 2026, these become $4 / $20 and $0.20. The template records `metadata.pricingValidUntil` and the scheduled rates without overriding upstream costs or automatically switching prices. Its 32,768 output limit is a conservative catalog setting. Argon is outside the curated `custom-provider` selection. See the [official migration and pricing guide](https://ai.google.dev/gemini-api/docs/latest-model).
The OpenAI template supplements the catalog with `gpt-image-2.5-sunburst` and
`gpt-image-2.5-flare`, following the existing `gpt-image-2` pricing structure.
Standard processing uses $5 text input / $1.25 cached text input and $8 image
input / $2 cached image input / $30 image output per million tokens. Generic
`cost` fields retain text input, cached text input, and image output rates;
`metadata.pricing` preserves the separate image input rates. Equal token rates
do not imply equal cost per image. Cached input pricing applies only to the
Responses API image generation tool, not direct Images API requests. No
unverified context/output token limits or release dates are added. See the
official [Sunburst model](https://developers.openai.com/api/docs/models/gpt-image-2.5-sunburst),
[Flare model](https://developers.openai.com/api/docs/models/gpt-image-2.5-flare), and
[image generation pricing guidance](https://developers.openai.com/api/docs/guides/image-generation#cached-input-pricing).
## Project Structure
```
src/
├─ cli.ts # CLI entry point (Commander)
├─ commands/ # fetch-all, fetch-providers commands
├─ config/ # default configuration and loaders
├─ fetcher/ # HTTP + file-based fetch orchestrators
├─ models/ # Type definitions and helpers
├─ output/ # Writers, validators, distribution helpers
└─ providers/ # Individual provider integrations
```
## Configuration & Environment
- Default endpoints and flags live in `src/config/app-config.ts`
- Override the models.dev endpoint with `MODELS_DEV_API_URL`
- Provide an offline fallback snapshot via `MODELS_DEV_SNAPSHOT_PATH`
- Set API secrets (e.g. `GROQ_API_KEY`) in your environment or CI secrets
## Development Tips
```bash
pnpm install # install deps
pnpm run dev # ts-node dev loop
pnpm build # Vite builds (library + CLI)
pnpm start # run bundled CLI
```
Use `DEBUG=true` when you need verbose fetcher logs.
## CI & Distribution
The `.github/workflows/fetch-models.yml` workflow builds the project, runs the fetch pipeline, validates JSON output, and can sync the `dist/` directory to a Qiniu CDN bucket for distribution. Ensure the Qiniu credentials are available as repository secrets when enabling the CDN path.
## Contributing
1. Fork the repository
2. Create a feature branch
3. Run `pnpm build` before raising a PR
4. Submit your PR with relevant test notes and updated templates if applicable
Issues and ideas are always welcome.
## License
Apache-2.0 license