// Check Kiln's model list for deprecated or sunset models across all providers. Use when the user wants to find deprecated models, check model availability, audit the model list for stale entries, or mentions model deprecation/sunset/end-of-life.
| name | kiln-check-deprecation |
| description | Check Kiln's model list for deprecated or sunset models across all providers. Use when the user wants to find deprecated models, check model availability, audit the model list for stale entries, or mentions model deprecation/sunset/end-of-life. |
Audit libs/core/kiln_ai/adapters/ml_model_list.py to find models that have been deprecated, removed, or are approaching end-of-life across all providers. All checks are free — they use model-listing endpoints, not inference.
curl, uv run, and python3 commands that hit the network MUST use required_permissions: ["all"]. The sandbox blocks network access..env before running check scripts: export $(grep -v '^#' .env | xargs)gcloud CLI authentication. Before running, ask the user to run gcloud auth login if they haven't recently. If gcloud auth print-access-token fails, prompt the user to authenticate.deprecated=True on individual provider entries. It never removes models or providers from the list.deprecated=False (the default). Don't re-check entries already marked deprecated.Run the extraction script from the repo root:
python3 .agents/skills/kiln-check-deprecation/scripts/extract_models.py > /tmp/kiln_extracted.json
This fetches the published model list from https://remote-config.getkiln.ai/kiln_config_v2.json and extracts:
It skips entries already marked deprecated=True.
Output: JSON to stdout (pipe to file), human summary to stderr.
The JSON contains:
deprecated_count: number of already-deprecated entriesproviders: dict of provider_name → sorted unique list of model_idsentries: list of {enum, provider, model_id} for each non-deprecated entryexport $(grep -v '^#' .env | xargs)
python3 .agents/skills/kiln-check-deprecation/scripts/check_provider.py all > /tmp/kiln_check_results.json
export $(grep -v '^#' .env | xargs)
python3 .agents/skills/kiln-check-deprecation/scripts/check_provider.py openrouter > /tmp/kiln_check_openrouter.json
The script handles all provider API quirks automatically:
{data: [...]})models/ — stripped automaticallyx-api-key header, not Authorization: Bearerexpiration_date fields/v1/models only lists serverless models; the script instead checks each model individually via the model detail API (GET /v1/{model_id}), which covers all tiers (serverless, on-demand, fine-tune). A model is only flagged as missing if it returns HTTP 404 from the detail API.:exacto is a virtual routing suffix (quality-first provider sorting) that never appears in model listings. The script strips it before checking. :free and :thinking are real model entries that appear in the listing when available — if they're missing, it's a genuine removal.x-goog-user-project header. Requires gcloud CLI auth. Kiln entries may use meta/ prefix for LiteLLM routing — stripped automatically. Versioned aliases (e.g. gemini-2.0-flash-001 → gemini-2.0-flash) are handled by stripping 3-digit version suffixes.Output: JSON to stdout with per-provider results, human summary to stderr.
Each provider result contains:
missing: model_ids not found in the provider's listingexpiring: model_ids with upcoming expiration dates (OpenRouter only)entries_to_deprecate: full enum/provider/model_id entries for each missing modelskipped / error: if credentials missing or API call failed| Provider | Env Var | Auth |
|---|---|---|
| OpenRouter | (none — public API) | None |
| OpenAI | OPENAI_API_KEY | Bearer |
| Anthropic | ANTHROPIC_API_KEY | x-api-key |
| Gemini API | GEMINI_API_KEY | Query param |
| Fireworks AI | FIREWORKS_API_KEY | Bearer |
| Together AI | TOGETHER_API_KEY | Bearer |
| SiliconFlow CN | SILICONFLOW_CN_API_KEY | Bearer |
| Cerebras | CEREBRAS_API_KEY | Bearer |
| Groq | GROQ_API_KEY | Bearer |
| Vertex AI | VERTEX_PROJECT_ID + gcloud CLI auth | OAuth (gcloud) |
Requires aws CLI configured. If not installed, skip.
aws bedrock list-foundation-models --output json | \
jq '[.modelSummaries[] | {modelId, status: .modelLifecycle.status, endOfLife: .modelLifecycle.endOfLifeTime}]'
Flag any model where status == "LEGACY" or endOfLifeTime is approaching.
Cross-reference against LiteLLM's public model database for deprecation_date fields. This mostly covers Azure/OpenAI date-stamped models but is a useful secondary signal.
curl -s https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json | \
jq 'to_entries[] | select(.value.deprecation_date != null) | {model: .key, deprecation_date: .value.deprecation_date}'
These don't have meaningful deprecation to check:
ollama — local models, user-managedkiln_fine_tune — internalkiln_custom_registry — internalopenai_compatible — user-configureddocker_model_runner — localazure_openai — typically mirrors OpenAI, covered by OpenAI checkhuggingface — serverless inference, model availability variesRead the JSON output from check_provider.py and present findings in a clear table to the user:
Deprecation Audit Results
=========================
❌ REMOVED (model not found in provider's model list):
- ModelName.gemini_1_5_pro → gemini_api (model_id: gemini-1.5-pro)
- ModelName.gemini_1_5_pro → openrouter (model_id: google/gemini-pro-1.5)
⚠️ EXPIRING SOON (within 90 days):
- ModelName.claude_3_7_sonnet → openrouter (model_id: anthropic/claude-3.7-sonnet, expires: 2026-05-05)
⚠️ LEGACY (Bedrock lifecycle status):
- ModelName.mistral_large → amazon_bedrock (model_id: mistral.mistral-large-2407-v1:0, status: LEGACY)
✅ ALL CLEAR:
- openai: 20/20 models found
- anthropic: 8/11 models found (3 deprecated above)
⏭️ SKIPPED (no credentials):
- groq
- amazon_bedrock
Group by provider within each category. Use the entries_to_deprecate field from the JSON to get the ModelName enum for each entry.
For each confirmed-removed model, set deprecated=True on the affected KilnModelProvider entry.
To find the line to edit, grep for the model_id in ml_model_list.py:
grep -n 'model_id="<model_id>"' libs/core/kiln_ai/adapters/ml_model_list.py
Then add deprecated=True, after the model_id= line in the matching KilnModelProvider block.
Rules:
KilnModel are deprecated, note this to the user — they may want to consider removing the model entirelyLEGACY status, mark deprecated (the model still works but is on its way out)Ask the user to confirm before making changes. Present the list of changes and wait for approval.
After marking providers deprecated, run a quick sanity check:
grep -c "deprecated=True" libs/core/kiln_ai/adapters/ml_model_list.py
Ensure the count matches what you expect (previous count + newly marked).
gcloud auth print-access-token works, or skip Vertex)deprecated=True set on confirmed entriesCheck Kiln's fine-tunable model list for deprecated or unsupported base models. Use when the user wants to audit fine-tuning support, check if fine-tune base models are still valid, or mentions fine-tune model deprecation.
Add new AI models to Kiln's ml_model_list.py and produce a Discord announcement. Use when the user wants to add, integrate, or register a new LLM model (e.g. Claude, GPT, DeepSeek, Gemini, Kimi, Qwen, Grok) into the Kiln model list, mentions adding a model to ml_model_list.py, or asks to discover/find new models that are available but not yet in Kiln.