Audit AI model providers in sources.js against live provider APIs. Detects deprecated, new, and config-changed models. Uses lightweight audit state, JSON-based researcher output, and streamlined diff application. Trigger with /skill:update_models or when user says "update models", "audit models", "check models", "verify providers", "bump models".
Audit AI model providers in sources.js against live provider APIs. Detects deprecated, new, and config-changed models. Uses lightweight audit state, JSON-based researcher output, and streamlined diff application. Trigger with /skill:update_models or when user says "update models", "audit models", "check models", "verify providers", "bump models".
Update Models — Provider Audit Skill v2
Audits model providers using a lightweight state file + structured JSON diffs. No report files, no archives — git history is the source of truth.
Architecture
File
Purpose
sources.js
All model definitions per provider — the only file that matters
audit_state.json
Per-provider audit timestamps + model ID fingerprints (skip unchanged)
changelog/vX.Y.Z.md
Per-version changelog (still maintained, but audit data lives in git commits
Core Principle
Git IS the archive. Every sources.js change is already committed with a descriptive message. The provider_updates/ folder and per-version report archives are dead weight — git log already shows what changed and when.
Workflow
Phase 1: Load State & Plan
# Read audit statecat audit_state.json 2>/dev/null || echo'{}'# Count models per provider in sources.js
node -e "
import {sources} from './sources.js';
for (const [k, v] of Object.entries(sources)) {
const count = v.models?.length || 0;
const last = JSON.parse(require('fs').readFileSync('audit_state.json','utf8')||'{}')[k]?.lastAudited||'never';
console.log(k + '|' + count + '|' + last);
}
"
Present a compact table:
| # | Provider | Models | Last Audited | Status |
|---|----------|--------|-------------|--------|
| 1 | nvidia | 26 | 2026-05-26 | ✅ OK |
| 2 | groq | 8 | never | 🔍 Stale |
**Note:** Providers will be audited sequentially (one at a time) to reduce system load.
Ask: "Select providers to audit:\n- all: all providers\n- stale: only stale (never or >30 days)\n- specific: comma-separated keys (e.g., groq,nvidia,openrouter)\nYour choice:"
Phase 2: Spawn Researchers (Sequential, JSON)
Process providers one at a time to avoid overwhelming the system. Each researcher gets the provider's current model list + outputs a structured JSON diff.
For each selected provider in sequence:
Spawn a researcher subagent
Wait for completion
Display a brief summary
Proceed to next provider
This prevents resource exhaustion and makes debugging easier.
Researcher Prompt Template
You are auditing the {PROVIDER_NAME} provider for the free-coding-models npm package.
## Current models in sources.js for {PROVIDER_NAME}:
{PASTE_MODEL_ARRAY_FROM_SOURCES_JS}
## Your task:
1. Search the web for the CURRENT {PROVIDER_NAME} model catalog — official docs, API pages, recent announcements.
2. For EACH model listed above, check if it still exists and what the current config is.
3. Identify any NEW models available that are NOT in the list.
4. Identify any config changes (context window, model ID, score).
## Output format:
Return a JSON object ONLY. No markdown, no explanation outside the JSON.
{
"provider": "{PROVIDER_KEY}",
"verificationDate": "{YYYY-MM-DD}",
"source": "{OFFICIAL_DOCS_URL}",
"removed": [
{"modelId": "...", "label": "...", "reason": "...", "replacement": "..."}
],
"added": [
{"modelId": "...", "label": "...", "tier": "...", "ctx": "...", "sweScore": "-", "url": "..."}
],
"fixed": [
{"modelId": "...", "field": "ctx|sweScore|id", "oldValue": "...", "newValue": "..."}
],
"confirmed": ["modelId1", "modelId2", ...],
"summary": {"removed": N, "added": N, "fixed": N, "confirmed": N},
"keyFinding": "One sentence summary of most important change"
}
Only include models in removed/added/fixed/confirmed — do not list every model.
Fingerprint logic: MD5 hash of confirmed.sort().join(',') + added.map(a=>a.modelId).join(','). If fingerprint unchanged since last audit, skip the provider (already up-to-date).
Special Cases
OpenCode Zen models are ephemeral
Zen free tier promotions come and go fast. Don't over-engineer tracking — just audit when user asks. The fingerprint will catch real changes.
Deprecated but not removed
If a model is marked deprecated but still functional (e.g., gemini-2.5-pro with Oct 2026 shutdown), mark it in sources.js with a comment: