菜单
Manually invoked only — run scripts/benchmark_models.py against the model-tier contract suites and interpret the champion/challenger gate. Real runs need creds and cost money.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/christopherqueenconsulting/linkedin_engagement_manager --skill model-benchmark命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Use when triaging a log_warning, a PostHog $exception, or an auto-filed error-tracking GitHub issue (body contains "posthog-issue-") — decide expected no-op vs real defect and apply the escalation contract.
Use when picking up a GitHub issue through to a merged PR — branch naming, flow labels, phased-issue close rules, PR conventions, CI gates, and the release:now fast-lane policy.
Use when writing or modifying tests — choose the unit or integration lane, the right markers and mock fixtures, respect the hermetic autouse guards, and hit 80% patch coverage.
基于 SOC 职业分类
正在显示 SKILL.md
| name | model-benchmark |
| description | Manually invoked only — run scripts/benchmark_models.py against the model-tier contract suites and interpret the champion/challenger gate. Real runs need creds and cost money. |
| disable-model-invocation | true |
poetry run python scripts/benchmark_models.py --print-suites # validate suites, no network
poetry run python scripts/benchmark_models.py --dry-run --models <m> --out-dir /tmp/bm # offline proof vs canned outputs
poetry run python scripts/benchmark_models.py --run --models <m> --results-out /tmp/bm.json # real run
poetry run python scripts/benchmark_models.py --render /tmp/bm.json --out-dir docs/model-benchmarks
Real runs require: BENCHMARK_ENABLED=true (else --run silently no-ops), OLLAMA_CLOUD_URL=https://ollama.com/v1 + OLLAMA_CLOUD_API_KEY, and BENCHMARK_USAGE_LEVELS — unsupplied levels grade unknown, and unknown holds a swap. Optional PostHog keys switch the judge to PostHog Evaluations.
Gate semantics: always benchmark candidates beside the current champion; contract checks are the absolute floor (production consumes the failure), repairable are advisory (a regeneration gate retries them); one run is not a verdict. A run where every case of every model errored — champion included — is a harness outage: refuse to publish (exit 1), never render a scorecard of zeros. Partial failures publish with an Unmeasured cases count.
Only recommend verdicts become swaps, and recommendations are RENDERED into the report, never written to config — .litellm/model_upgrades.yaml is the retirement map, owned by the weekly cron (scripts/weekly_model_check.sh). Spend-policy holds belong to the owner, not a config PR.
Authoritative: docs/model-benchmarks/README.md (methodology, leaderboard, unattended-run env table).