메뉴
Inspect Model CI status for a model checkpoint, including failed Buildkite job log extraction, Eval Studio info, and Shadow Gym execution metadata. Use when a user asks to debug CI/eval failures for a model.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
End-to-end run-image classification with Gemini, including image fetch, prompt control, and JSON output. Use when you need parking or pull-over labels from a run_id.
Find a Gen2 MCAP from a run (or model-derived run), list topics, and extract topic/field values at a specific log time. Use when debugging run-level protobuf signals such as controller_state.
Fetch Buildkite job logs and extract failure lines for debugging CI jobs. Use when a user provides build/job ids or requests failure signal from Buildkite.
Use when forming Spark/SQL queries for Databricks parking evaluation - provides table schemas, PySpark patterns, joins, and filter patterns for PUDO/UNPUDO analysis
Work with Databricks notebooks using the REST API - list, read, modify, and write notebooks back to Databricks workspace
Inspect Flyte execution status and per-node/per-task log URIs from a Flyte Console execution URL. Use when a user asks for workflow status, step status, or logs for flyte.data.wayve.ai/dev executions.
| name | modelci-shadowgym-debug |
| description | Inspect Model CI status for a model checkpoint, including failed Buildkite job log extraction, Eval Studio info, and Shadow Gym execution metadata. Use when a user asks to debug CI/eval failures for a model. |
Use shell scripts in scripts/ for Model CI / eval debugging.
This skill depends on:
$model-catalogue-core for model/checkpoint resolution.$obs-buildkite-jobs for Buildkite log retrieval and failure extraction.BASE_URL="${BASE_URL:-https://model-catalogue-api.azr.internal.wayve.ai}"
export BUILDKITE_TOKEN="<token>" # required only for Buildkite failing-job logs
Required commands:
curljqcolumnOptional:
perl (ANSI cleanup for Buildkite logs)From this skill folder:
cd /home/borisindelman/.codex/skills/modelci-shadowgym-debug
./scripts/modelci_evalstudio_shadowgym.sh <model_ref> [checkpoint_num]
If checkpoint_num is omitted, latest checkpoint is used.
/v2/model/{artefact_id}/eval_studio_info when available.BUILDKITE_TOKEN is set, fetches logs to:
/tmp/modelci_<build>_<job>.json/tmp/modelci_<build>_<job>.log