메뉴
Launch and manage CodeScaleBench benchmark runs with paired-run guardrails, quick reruns, and execution orchestration.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
Archive old completed benchmark runs to save disk space and speed up scans. Triggers on archive runs, clean up runs, disk space, old runs.
Audit benchmark suites against ABC framework (Task/Outcome/Reporting validity). Checks instruction quality, verifier correctness, reproducibility. Triggers on benchmark audit, audit benchmark, abc audit, task validity.
Verify infrastructure readiness before launching benchmark runs — tokens, Docker, disk, credentials. Triggers on check infra, infrastructure check, ready to run, pre-run check.
Compare benchmark results across agent configurations (baseline, SG_full). Show where configs diverge. Triggers on compare configs, config comparison, which config wins, MCP impact.
Token and cost analysis per run, suite, and config. Shows most expensive tasks and config cost comparison. Triggers on cost report, how much did it cost, token usage, spending.
Generate the aggregate CSB evaluation report from completed Harbor runs. Triggers on generate report, eval report, ccb report, benchmark report.
SOC 직업 분류 기준
| name | run |
| description | Launch and manage CodeScaleBench benchmark runs with paired-run guardrails, quick reruns, and execution orchestration. |
Use this skill when the user asks to:
Before executing any benchmark run, confirm with the user:
anthropic/claude-haiku-4-5-20251001 for test runs)--selection-file?--baseline-only, or --full-only? Which --full-config?staging (default) or official?Do NOT launch a run until the user has confirmed these five parameters.
./configs/harnesses/<suite>_2config.sh./configs/harnesses/run_selected_tasks.shconfigs/eval_matrix.json--rerun flag on base commandbaseline + sourcegraph_full--baseline-only requires valid existing sourcegraph_full counterpart runs--full-only requires valid existing baseline counterpart runsALLOW_UNPAIRED_SINGLE_CONFIG=truepython3 scripts/infra/account_health.py status before launching# Paired per-suite run
./configs/harnesses/pytorch_2config.sh --parallel 4
# Paired selected-task run
./configs/harnesses/run_selected_tasks.sh --benchmark csb_sdlc_pytorch
# Gap-fill baseline only (guarded)
./configs/harnesses/run_selected_tasks.sh --benchmark csb_sdlc_pytorch --baseline-only
# Quick rerun of failed tasks
./configs/harnesses/run_selected_tasks.sh --benchmark csb_sdlc_pytorch --rerun failed
docs/DAYTONA.md)--parallel Nscripts/running/control_plane.py manages multi-account schedulingscripts/running/monitor_and_queue.sh watches active runs/status — monitor active runs, check completion/audit — post-run validation and integrity checks/evaluate — extract and score results