メニュー
Reconcile task metadata between selected_benchmark_tasks.json and task.toml files. Finds and fixes drift. Triggers on sync metadata, check metadata, metadata mismatch, reconcile tasks.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Archive old completed benchmark runs to save disk space and speed up scans. Triggers on archive runs, clean up runs, disk space, old runs.
Audit benchmark suites against ABC framework (Task/Outcome/Reporting validity). Checks instruction quality, verifier correctness, reproducibility. Triggers on benchmark audit, audit benchmark, abc audit, task validity.
Verify infrastructure readiness before launching benchmark runs — tokens, Docker, disk, credentials. Triggers on check infra, infrastructure check, ready to run, pre-run check.
Compare benchmark results across agent configurations (baseline, SG_full). Show where configs diverge. Triggers on compare configs, config comparison, which config wins, MCP impact.
Token and cost analysis per run, suite, and config. Shows most expensive tasks and config cost comparison. Triggers on cost report, how much did it cost, token usage, spending.
Generate the aggregate CSB evaluation report from completed Harbor runs. Triggers on generate report, eval report, ccb report, benchmark report.
| name | sync-metadata |
| description | Reconcile task metadata between selected_benchmark_tasks.json and task.toml files. Finds and fixes drift. Triggers on sync metadata, check metadata, metadata mismatch, reconcile tasks. |
| user-invocable | true |
Ensure task.toml files match the authoritative selected_benchmark_tasks.json registry.
cd ~/CodeScaleBench && python3 scripts/sync_task_metadata.py
Show any fields where task.toml disagrees with selected_benchmark_tasks.json.
If mismatches found:
python3 scripts/sync_task_metadata.py --fix
This updates the task.toml files to match the selection registry.
python3 scripts/sync_task_metadata.py --suite csb_sdlc_pytorch
python3 scripts/sync_task_metadata.py --format json