メニュー
Build and run DEC Bench scenarios, inspect results, and open the audit UI. Use when a user says "run scenario", "run eval", "benchmark", "run foo-bar-*", "run matrix", "compare agents", or asks about specific scenario results.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | dec-bench-run |
| description | Build and run DEC Bench scenarios, inspect results, and open the audit UI. Use when a user says "run scenario", "run eval", "benchmark", "run foo-bar-*", "run matrix", "compare agents", or asks about specific scenario results. |
Build and run existing DEC Bench scenarios. This skill covers running scenarios, not creating them — for that, use dec-bench-create-scenario.
Confirm the user has:
dec-bench list to verify)If any of these are missing, point them to the dec-bench-quickstart skill.
dec-bench list
This shows all scenarios with their tier, domain, and description.
Agents and models:
claude-code: claude-sonnet-4-20250514, claude-sonnet-4-6, claude-opus-4-6codex: gpt-5.4cursor: composer-2Personas: baseline (plain language, no hints) or informed (specific tools, schemas, constraints)
Harnesses: base-rt, classic-de, olap-for-swe
dec-bench build --scenario <scenario-id>
dec-bench run --scenario <scenario-id>
The CLI validates the agent, model, and API key before building. If validation fails, it tells the user what to fix.
dec-bench build --scenario <scenario-id> --agent codex --model gpt-5.4
dec-bench run --scenario <scenario-id> --agent codex --model gpt-5.4
Defaults: --agent claude-code --model claude-sonnet-4-20250514
dec-bench build --scenario <scenario-id> --harness classic-de
dec-bench run --scenario <scenario-id> --harness classic-de
Default: --harness base-rt
dec-bench results --latest --scenario <scenario-id>
dec-bench results --run-id <run-id>
This prints gate scores, normalized score, and paths to all artifacts (stdout, trace, assertion log, session JSONL).
dec-bench audit open --scenario <scenario-id> --run-id <run-id>
Opens a local web UI with agent traces, assertion results, and run metadata.
Run all agent/harness/persona combinations:
dec-bench run --matrix --parallel auto
This validates all agent/model/key combos upfront before any Docker work.
For --parallel > 1, supply multiple provider keys via the plural env vars
so each container gets its own key:
export ANTHROPIC_API_KEYS=sk-ant-key1,sk-ant-key2
dec-bench run --matrix --parallel 2
With only the singular *_API_KEY set, the CLI warns and proceeds — expect
rate-limit errors from the provider.
Heads up: Anthropic (and most major providers) apply rate limits at the
organization level, not the key level. Two keys from the same org share
one rate-limit pool, so multi-key rotation only adds real capacity when the
keys come from different organizations. Within one org, the only ways to
get more capacity are a higher usage tier or lower --parallel.
--agent codex --model claude-sonnet-4-6)--parallel, raise your usage tier, or use keys from different organizationsCapture a DEC Bench (AXP) comparison run set as a Linear snapshot — required GCS trace upload, parent index issue (with eval-validity / cheating / decbench-DX / Moose-feedback findings), Document, and labeled follow-up sub-issues. Halts if `gcloud` is not authenticated and asks the user to log in. Use when a user says "snapshot the results", "publish this comparison to Linear", "create a snapshot from these runs", "document the latest matrix", or after running a multi-cell harness comparison and asking "now what".
Create a new DEC Bench evaluation scenario from scratch. Use when a user says "create scenario", "new scenario", "add eval", "write a benchmark for", "create a performance test", or describes a data engineering task they want to benchmark.
Substitute a locally-built artifact — a binary (moose-cli, moose-lib, clickhouse, 514 CLI), a directory (Claude skills, template-packages), or a JSON fragment (MCP servers) — into a DEC Bench scenario image so changes can be tested end-to-end without cutting a release. Use when a developer says "test my local moose build", "try my patched moose-lib", "iterate on a skill inside dec-bench", "test a clickhouse patch against scenarios", "run moose-user scenarios against a patched binary", or "override <X> in a scenario".
Validate your DEC Bench setup and run your first scenario. Use when a user says "get started", "first run", "test my setup", or "run a quick benchmark".
Build and run DEC Bench scenarios locally using the Cargo-built CLI and Docker. Use when testing scenario changes, assertion fixes, entrypoint modifications, or harness configurations before pushing to CI.
Apply Eugene Schwartz's "Breakthrough Advertising" framework to write high-converting digital copy. Use for landing pages, Facebook/Google ads, email sequences, sales pages, and conversion funnels. Use when writing persuasive copy, conversion copywriting, sales copy, ad copy, landing page copy, or when optimizing copy for specific audience awareness levels.