geobench-run-grok

name	geobench-run-grok
description	Run GeoBench with the Grok CLI. Use when the user asks to run Grok, grok-4.6, grok-4.5, or /geobench-run-grok. Shared reset, isolation, grade, and import live in the geobench-run skill.

GeoBench via Grok CLI

Read geobench-run first. --harness is grok.

Invoke

Runs in the geobench-agent container. Headless Grok ignores piped stdin; the instruction is --prompt-file, and the ro mount puts it at /instruction.md.

export MSYS_NO_PATHCONV=1
docker run --rm --security-opt seccomp=unconfined \
  -v geobench-agent-home:/home/agent \
  -v "$(cygpath -w "$root/tasks/$task/workspace"):/work" \
  -v "$(cygpath -w "$root/tasks/$task/instruction.md"):/instruction.md:ro" \
  -w /work geobench-agent \
  grok --prompt-file /instruction.md --cwd /work --model "<grok-model-id>" \
    --always-approve --output-format streaming-json \
    --no-memory --no-plan --disable-web-search --verbatim

Models

Ids that have completed a suite on this host: grok-4.6, grok-4.5.

The CLI may print a runtime id such as grok-4.6-build when the selected model is grok-4.6. Grade with the selected id, not the runtime alias.

Pitfalls

• $prompt | grok … → empty run, no workspace edits.
• Putting the prompt on argv instead of --prompt-file → fragile quoting; do not.
• Container login missing → grok login --device-auth in the login container (docker/README.md).

On Windows Docker Desktop, use the persistent geobench-agent-home volume and one disposable --rm task container at a time. The shared runner adds --security-opt seccomp=unconfined for the bundled container tools. If a rate limit, token budget, or outer session stop interrupts a combo, keep its logs and timing, clean only the exact stale task container, regenerate tasks, and rerun the same model later. Do not advance from a partial workspace. A detected provider quota/session response is recorded as rate_limited and aborts promptly.