| name | geobench-run-grok |
| description | Run GeoBench with the Grok CLI. Use when the user asks to run Grok, grok-4.6, grok-4.5, or /geobench-run-grok. Shared reset, isolation, grade, and import live in the geobench-run skill.
|
GeoBench via Grok CLI
Read geobench-run first. --harness is grok.
Invoke
Runs in the geobench-agent container. Headless Grok ignores piped stdin; the instruction is --prompt-file, and the ro mount puts it at /instruction.md.
export MSYS_NO_PATHCONV=1
docker run --rm --security-opt seccomp=unconfined \
-v geobench-agent-home:/home/agent \
-v "$(cygpath -w "$root/tasks/$task/workspace"):/work" \
-v "$(cygpath -w "$root/tasks/$task/instruction.md"):/instruction.md:ro" \
-w /work geobench-agent \
grok --prompt-file /instruction.md --cwd /work --model "<grok-model-id>" \
--always-approve --output-format streaming-json \
--no-memory --no-plan --disable-web-search --verbatim
Models
Ids that have completed a suite on this host: grok-4.6, grok-4.5.
The CLI may print a runtime id such as grok-4.6-build when the selected model is grok-4.6. Grade with the selected id, not the runtime alias.
Pitfalls
$prompt | grok … → empty run, no workspace edits.- Putting the prompt on argv instead of
--prompt-file → fragile quoting; do not.
- Container login missing →
grok login --device-auth in the login container (docker/README.md).
On Windows Docker Desktop, use the persistent geobench-agent-home volume and
one disposable --rm task container at a time. The shared runner adds
--security-opt seccomp=unconfined for the bundled container tools. If a rate
limit, token budget, or outer session stop interrupts a combo, keep its logs and
timing, clean only the exact stale task container, regenerate tasks, and rerun
the same model later. Do not advance from a partial workspace. A detected
provider quota/session response is recorded as rate_limited and aborts
promptly.
