| name | geobench-run-opencode |
| description | Run GeoBench with OpenCode against any authenticated provider/model. Use when the user asks to run OpenCode, opencode run, google/gemini-3.1-flash-lite, zai/glm-5.2, or /geobench-run-opencode. Shared reset, isolation, grade, and import live in the geobench-run skill.
|
GeoBench via OpenCode
Read geobench-run first. --harness is opencode.
--model is always provider/model. --variant is a separate optional effort knob (high, max, minimal, …), not part of the model id. Omit it unless the user named one.
Invoke
Runs in the geobench-agent container. run is the non-interactive command. --auto permits unattended tools; the container mounts only the task workspace, so that is the isolation.
export MSYS_NO_PATHCONV=1
prompt="$(<"$root/tasks/$task/instruction.md")"
docker run --rm --security-opt seccomp=unconfined \
-v geobench-agent-home:/home/agent \
-v "$(cygpath -w "$root/tasks/$task/workspace"):/work" \
-v "$(cygpath -w "$root/tasks/$task/instruction.md"):/instruction.md:ro" \
-w /work geobench-agent \
opencode run --dir /work --format json --auto \
--model "<provider/model-id>" "$prompt"
When the user named a variant, insert --variant high (or the named value) after --model and record --notes "variant high".
401 from a configured OAuth provider → opencode auth login in the login container, or pick another authenticated pair.
Models
Any authenticated provider/model pair. Catalog/runner forms seen on this host:
--model | --variant | Scope |
|---|
google/gemini-3.1-flash-lite | omit | smoke, task 01 |
zai/glm-5.2 | high | full 30-task suite |
opencode/nemotron-3.5-lightning-free | omit | catalog free route |
opencode/laguna-s-2.1-free | omit | catalog free route |
opencode/hy3-free | omit | catalog free route |
opencode/nemotron-3-ultra-free | omit | catalog free route |
opencode/mimo-v2.5-free | omit | catalog free route |
opencode/big-pickle | omit | catalog free route |
Pitfalls
- Using the interactive
opencode TUI instead of opencode run → not a benchmark attempt.
- Treating
--variant as part of the model id (zai/glm-5.2-high) → wrong model.
Rate limits and resume
On this Windows Docker host, use the shared runner's
--security-opt seccomp=unconfined, the retained geobench-agent-home volume,
and one disposable --rm task container at a time. The launcher retries
transient rate-limit/network failures with bounded backoff. If a provider rate
limit or outer session stop persists, leave the current queue item pending,
preserve its logs/timing, clean only its exact stale task container, reset with
python scripts/generate_tasks.py, and resume that item later. Do not advance
to another model using a partial workspace; a completed item requires its
matching verify artifact. The shared runner records a detected provider quota
response as rate_limited and aborts promptly instead of retrying every
remaining task.
