geobench-run-opencode

name	geobench-run-opencode
description	Run GeoBench with OpenCode against any authenticated provider/model. Use when the user asks to run OpenCode, opencode run, google/gemini-3.1-flash-lite, zai/glm-5.2, or /geobench-run-opencode. Shared reset, isolation, grade, and import live in the geobench-run skill.

GeoBench via OpenCode

Read geobench-run first. --harness is opencode.

--model is always provider/model. --variant is a separate optional effort knob (high, max, minimal, …), not part of the model id. Omit it unless the user named one.

Invoke

Runs in the geobench-agent container. run is the non-interactive command. --auto permits unattended tools; the container mounts only the task workspace, so that is the isolation.

export MSYS_NO_PATHCONV=1
prompt="$(<"$root/tasks/$task/instruction.md")"
docker run --rm --security-opt seccomp=unconfined \
  -v geobench-agent-home:/home/agent \
  -v "$(cygpath -w "$root/tasks/$task/workspace"):/work" \
  -v "$(cygpath -w "$root/tasks/$task/instruction.md"):/instruction.md:ro" \
  -w /work geobench-agent \
  opencode run --dir /work --format json --auto \
    --model "<provider/model-id>" "$prompt"

When the user named a variant, insert --variant high (or the named value) after --model and record --notes "variant high".

401 from a configured OAuth provider → opencode auth login in the login container, or pick another authenticated pair.

Models

Any authenticated provider/model pair. Catalog/runner forms seen on this host:

--model	--variant	Scope
google/gemini-3.1-flash-lite	omit	smoke, task 01
zai/glm-5.2	high	full 30-task suite
opencode/nemotron-3.5-lightning-free	omit	catalog free route
opencode/laguna-s-2.1-free	omit	catalog free route
opencode/hy3-free	omit	catalog free route
opencode/nemotron-3-ultra-free	omit	catalog free route
opencode/mimo-v2.5-free	omit	catalog free route
opencode/big-pickle	omit	catalog free route

Pitfalls

• Using the interactive opencode TUI instead of opencode run → not a benchmark attempt.
• Treating --variant as part of the model id (zai/glm-5.2-high) → wrong model.

Rate limits and resume

On this Windows Docker host, use the shared runner's --security-opt seccomp=unconfined, the retained geobench-agent-home volume, and one disposable --rm task container at a time. The launcher retries transient rate-limit/network failures with bounded backoff. If a provider rate limit or outer session stop persists, leave the current queue item pending, preserve its logs/timing, clean only its exact stale task container, reset with python scripts/generate_tasks.py, and resume that item later. Do not advance to another model using a partial workspace; a completed item requires its matching verify artifact. The shared runner records a detected provider quota response as rate_limited and aborts promptly instead of retrying every remaining task.