geobench-run-claude

name	geobench-run-claude
description	Run GeoBench with Claude Code. Use when the user asks to run Claude, claude-code, opus, sonnet, or /geobench-run-claude. Shared reset, isolation, grade, and import live in the geobench-run skill.

GeoBench via Claude Code

Read geobench-run first. --harness is claude-code.

Invoke

Runs in the geobench-agent container; -w /work is the working directory. Put the prompt after -- so instruction text such as -File is not parsed as a Claude flag.

export MSYS_NO_PATHCONV=1
prompt="$(<"$root/tasks/$task/instruction.md")"
docker run --rm --security-opt seccomp=unconfined \
  -v geobench-agent-home:/home/agent \
  -v "$(cygpath -w "$root/tasks/$task/workspace"):/work" \
  -v "$(cygpath -w "$root/tasks/$task/instruction.md"):/instruction.md:ro" \
  -w /work geobench-agent \
  claude -p --model "<claude-model-id>" --permission-mode acceptEdits \
    --allowedTools "Bash,Read,Edit,Write,Glob,Grep" \
    --no-session-persistence --verbose --output-format stream-json -- "$prompt"

Host-native fallback (Docker unavailable — say so in the run notes): same flags, cd into the task workspace first.

On Windows Docker Desktop, retain the shared runner's --security-opt seccomp=unconfined, use exactly one disposable --rm task container at a time, and never remove geobench-agent-home. If a rate limit, token budget, or outer session stop interrupts a combo, preserve its run directory, clean only its named stale container, regenerate tasks, and resume the same bare model id later. Do not advance from a partial workspace; require the matching finished verify artifact.

Models

Installed/accepted ids seen on this host include claude-opus-4-8, claude-sonnet-5, and claude-opus-5. Use the bare ids claude-opus-5 and claude-sonnet-5 for the requested high queue entries; record effort high in the verify notes when that queue knob is selected.

A settings default such as us.anthropic.claude-opus-5 can be invalid. Use a bare id the installed CLI accepts.

Pitfalls

• Omitting -- before "$prompt" → Claude treats -File (and similar) as options.
• With --output-format stream-json in print mode, Claude CLI 2.1.233 also requires --verbose; otherwise every task exits before making an agent request.
• A Claude JSONL rate_limit_event with out_of_credits / HTTP 429 is a provider checkpoint, not a task failure. The shared runner stops the suite, preserves the log, and resumes the same model after the reset window.
• Container login missing → first run claude interactively in the login container (docker/README.md).

Bedrock

When Claude Code is authenticated through AWS Bedrock, configure the AWS profile in the retained geobench-agent-home volume, verify it with aws sts get-caller-identity --profile <profile>, then export these selectors before running the shared suite:

export CLAUDE_CODE_USE_BEDROCK=1
export AWS_PROFILE=Claude_on_Bedrock-986192223246
export AWS_REGION=us-east-1

The shared runner forwards only these mode/profile/region selectors into each task container. It does not forward raw AWS credentials.