| name | llmbench |
| description | Full LLM benchmark pipeline — runs same prompt across 4 models in parallel via OpenCode, then renders a 2x2 grid comparison video using Remotion. |
| arguments | The creative prompt to send to all models. |
| allowed-tools | Bash, Read, Write |
LLM Benchmark Pipeline
Full pipeline: parallel model testing + Remotion 2x2 grid video comparison.
Default Models
openrouter/z-ai/glm-5 -> llmtest/game-glm5.htmlopenrouter/minimax/minimax-m2.5 -> llmtest/game-minimax25.htmlopenrouter/google/gemini-3-pro-preview -> llmtest/game-gemini3pro.htmlopenrouter/anthropic/claude-opus-4.6 -> llmtest/game-opus46.html
Pipeline Stages
Stage 1: Parallel Model Execution
- Source environment:
set -a && source .env 2>/dev/null; set +a
- Clean previous outputs in
llmtest/
- Run all 4 models in parallel as background processes:
opencode run -m openrouter/z-ai/glm-5 "$PROMPT Save the complete file as llmtest/game-glm5.html" > /tmp/opencode-glm5.log 2>&1 &
PID1=$!
opencode run -m openrouter/minimax/minimax-m2.5 "$PROMPT Save the complete file as llmtest/game-minimax25.html" > /tmp/opencode-minimax25.log 2>&1 &
PID2=$!
opencode run -m openrouter/google/gemini-3-pro-preview "$PROMPT Save the complete file as llmtest/game-gemini3pro.html" > /tmp/opencode-gemini3pro.log 2>&1 &
PID3=$!
opencode run -m openrouter/anthropic/claude-opus-4.6 "$PROMPT Save the complete file as llmtest/game-opus46.html" > /tmp/opencode-opus46.log 2>&1 &
PID4=$!
wait $PID1 $PID2 $PID3 $PID4
- Verify all 4 HTML files exist and are >1KB
Stage 2: Copy HTML to Remotion
cp llmtest/game-*.html remotion/public/
Stage 3: Render Video
cd remotion && npx remotion render index.ts GridComparison ../out/comparison.mp4 --concurrency=1 --timeout=90000 --public-dir=public
--concurrency=1 is required because HTML files use requestAnimationFrame canvas animations- Remotion can't sync with those, so sequential single-tab rendering lets animations advance naturally
Stage 4: Verify Output
- Check
out/comparison.mp4 exists
- Report file size
- Report success/failure
Important Notes
- Remotion must be installed first:
cd remotion && npm install
- The HTML files must use only vanilla HTML/CSS/JS (no external dependencies)
- Video output: 1920x1080, 30fps, 30 seconds (900 frames)
- Each quadrant shows one model's output with a label bar at the bottom
