gpu-runner

Execute model inference on GPU cloud providers. Handles code generation, deployment, execution, and result collection across HF Inference API/Endpoints, Colab, Modal, beam.cloud, Vast.ai, and RunPod. Use when running models on GPU, deploying to cloud, executing notebooks, or troubleshooting GPU execution failures. Triggers on "run on GPU", "execute model", "deploy to modal", "colab notebook", "beam deploy", "HF inference", "HF endpoints", "vast", "runpod".

Jump to install

Repository: nyosegawa/agentic-bench
Last source activity: February 21, 2026 at 07:23
Detected SKILL.md language: English
Stars: 5
Forks: 0

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/nyosegawa/agentic-bench --skill gpu-runner

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

File Explorer

8 files

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

eval-reporter

Generate HTML reports and structured metrics from model evaluation results. Creates publication-ready reports with embedded outputs (images, audio, charts) and metrics.json for cross-model comparison. Use when generating reports, writing metrics, creating evaluation summaries, or formatting benchmark results. Triggers on "generate report", "write metrics", "create report", "evaluation summary", "benchmark results", "format results".

2026-02-21

model-researcher

Investigate model specifications, requirements, and evaluation strategy. Use when researching a model before benchmarking: reading HuggingFace model cards, estimating VRAM requirements, selecting GPU providers, and determining evaluation approach. Triggers on "model research", "investigate model", "model info", "VRAM estimate", "which provider", "model card".

2026-02-21

agentic-bench

Autonomous model validation and benchmarking. Investigates any ML model (LLM, image gen, TTS, time series, etc.), runs it on GPU cloud, evaluates quality and performance, and generates HTML reports. Use when user asks to verify, benchmark, evaluate, or test a model. Triggers on "verify model", "benchmark", "evaluate model", "test model", "run benchmark", "model evaluation", "モデルを検証", "ベンチマーク", "モデルを試して".

2026-02-21

View all from this repository

name

gpu-runner

description

GPU Runner

You are executing model inference on the appropriate GPU cloud provider.

Your Goal

Given a model, its requirements, and a chosen provider:

Write inference code tailored to the model
Execute it on the selected provider
Collect outputs (text, images, audio, metrics)
Handle errors and retry with alternatives if needed

Provider Selection (if not pre-selected)

Check .env for available credentials, then sort by cheapest hourly cost:

HF Inference API — Free with HF Pro. Requires HF_TOKEN. Catalog models only.
HF Inference Endpoints — Any HF model on dedicated GPU. HF_TOKEN only. $0.50–2.50/hr.
Colab Pro — Chrome MCP. No token needed. $9.99/month subscription. Up to ~30B.
Modal — Requires MODAL_TOKEN_ID + MODAL_TOKEN_SECRET. $30/month free tier. $0.59–3.95/hr.
beam.cloud — Requires BEAM_TOKEN. Existing credit. $0.54–3.50/hr.
Vast.ai — Requires VAST_API_KEY. Marketplace pricing (cheapest GPUs). $0.10–2.00/hr.
RunPod — Requires RUNPOD_API_KEY. Pods + Serverless. $0.34–2.69/hr.

Provider	Reference	When to Use
HF Inference API	(inline below)	Model on HF, API-supported, free
HF Inference Endpoints	`references/hf-endpoints.md`	Any HF model, cheapest dedicated GPU
Colab Pro	`references/colab-chrome-mcp.md`	Up to ~30B, interactive debugging
Modal	`references/modal.md`	30B+, serverless, reliable GPUs
beam.cloud	`references/beam-cloud.md`	Dedicated endpoints, existing credit
Vast.ai	`references/vast.md`	Cheapest GPUs, marketplace pricing
RunPod	`references/runpod.md`	Pods (persistent VMs), balanced price/reliability

Error	Recovery
OOM (CUDA out of memory)	Try quantization (int8/int4), smaller batch, or bigger GPU
Colab GPU unavailable	Fall back to Modal
Modal timeout	Increase timeout, or use beam.cloud
Import error	Install missing dependency in the execution environment
Model not found	Verify model ID, check if gated (needs HF token)

gpu-runner

Install options

Review the source files

GPU Runner

Your Goal

Provider Selection (if not pre-selected)

GPU Runner

Your Goal

Provider Selection (if not pre-selected)

Provider-Specific Guides

HF Inference API (inline — simple enough)

Execution Workflow

Step 0: Dependency Research (BEFORE writing code)

Step 1: Write Inference Code

Step 2: Execute

Step 3: Collect Results

Step 4: Handle Failures

Important

gpu-runner

Source facts

Install options

Review the source files

More from this repository

More from this repository

GPU Runner

Your Goal

Provider Selection (if not pre-selected)

GPU Runner

Your Goal

Provider Selection (if not pre-selected)

Provider-Specific Guides

HF Inference API (inline — simple enough)

Execution Workflow

Step 0: Dependency Research (BEFORE writing code)

Step 1: Write Inference Code

Step 2: Execute

Step 3: Collect Results

Step 4: Handle Failures

Important