| name | vastai-core-workflow-b |
| description | Execute Vast.ai secondary workflow: multi-instance orchestration, spot recovery, and cost optimization.
Use when running distributed training, handling spot preemption,
or optimizing GPU spend across multiple instances.
Trigger with phrases like "vastai distributed training", "vastai spot recovery",
"vastai multi-gpu", "vastai cost optimization".
|
| allowed-tools | Read, Write, Edit, Bash(vastai:*), Bash(curl:*), Grep |
| version | 1.11.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
| tags | ["saas","vast-ai","workflow"] |
| compatibility | Designed for Claude Code |
Vast.ai Core Workflow B: Multi-Instance & Cost Optimization
Overview
Secondary workflow for Vast.ai: orchestrate multiple GPU instances for distributed training, implement automatic spot interruption recovery with checkpoint-based resume, and analyze spending to reduce per-job cost.
Prerequisites
- Completed
vastai-core-workflow-a
- Understanding of distributed training (PyTorch DDP, DeepSpeed)
- Checkpoint-based training pipeline
Instructions
Step 1: Multi-Instance Provisioning
import subprocess, json, time
from concurrent.futures import ThreadPoolExecutor
def provision_cluster(num_nodes, gpu_name="A100", min_vram=80, image=""):
"""Provision multiple GPU instances for distributed training."""
query = (f"num_gpus=1 gpu_name={gpu_name} gpu_ram>={min_vram} "
f"reliability>0.98 inet_down>500 rentable=true")
result = subprocess.run(
["vastai", "search", "offers", query, "--order", "dph_total",
"--raw", "--limit", str(num_nodes * 3)],
capture_output=True, text=True, check=True,
)
offers = json.loads(result.stdout)
if len(offers) < num_nodes:
raise RuntimeError(f"Only {len(offers)} offers, need {num_nodes}")
instances = []
i, offer (offers[:num_nodes]):
inst_id = provision_single(offer[], image, rank=i)
instances.append({: inst_id, : i, : offer})
inst instances:
info = wait_for_running(inst[])
inst.update({: info[], : info[]})
instances
