| name | vastai-reference-architecture |
| description | Implement Vast.ai reference architecture for GPU compute workflows.
Use when designing ML training pipelines, structuring GPU orchestration,
or establishing architecture patterns for Vast.ai applications.
Trigger with phrases like "vastai architecture", "vastai design pattern",
"vastai project structure", "vastai ml pipeline".
|
| allowed-tools | Read, Grep |
| version | 1.11.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
| tags | ["saas","vast-ai","architecture"] |
| compatibility | Designed for Claude Code, also compatible with Codex and OpenClaw |
Vast.ai Reference Architecture
Overview
Production architecture for GPU compute workflows on Vast.ai. Covers the three-tier pattern (orchestrator, GPU workers, artifact storage), job queue design, and fault-tolerant training pipelines.
Prerequisites
- Vast.ai account with CLI
- Cloud storage (S3, GCS, or MinIO) for artifacts
- Understanding of ML training pipelines
Instructions
Architecture: Three-Tier GPU Compute
┌─────────────────────────────────────────────────┐
│ ORCHESTRATOR (your server / CI / cloud function) │
│ - Job queue management │
│ - Instance provisioning via Vast.ai API │
│ - Status monitoring and auto-recovery │
│ - Cost tracking and budget enforcement │
└───────────────┬─────────────────────────────────┘
│ Vast.ai REST API
┌───────────────▼─────────────────────────────────┐
│ GPU WORKERS (Vast.ai rented instances) │
│ - Training / inference execution │
│ - Checkpoint saving to cloud storage │
│ - Health reporting back to orchestrator │
│ - Graceful shutdown on SIGTERM (spot preemption)│
└───────────────┬─────────────────────────────────┘
│ S3 / GCS / MinIO
┌───────────────▼─────────────────────────────────┐
│ ARTIFACT STORAGE (persistent) │
│ - Model checkpoints │
│ - Training logs and metrics │
│ - Dataset cache │
│ - Final model artifacts │
└─────────────────────────────────────────────────┘
Project Structure
ml-pipeline/
orchestrator/
job_queue.py # Job definition and scheduling
provisioner.py # Vast.ai instance lifecycle
monitor.py # Status polling and auto-recovery
cost_tracker.py # Budget enforcement
worker/
Dockerfile # GPU worker image
train.py # Training entry point
checkpoint.py # Cloud storage checkpoint manager
health.py # Report status back to orchestrator
config/
gpu_profiles.yaml # GPU selection criteria per job type
budgets.yaml # Cost limits per team/project
scripts/
deploy.py # CLI for launching jobs
cost_report.py # Spending analysis
GPU Profile Configuration
profiles:
dev-test:
gpu_name: RTX_4090
num_gpus: 1
max_dph:
