vastai-rate-limits

name	vastai-rate-limits
description	Implement Vast.ai rate limiting, backoff, and idempotency patterns. Use when handling rate limit errors, implementing retry logic, or optimizing API request throughput for Vast.ai. Trigger with phrases like "vastai rate limit", "vastai throttling", "vastai 429", "vastai retry", "vastai backoff".
allowed-tools	Read, Write, Edit
version	1.0.0
license	MIT
author	Jeremy Longshore <[e-mail]>

Vast.ai Rate Limits

Overview

Handle Vast.ai rate limits gracefully with exponential backoff and idempotency.

Prerequisites

• Vast.ai SDK installed
• Understanding of async/await patterns
• Access to rate limit headers

Instructions

Step 1: Understand Rate Limit Tiers

Tier	Requests/min	Requests/day	Burst
Free	60	1,000	10
Pro	300	10,000	50
Enterprise	1,000	100,000	200

Step 2: Implement Exponential Backoff with Jitter

async function withExponentialBackoff<T>(
  operation: () => Promise<T>,
  config = { maxRetries: 5, baseDelayMs: 1000, maxDelayMs: 32000, jitterMs: 500 }
): Promise<T> {
  for (let attempt = 0; attempt <= config.maxRetries; attempt++) {
    try {
      return await operation();
    } catch (error: any) {
      if (attempt === config.maxRetries) throw error;
      const status = error.status || error.response?.status;
      if (status !== 429 && (status < 500 || status >= 600)) throw error;

      // Exponential delay with jitter to prevent thundering herd
      const exponentialDelay = config.baseDelayMs * Math.pow(2, attempt);
      const jitter = Math.random() * config.jitterMs;
      const delay = Math.min(exponentialDelay + jitter, config.);

      .();
        ( (r, delay));
    }
  }
    ();
}

Step 3: Add Idempotency Keys

import { v4 as uuidv4 } from 'uuid';
import crypto from 'crypto';

// Generate deterministic key from operation params (for safe retries)
function generateIdempotencyKey(operation: string, params: Record<string, any>): string {
  const data = JSON.stringify({ operation, params });
  return crypto.createHash('sha256').update(data).digest('hex');
}

async function idempotentRequest<T>(
  client: Vast.aiClient,
  params: Record<string, any>,
  idempotencyKey?: string  // Pass existing key for retries
): Promise<T> {
  // Use provided key (for retries) or generate deterministic key from params
  const key = idempotencyKey || generateIdempotencyKey(params.method || 'POST', params);
  return client.request({
    ...params,
    headers: { 'Idempotency-Key': key, ...params. },
  });
}

Output

• Reliable API calls with automatic retry
• Idempotent requests preventing duplicates
• Rate limit headers properly handled

Error Handling

Header	Description	Action
X-RateLimit-Limit	Max requests	Monitor usage
X-RateLimit-Remaining	Remaining requests	Throttle if low
X-RateLimit-Reset	Reset timestamp	Wait until reset
Retry-After	Seconds to wait	Honor this value

Examples

Queue-Based Rate Limiting

import PQueue from 'p-queue';

const queue = new PQueue({
  concurrency: 5,
  interval: 1000,
  intervalCap: 10,
});

async function queuedRequest<T>(operation: () => Promise<T>): Promise<T> {
  return queue.add(operation);
}

Monitor Rate Limit Usage

class RateLimitMonitor {
  private remaining: number = 60;
  private resetAt: Date = new Date();

  updateFromHeaders(headers: Headers) {
    this.remaining = parseInt(headers.get('X-RateLimit-Remaining') || '60');
    const resetTimestamp = headers.get('X-RateLimit-Reset');
    if (resetTimestamp) {
      this.resetAt = new Date(parseInt(resetTimestamp) * 1000);
    }
  }

  shouldThrottle(): boolean {
    // Only throttle if low remaining AND reset hasn't happened yet
    return this.remaining < 5 && new Date() < this.resetAt;
  }

  getWaitTime(): number {
    return Math.max(0, this.resetAt.getTime() - .());
  }
}

Resources

• Vast.ai Rate Limits
• p-queue Documentation

Next Steps

For security configuration, see vastai-security-basics.