implement-throttling

name	implement-throttling
description	Implement API throttling and quotas
shortcut	thro

Implement API Throttling

Implement sophisticated API throttling with dynamic rate limits, quota management, tiered pricing, and advanced traffic control strategies to ensure fair usage and optimal performance.

When to Use This Command

Use /implement-throttling when you need to:

• Protect APIs from abuse and overload
• Implement usage-based billing and quotas
• Provide differentiated service tiers (free/premium)
• Ensure fair resource allocation among users
• Prevent cascade failures from traffic spikes
• Comply with third-party API rate limits

DON'T use this when:

• Building internal-only APIs with trusted clients (may be overkill)
• Prototype or MVP phase (premature optimization)
• Already using API gateway with throttling (avoid duplication)

Design Decisions

This command implements Token Bucket + Sliding Window as the primary approach because:

• Allows burst traffic while maintaining overall limits
• Provides smooth rate limiting without hard cutoffs
• Memory-efficient for high-traffic scenarios
• Supports dynamic rate adjustment
• Works well with distributed systems
• Industry-proven algorithm combination

Alternative considered: Fixed Window

• Simpler implementation
• Susceptible to thundering herd at window boundaries
• Less smooth traffic distribution
• Recommended for simple use cases

Alternative considered: Leaky Bucket

• Constant output rate
• Better for streaming scenarios
• Less flexible for burst traffic
• Recommended for bandwidth limiting

Prerequisites

Before running this command:

1. Define rate limit tiers and quotas
2. Choose storage backend (Redis recommended)
3. Determine billing/pricing model if applicable
4. Plan graceful degradation strategy
5. Set up monitoring and alerting

Implementation Process

Step 1: Configure Rate Limit Storage

Set up Redis or similar for distributed rate limit tracking.

Step 2: Implement Throttling Algorithms

Deploy token bucket and sliding window algorithms with configurable parameters.

Step 3: Create Middleware

Build middleware for automatic rate limit enforcement.

Step 4: Add Usage Tracking

Implement detailed usage tracking for analytics and billing.

Step 5: Set Up Management API

Create API for managing rate limits, quotas, and user tiers.

Output Format

The command generates:

• middleware/rate-limiter.js - Core throttling middleware
• services/throttling-manager.js - Rate limit management service
• models/usage-tracking.js - Usage data models
• config/rate-limits.json - Tier configurations
• api/rate-limit-api.js - Management endpoints
• monitoring/throttling-metrics.js - Prometheus metrics

Code Examples

Example 1: Advanced Token Bucket + Sliding Window Implementation

// services/throttling-manager.js
const Redis = require('ioredis');
const crypto = require('crypto');

class ThrottlingManager {
  constructor(redisClient = new Redis()) {
    this.redis = redisClient;
    this.tiers = {
      free: {
        rateLimit: 100,        // requests per hour
        burst: 10,             // burst allowance
        dailyQuota: 1000,      // daily limit
        monthlyQuota: 10000,   // monthly limit
        priority: 1            // queue priority (lower = higher priority)
      },
      basic: {
        rateLimit: 1000,
        burst: 50,
        dailyQuota: 10000,
        monthlyQuota: 250000,
        priority: 2
      },
      premium: {
        rateLimit: 10000,
        burst: 200,
        dailyQuota: 100000,
        : ,
        : 
      },
      : {
        : -,         
        : ,
        : -,        
        : -,      
        : 
      }
    };
  }

   () {
     config = .[tier];
     (!config) {
        ();
    }

    
     (config. === -) {
       {
        : ,
        : -,
        : -,
        : 
      };
    }

    
     tokenBucket =  .(
      userId,
      config.,
      config.,
      weight
    );

     (!tokenBucket.) {
       tokenBucket;
    }

    
     quotaCheck =  .(userId, config, weight);

     quotaCheck. ? tokenBucket : quotaCheck;
  }

   () {
     now = .();
      = ; 
     key = ;

    
     luaScript = ;

     result =  ..(
      luaScript,
      ,
      key,
      limit,
      burst,
      weight,
      now,
      
    );

     {
      : result[] === ,
      : limit,
      : result[],
      :  (result[])
    };
  }

   () {
     now =  ();
     dailyKey = ;
     monthlyKey = ;

    
     (config. > ) {
       dailyUsage =  ..(dailyKey, );
       (dailyUsage + weight > config.) {
         {
          : ,
          : config.,
          : .(, config. - dailyUsage),
          : .(now),
          : 
        };
      }
    }

    
     (config. > ) {
       monthlyUsage =  ..(monthlyKey, );
       (monthlyUsage + weight > config.) {
         {
          : ,
          : config.,
          : .(, config. - monthlyUsage),
          : .(now),
          : 
        };
      }
    }

    
     pipeline = ..();
     (config. > ) {
      pipeline.(dailyKey, weight);
      pipeline.(dailyKey, ); 
    }
     (config. > ) {
      pipeline.(monthlyKey, weight);
      pipeline.(monthlyKey, ); 
    }
     pipeline.();

     {
      : ,
      : config.,
      : .(, config. - ( ..(dailyKey) || )),
      : .(now)
    };
  }

   () {
     now =  ();
     dailyKey = ;
     monthlyKey = ;
     bucketKey = ;

     [dailyUsage, monthlyUsage, bucket] =  .([
      ..(dailyKey),
      ..(monthlyKey),
      ..(bucketKey)
    ]);

     {
      : {
        : (dailyUsage) || ,
        : .(now)
      },
      : {
        : (monthlyUsage) || ,
        : .(now)
      },
      : {
        : (bucket.) || ,
        : bucket. ?  ((bucket.)) : 
      }
    };
  }

   () {
     keys = [];

     (type ===  || type === ) {
      keys.();
    }
     (type ===  || type === ) {
      keys.();
    }
     (type ===  || type === ) {
      keys.();
    }

     (keys. > ) {
       ..(...keys);
    }
  }

  () {
     ;
  }

  () {
     ;
  }

  () {
     tomorrow =  (date);
    tomorrow.(tomorrow.() + );
    tomorrow.(, , , );
     tomorrow;
  }

  () {
     nextMonth =  (date);
    nextMonth.(nextMonth.() + );
    nextMonth.();
    nextMonth.(, , , );
     nextMonth;
  }
}


  = ();

 () {
   throttling =  (options.);
   {
    keyGenerator =  req.?. || req.,
    tierResolver =  req.?. || ,
    weightResolver =  ,
    skipRoutes = [],
    onLimitExceeded = 
  } = options;

     () {
    
     (skipRoutes.(req.)) {
       ();
    }

     userId = (req);
     tier =  (req);
     weight = (req);

     {
       result =  throttling.(userId, tier, weight);

      
      res.({
        : result.,
        : result.,
        : result. ? result..() : 
      });

       (!result.) {
        
         (onLimitExceeded) {
           (req, res, result);
        }

        
         res.().({
          : ,
          : result. || ,
          : result. ? .((result. - .()) / ) : 
        });
      }

      
      req. = result;
      ();
    }  (error) {
      .(, error);
      
      ();
    }
  };
}


 express = ();
 app = ();

app.(({
  :  ({
    : ,
    : 
  }),
  :  {
    
     req.[] || req.;
  },
  :  (req) => {
    
     (req.[]) {
       user =  (req.[]);
       user?. || ;
    }
     ;
  },
  :  {
    
     weights = {
      : ,
      : ,
      : ,
      : 
    };
     weights[req.] || ;
  },
  :  {
    
    .();
    res.().({
      : ,
      : ,
      : result.
    });
  }
}));

Example 2: Distributed Rate Limiting with Priority Queues

// services/priority-queue-throttler.js
const Bull = require('bull');
const Redis = require('ioredis');

class PriorityQueueThrottler {
  constructor(options = {}) {
    this.redis = options.redis || new Redis();
    this.queues = new Map();
    this.processors = new Map();
    this.config = {
      maxConcurrent: options.maxConcurrent || 100,
      processingTimeout: options.processingTimeout || 30000,
      retryAttempts: options.retryAttempts || 3
    };

    // Initialize priority queues
    this.initializeQueues();
  }

  initializeQueues() {
    const priorities = ['critical', 'high', 'normal', 'low'];

    priorities.forEach(priority => {
       queue =  (, {
        : .,
        : {
          : ,
          : ,
          : ..,
          : {
            : ,
            : 
          }
        }
      });

      ..(priority, queue);

      
      queue.(..,  (job) => {
          .(job.);
      });

      
      queue.(,  {
        .();
      });

      queue.(,  {
        .(, err);
      });
    });
  }

   () {
     queue = ..(priority);
     (!queue) {
        ();
    }

    
     pendingCount =  .(request.);
     maxPending = .(request.);

     (pendingCount >= maxPending) {
        ();
    }

    
     job =  queue.(request, {
      : .(priority),
      : .(request., pendingCount)
    });

    
     ..(
      ,
      ,
      .({
        : request.,
        priority,
        : .()
      })
    );

     {
      : job.,
      :  .(job., priority),
      :  .(priority)
    };
  }

   () {
     startTime = .();

     {
      
       result =  .(request);

      
       .({
        : request.,
        : .() - startTime,
        : 
      });

       result;
    }  (error) {
       .({
        : request.,
        : .() - startTime,
        : ,
        : error.
      });

       error;
    }
  }

   () {
     priorities = [, , , ];
     total = ;

     ( priority  priorities) {
       queue = ..(priority);
       jobs =  queue.([, ]);
      total += jobs.( job.. === userId).;
    }

     total;
  }

  () {
     limits = {
      : ,
      : ,
      : ,
      : 
    };
     limits[tier] || ;
  }

  () {
     values = {
      : ,
      : ,
      : ,
      : 
    };
     values[priority] || ;
  }

  () {
    
     baseDelay = {
      : ,
      : ,
      : ,
      : 
    };

     delay = baseDelay[tier] || ;
     delay * .(, pendingCount / );
  }

   () {
     queue = ..(priority);
     jobs =  queue.([]);
     position = jobs.( job. === jobId);
     position + ;
  }

   () {
     queue = ..(priority);
     [waiting, active] =  .([
      queue.(),
      queue.()
    ]);

     avgProcessingTime = ; 
     totalPending = waiting + active;
     estimatedMs = (totalPending * avgProcessingTime) / ..;

     .(estimatedMs / ); 
  }

   () {
     key = ;

     ..(key, metrics. ?  : , );
     ..(key, , metrics.);
     ..(key, ); 
  }

   () {
    
      ( {
      ( {
        ({
          : ,
          : request.,
          : .()
        });
      }, .() * );
    });
  }

  () {
     ;
  }

   () {
     stats = {};

     ( [priority, queue]  .) {
       [waiting, active, completed, failed] =  .([
        queue.(),
        queue.(),
        queue.(),
        queue.()
      ]);

      stats[priority] = {
        waiting,
        active,
        completed,
        failed
      };
    }

     stats;
  }
}


 express = ();
 router = express.();
 throttler =  ();

router.(,  (req, res) => {
   {
     priority = req.?. ===  ?  : ;

     result =  throttler.({
      : req.?. || req.,
      : req.?. || ,
      : req.
    }, priority);

    res.().({
      : ,
      ...result
    });
  }  (error) {
    res.().({
      : error.
    });
  }
});

router.(,  (req, res) => {
  
   job =  queue.(req..);

   (!job) {
     res.().({ :  });
  }

  res.({
    : job.,
    :  job.(),
    : job.(),
    : job.,
    : job.
  });
});

. = router;

Example 3: Adaptive Rate Limiting with Machine Learning

# adaptive_throttling.py
import time
import numpy as np
from sklearn.linear_model import LinearRegression
from collections import deque
import redis
import json
from datetime import datetime, timedelta

class AdaptiveThrottling:
    """
    Machine learning-based adaptive rate limiting that adjusts
    limits based on system performance and user behavior.
    """

    def __init__(self, redis_client=None):
        self.redis = redis_client or redis.Redis()
        self.performance_history = deque(maxlen=1000)
        self.model = LinearRegression()
        self.base_limits = {
            'free': 100,
            'basic': 500,
            'premium': 2000,
            'enterprise': 10000
        }
        self.initialize_model()

    def initialize_model(self):
        """Initialize ML model with synthetic training data."""
        # Features: [hour_of_day, day_of_week, current_load, user_history]
        X_train = np.random.rand(100, 4) * [24, 7, 1, 100]
        
        y_train =  +  * np.sin(X_train[:, ] * np.pi / ) + np.random.rand() * 
        .model.fit(X_train, y_train)

     ():
        
        base_limit = .base_limits.get(tier, )

        
        features = .extract_features(user_id)

        
        multiplier = .model.predict([features])[]
        multiplier = (, (, multiplier))  

        
        dynamic_limit = (base_limit * multiplier)

        
        .redis.setex(
            ,
            ,
            json.dumps({
                : base_limit,
                : multiplier,
                : dynamic_limit,
                : time.time()
            })
        )

         dynamic_limit

     ():
        
        now = datetime.now()

        
        hour_of_day = now.hour
        day_of_week = now.weekday()

        
        current_load = .get_system_load()

        
        user_history = .get_user_history(user_id)

         [hour_of_day, day_of_week, current_load, user_history]

     ():
        
        
        total_requests = .redis.get()
        max_capacity = 
         (, (total_requests  ) / max_capacity)

     ():
        
        history_key = 
        history = .redis.lrange(history_key, , -)

          history:
               

        
        rates = [(r)  r  history]
         np.mean(rates[-:])  

     ():
        
         (.performance_history) < :
            

        
        X = []
        y = []

         entry  .performance_history:
            X.append(entry[])
            
            y.append(entry[])

        
        .model.fit(X, y)

        ()

     ():
        
        performance_score = .calculate_performance_score(performance_metrics)

        .performance_history.append({
            : features,
            : performance_score,
            : time.time()
        })

        
         (.performance_history) %  == :
            .update_model()

     ():
        
        
        score = 
        score +=  * ( - metrics.get(, ))
        score +=  * ( - (, metrics.get(, ) / ))
        score +=  * ( - metrics.get(, ))
        score +=  * metrics.get(, ) / 

         (, (, score))


 __name__ == :
     sys

    adaptive = AdaptiveThrottling()

    
    request = json.loads(sys.stdin.read())

    
    limit = adaptive.calculate_dynamic_limit(
        request[],
        request[]
    )

    
    (json.dumps({: limit}))

Error Handling

Error	Cause	Solution
"Redis connection failed"	Redis server down	Implement fallback to local memory
"Rate limit exceeded"	Too many requests	Implement retry with backoff
"Invalid tier"	Unknown subscription tier	Use default tier as fallback
"Queue overflow"	Too many pending requests	Increase queue capacity or reject requests
"Quota calculation error"	Time sync issues	Ensure NTP synchronization

Configuration Options

Rate Limiting Algorithms

• token-bucket: Allows burst traffic
• sliding-window: Smooth rate distribution
• fixed-window: Simple time-based limits
• leaky-bucket: Constant output rate

Storage Backends

• redis: Recommended for distributed systems
• memory: For single-server deployments
• dynamodb: For serverless architectures
• postgresql: For persistent quota tracking

Best Practices

DO:

• Use distributed storage for multi-server deployments
• Implement graceful degradation when limits are reached
• Provide clear error messages with retry information
• Monitor rate limit effectiveness
• Adjust limits based on actual usage patterns
• Implement different weights for different operations

DON'T:

• Use only client-side rate limiting
• Ignore time synchronization issues
• Set limits too restrictive initially
• Forget to handle rate limiter failures
• Apply same limits to all operations

Performance Considerations

• Use Lua scripts for atomic Redis operations
• Implement connection pooling for Redis
• Cache tier information to reduce lookups
• Use sliding window for better distribution
• Consider read-heavy vs write-heavy operations

Monitoring and Analytics

// monitoring/throttling-metrics.js
const promClient = require('prom-client');

// Metrics
const rateLimitHits = new promClient.Counter({
  name: 'rate_limit_hits_total',
  help: 'Total number of rate limited requests',
  labelNames: ['tier', 'reason']
});

const quotaUsage = new promClient.Gauge({
  name: 'quota_usage_ratio',
  help: 'Current quota usage ratio',
  labelNames: ['user_id', 'quota_type']
});

const requestsQueued = new promClient.Gauge({
  name: 'requests_queued',
  help: 'Number of requests in queue',
  labelNames: ['priority']
});

Related Commands

• /api-rate-limiter - Basic rate limiting implementation
• /api-monitoring-dashboard - Monitor throttling metrics
• /api-billing-system - Usage-based billing
• /api-gateway-builder - Gateway-level throttling

Version History

• v1.0.0 (2024-10): Initial implementation with token bucket and quotas
• Planned v1.1.0: Add machine learning-based adaptive throttling