| name | Configuring Budget Limits |
| description | Imposes hard limits on maximum computational timesteps and financial cloud spend for hyperparameter sweeps and training loops. |
| version | 0.1.0 |
Configuring Budget Limits
Reinforcement Learning loops can easily consume infinite compute resources scaling over thousands of parallel CPU rollouts and GPU gradient updates. To control costs, hard analytical caps must be established inside the training architectures (Ray Tune / Optuna).
Step 1 — Calculate Max Timesteps
Determine the theoretical limit of total_timesteps expected to achieve convergence based on similar algorithm families.
- PPO: typically 1M to 10M steps for continuous control.
- DQN: typically 500k to 5M steps for discrete games.
Step 2 — Establish Cloud Budget Cap
Define explicit termination criteria based on either time bounds or total iterations:
- Time cap: E.g.,
time_total_s=3600*24 (Max 24 hours).
- Compute cap: Enforce max trial bounds in the search algorithm (
num_samples=50 in Ray Tune).
Step 3 — Early Stopping Criteria
Setup hyperparameter orchestrators with aggressive pruning (e.g., ASHA - Asynchronous Successive Halving algorithm).
- Models that are drastically underperforming the
heuristic_agent score by epoch 10 must be killed instantly to save compute.
Output Format
Budget Configuration:
Max_Timesteps_Per_Run: [Value]
Max_Trials_In_Sweep: [Value]
Early_Stopping_Threshold: [Value based on Heuristic]
