training-workflows

# AgileRL Training Workflows Use this sub-skill for AgileRL classical single-agent RL training with Gymnasium environments: PPO on-policy, DQN/RainbowDQN/DDPG/TD3 off-policy, recurrent variants, population setup, replay or rollout buffers, and safe pre-flight checks before long runs. ## Read First - `references/workflows.md` for on-policy/off-policy training recipes and population flow. - `references/api-reference.md` for important training helper and buffer APIs. - `references/configuration.md` for `INIT_HP`, `net_config`, YAML, logging, and environment patterns. - `references/distributed-training.md` for Accelerate/distributed constraints. - `references/troubleshooting.md` for shape, dependency, vector-env, W&B, and runtime failures. - `scripts/agilerl_classic_smoke.py --help` for a no-training import/config probe. ## Route Within AgileRL | Need | Route | | --- | --- | | Evolution/tournament/mutation configuration | `../hpo-and-mutation/SKILL.md` | | Custom `net_config`, `EvolvableNetwork`, CNN/LSTM/MultiInput setup | `../evolvable-modules/SKILL.md` | | Multi-agent PettingZoo training | `../multi-agent-and-wrappers/SKILL.md` | | Offline datasets or contextual bandits | `../offline-bandits-data/SKILL.md` | | LLM fine-tuning | `../llm-fine-tuning/SKILL.md` | ## Standard Classical Training Flow 1. Install AgileRL and verify imports; use the root `scripts/check_agilerl_install.py` if unsure. 2. Create or wrap a Gymnasium environment; prefer `make_vect_envs(env_name, num_envs=...)` for vectorized classical training. 3. Define `INIT_HP` with algorithm-specific keys and `POP_SIZE`. 4. Define `net_config` with `encoder_config` and `head_config` unless the default network is sufficient. 5. Create a population with `create_population(algo=..., observation_space=..., action_space=..., net_config=..., INIT_HP=..., population_size=..., num_envs=..., device=...)`. 6. For off-policy algorithms, create `ReplayBuffer`; for on-policy PPO, use rollout collection/training helpers rather than replay memory. 7. Configure `TournamentSelection` and `Mutations` in the HPO sub-skill. 8. Call `train_on_policy(...)` or `train_off_policy(...)`, or implement a custom loop following `references/workflows.md`. ## Algorithm Selection - Use `PPO` for on-policy, stable policy-gradient workflows and recurrent PPO variants. - Use `DQN` or `RainbowDQN` for discrete-action off-policy value learning. - Use `DDPG` or `TD3` for continuous-action off-policy actor-critic workflows. - Use `train_on_policy` for PPO and `train_off_policy` for DQN/RainbowDQN/DDPG/TD3 families. ## Safe Validation Run a smoke check before training: ```bash python scripts/agilerl_classic_smoke.py --algorithm DQN --env CartPole-v1 python scripts/agilerl_classic_smoke.py --algorithm PPO --env CartPole-v1 ``` The helper checks imports, Gymnasium spaces, basic config shape, and algorithm family routing. It does not train or download anything.