gym

# gym Skill > *Unified catalog of Gymnasium/OpenAI Gym environments for RL across all domains* ## Environment Taxonomy ``` ┌─────────────────────┐ │ GYMNASIUM │ │ (OpenAI Gym API) │ └──────────┬──────────┘ │ ┌───────────────┬───────────────┼───────────────┬───────────────┐ │ │ │ │ │ ┌────▼────┐ ┌─────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐ │ PHYSICS │ │ ROBOTICS │ │ ENERGY │ │ CHEMISTRY │ │ GAMES │ └────┬────┘ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ │ │ │ │ │ MuJoCo Isaac Gym Microgrid ChemistryGym Atari PyBullet RoboGym PowerGrid rlmolecule NetHack dm_control Softrobot GEM (Motor) SynthesisNet Doom ``` ## Core Environments ### Physics Simulation | Environment | Stars | Domain | Backend | |-------------|-------|--------|---------| | [gymnasium](https://github.com/Farama-Foundation/Gymnasium) | 7k+ | Classic control, Box2D, MuJoCo | Native | | [dm_control](https://github.com/google-deepmind/dm_control) | 3.5k | Continuous control | MuJoCo | | [pybullet-gym](https://github.com/benelot/pybullet-gym) | 900+ | MuJoCo alternatives | PyBullet | | [mujoco-py](https://github.com/openai/mujoco-py) | 2.8k | Physics simulation | MuJoCo | ```python import gymnasium as gym # Classic control env = gym.make("CartPole-v1") env = gym.make("Pendulum-v1") env = gym.make("Acrobot-v1") # MuJoCo env = gym.make("Humanoid-v4") env = gym.make("Ant-v4") env = gym.make("HalfCheetah-v4") ``` ### Robotics | Environment | Stars | Domain | Features | |-------------|-------|--------|----------| | [OmniIsaacGymEnvs](https://github.com/isaac-sim/OmniIsaacGymEnvs) | 1k+ | GPU-accelerated robotics | NVIDIA Isaac Sim | | [robogym](https://github.com/openai/robogym) | 400+ | Dexterous manipulation | OpenAI | | [gym-softrobot](https://github.com/skim0119/gym-softrobot) | 100+ | Soft robotics | Elastica | | [safe-control-gym](https://github.com/utiasDSL/safe-control-gym) | 500+ | Safe RL benchmarks | PyBullet | ```python # Isaac Gym (GPU parallel) from omni.isaac.gym.vec_env import VecEnvBase env = VecEnvBase(headless=True, num_envs=4096) # Safe control import safe_control_gym env = gym.make("CartPole-v0", ctrl_freq=50) ``` ### Energy & Power Systems | Environment | Stars | Domain | Features | |-------------|-------|--------|----------| | [openmodelica-microgrid-gym](https://github.com/upb-lea/openmodelica-microgrid-gym) | 214 | Microgrids | FMU, SafeOpt | | [gym-electric-motor](https://github.com/upb-lea/gym-electric-motor) | 200+ | Electric motors | GEM | | [PowerGridworld](https://github.com/NREL/PowerGridworld) | 100+ | Multi-agent grid | NREL | | [RL-Energy](https://github.com/pnnl/RL-Energy) | 50+ | Energy systems | PNNL | ```python # Microgrid (FMU-based) env = gym.make('openmodelica_microgrid_gym:ModelicaEnv-v1', net='net/net.yaml', model_path='omg_grid/grid.network.fmu') # Electric motor import gym_electric_motor as gem env = gem.make('Finite-SC-PermExcDC-v1') # Power grid world from gridworld import GridWorld env = GridWorld(num_agents=3) ``` ### Chemistry & Molecular | Environment | Stars | Domain | Features | |-------------|-------|--------|----------| | [chemistrygym](https://github.com/CLEANit/chemistrygym) | 100+ | Lab reactions | Reaction vessels | | [rlmolecule](https://github.com/NREL/rlmolecule) | 80+ | Molecule optimization | MCTS | | [SynthesisNet](https://github.com/shiningsunnyday/SynthesisNet) | New | Synthesizable molecules | ICLR 2025 | | [DistillationTrain-Gym](https://github.com/lollcat/DistillationTrain-Gym) | 50+ | Chemical engineering | Process synthesis | | [SynGameZero](https://github.com/grimmlab/SynGameZero) | 30+ | Flowsheet synthesis | AlphaZero | ```python # Chemistry Gym from chemgym import ReactionEnv env = ReactionEnv(vessels=2, max_steps=100) # Molecule RL from rlmolecule import MoleculeEnv env = MoleculeEnv(target_property='logP') # Distillation from distillation_gym import DistillationEnv env = DistillationEnv(num_components=3) ``` ### Games & Simulation | Environment | Stars | Domain | Features | |-------------|-------|--------|----------| | [ALE (Atari)](https://github.com/Farama-Foundation/Arcade-Learning-Environment) | 2k+ | Atari games | 57 games | | [NetHack](https://github.com/facebookresearch/nle) | 900+ | Roguelike | NLE | | [VizDoom](https://github.com/Farama-Foundation/ViZDoom) | 1.7k | First-person shooter | Doom | | [MiniGrid](https://github.com/Farama-Foundation/Minigrid) | 2k+ | Grid worlds | Procedural | | [PufferLib](https://github.com/PufferAI/PufferLib) | 500+ | Multi-game | High throughput | ```python # Atari env = gym.make("ALE/Breakout-v5") # NetHack import nle env = gym.make("NetHackScore-v0") # PufferLib (vectorized) import pufferlib env = pufferlib.make("atari_breakout") ``` ## Gymnasium API (Modern Standard) ```python import gymnasium as gym from gymnasium import spaces class CustomEnv(gym.Env): """Template for custom environment.""" metadata = {"render_modes": ["human", "rgb_array"]} def __init__(self, render_mode=None): super().__init__() self.observation_space = spaces.Box(low=-1, high=1, shape=(4,)) self.action_space = spaces.Discrete(2) self.render_mode = render_mode def reset(self, seed=None, options=None): super().reset(seed=seed) observation = self.observation_space.sample() info = {} return observation, info def step(self, action): observation = self.observation_space.sample() reward = 1.0 terminated = False truncated = False info = {} return observation, reward, terminated, truncated, info def render(self): if self.render_mode == "rgb_array": return self._render_frame() def close(self): pass ``` ## Vectorized Environments ```python # Gymnasium native envs = gym.vector.make("CartPole-v1", num_envs=8) # Stable-Baselines3 from stable_baselines3.common.vec_env import SubprocVecEnv envs = SubprocVecEnv([make_env(i) for i in range(8)]) # PufferLib (high-performance) import pufferlib.vectorization envs = pufferlib.vectorization.make( "CartPole-v1", num_envs=1024, backend="multiprocessing" ) ``` ## Wrappers ```python from gymnasium.wrappers import ( TimeLimit, # Max steps RecordVideo, # Video recording NormalizeObservation,# Normalize obs NormalizeReward, # Normalize rewards ClipAction, # Clip actions FrameStack, # Stack frames GrayscaleObservation,# Convert to grayscale ) env = gym.make("CartPole-v1") env = TimeLimit(env, max_episode_steps=500) env = NormalizeObservation(env) ``` ## Skill Tension Resolution via Gyms Each gym domain resolves specific skill tensions: | Gym Domain | Tensions Resolved | Bridge Skills | |------------|-------------------|---------------| | **Physics** | continuous ↔ discrete | `persistent-homology`, `acsets` | | **Robotics** | local ↔ global | `sheaf-laplacian`, `forward-forward` | | **Energy** | temporal ↔ atemporal | `unworld`, `temporal-coalgebra` | | **Chemistry** | symbolic ↔ subsymbolic | `sicp`, `gflownet` | | **Games** | maximize ↔ sample | `compression-progress`, `curiosity-driven` | ## Gay.jl Integration Color-code environments by domain: ```python GYM_COLORS = { 'physics': '#63B6F0', # Stream 3 (continuous) 'robotics': '#89DF91', # Stream 3 (embodied) 'energy': '#E6F463', # Stream 2 (temporal) 'chemistry': '#5713C0', # Stream 4 (synthesis) 'games': '#CF6971', # Stream 3 (discrete) } def color_for_env(env_id: str) -> str: if 'MuJoCo' in env_id or 'Pendulum' in env_id: return GYM_COLORS['physics'] elif 'Isaac' in env_id or 'Robot' in env_id: return GYM_COLORS['robotics'] elif 'Microgrid' in env_id or 'Motor' in env_id: return GYM_COLORS['energy'] elif 'Chem' in env_id or 'Molecule' in env_id: return GYM_COLORS['chemistry'] else: return GYM_COLORS['games'] ``` ## Training Frameworks | Framework | Gyms Supported | Best For | |-----------|----------------|----------| | [Stable-Baselines3](https://github.com/DLR-RM/stable-baselines3) | All Gymnasium | Easy PPO/SAC | | [RLlib](https://docs.ray.io/en/latest/rllib/index.html) | All Gymnasium | Multi-agent, distributed | | [CleanRL](https://github.com/vwxyzjn/cleanrl) | Standard | Single-file implementations | | [PufferLib](https://github.com/PufferAI/PufferLib) | High-throughput | Games, speed | | [Sample Factory](https://github.com/alex-petrenko/sample-factory) | Doom, Atari | Asynchronous | ```python # Stable-Baselines3 from stable_baselines3 import PPO model = PPO("MlpPolicy", env, verbose=1) model.learn(total_timesteps=100000) # RLlib from ray.rllib.algorithms.ppo import PPOConfig config = PPOConfig().environment("CartPole-v1") algo = config.build() # CleanRL (single file) # python cleanrl/ppo.py --env-id CartPole-v1 ``` ## Local Environments (from codebase) Your codebase includes these custom gyms: | File | Environment | Domain | |------|-------------|--------| | `economic_market_rl.py` | `EconomicMarketEnv` | Markets | | `property_stablecoin_env.py` | `PropertyStablecoinEnv` | DeFi | | `pufferlib_stablecoin_env.py` | `StablecoinEnv` | Stablecoins | | `sims3_fast_env.py` | `FastSims3Env` | Game simulation | | `rio/GayMCP/pufferlib_env.py` | `GayColorEnv` | Color prediction | | `rio/GayMCP/compute_market_env.py` | `ComputeMarketEnv` | Compute markets | | `free_energy_reward_shaper.py` | `FreeEnergyWrapper` | Active inference | | `golden_thread_exploration.py` | `GoldenThreadWrapper` | Exploration | ## Neighbor Skills - **omg-tension-resolver**: Microgrid gym for skill tension resolution - **alife**: Artificial life environments - **gflownet**: Sampling environments for molecule design