Skip to main content
pufferlib This skill should be used when working with reinforcement learning tasks including high-performance RL training, custom environment development, vectorized parallel simulation, multi-agent systems, or integration with existing RL environments (Gymnasium, PettingZoo, Atari, Procgen, etc.). Use this skill for implementing PPO training, creating PufferEnv environments, optimizing RL performance, or developing policies with CNNs/LSTMs.
跳到安装 Skills Marketplace 发现并探索由社区构建的 Agent Skills
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/davila7/claude-code-templates --skill pufferlib命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
下载 Zip 下载中... 同仓库更多 Skills Build AI agents that interact with computers like humans do - viewing screens, moving cursors, clicking buttons, and typing text. Covers Anthropic's Computer Use, OpenAI's Operator/CUA, and open-source alternatives. Critical focus on sandboxing, security, and handling the unique challenges of vision-based control. Use when: computer use, desktop automation agent, screen control AI, vision-based agent, GUI automation.
Expert in building products that wrap AI APIs (OpenAI, Anthropic, etc.) into focused tools people will pay for. Not just 'ChatGPT but different' - products that solve specific problems with AI. Covers prompt engineering for products, cost management, rate limiting, and building defensible AI businesses. Use when: AI wrapper, GPT product, AI tool, wrap AI, AI SaaS.
github-workflow-automation Automate GitHub workflows with AI assistance. Includes PR reviews, issue triage, CI/CD integration, and Git operations. Use when automating GitHub workflows, setting up PR review automation, creating GitHub Actions, or triaging issues.
name pufferlib description This skill should be used when working with reinforcement learning tasks including high-performance RL training, custom environment development, vectorized parallel simulation, multi-agent systems, or integration with existing RL environments (Gymnasium, PettingZoo, Atari, Procgen, etc.). Use this skill for implementing PPO training, creating PufferEnv environments, optimizing RL performance, or developing policies with CNNs/LSTMs.
PufferLib - High-Performance Reinforcement Learning
Overview
PufferLib is a high-performance reinforcement learning library designed for fast parallel environment simulation and training. It achieves training at millions of steps per second through optimized vectorization, native multi-agent support, and efficient PPO implementation (PuffeRL). The library provides the Ocean suite of 20+ environments and seamless integration with Gymnasium, PettingZoo, and specialized RL frameworks.
When to Use This Skill
Use this skill when:
Training RL agents with PPO on any environment (single or multi-agent)Creating custom environments using the PufferEnv APIOptimizing performance for parallel environment simulation (vectorization)Integrating existing environments from Gymnasium, PettingZoo, Atari, Procgen, etc.Developing policies with CNN, LSTM, or custom architecturesScaling RL to millions of steps per second for faster experimentationMulti-agent RL with native multi-agent environment support
Core Capabilities
1. High-Performance Training (PuffeRL)
PuffeRL is PufferLib's optimized PPO+LSTM training algorithm achieving 1M-4M steps/second.
Quick start training:
puffer train procgen-coinrun --train.device cuda --train.learning-rate 3e-4
import pufferlib
from pufferlib import PuffeRL
env = pufferlib.make('procgen-coinrun' , num_envs=256 )
trainer = PuffeRL(
env=env,
policy=my_policy,
device='cuda' ,
learning_rate=3e-4 ,
batch_size=32768
)
for iteration in range (num_iterations):
trainer.evaluate()
trainer.train()
trainer.mean_and_log()
For comprehensive training guidance , read references/training.md for:
Complete training workflow and CLI options
Hyperparameter tuning with Protein
Distributed multi-GPU/multi-node training
Logger integration (Weights & Biases, Neptune)
Checkpointing and resume training
Performance optimization tips
Curriculum learning patterns
2. Environment Development (PufferEnv) Create custom high-performance environments with the PufferEnv API.
Basic environment structure:
import numpy as np
from pufferlib import PufferEnv
class MyEnvironment (PufferEnv ):
def __init__ (self, buf=None ):
super ().__init__(buf)
self .observation_space = self .make_space((4 ,))
self .action_space = self .make_discrete(4 )
self .reset()
def reset (self ):
return np.zeros(4 , dtype=np.float32)
def step (self, action ):
obs = self ._get_observation()
reward = self ._compute_reward()
done = self ._is_done()
info = {}
return obs, reward, done, info
Use the template script: scripts/env_template.py provides complete single-agent and multi-agent environment templates with examples of:
Different observation space types (vector, image, dict)
Action space variations (discrete, continuous, multi-discrete)
Multi-agent environment structure
Testing utilities
For complete environment development , read references/environments.md for:
PufferEnv API details and in-place operation patterns
Observation and action space definitions
Multi-agent environment creation
Ocean suite (20+ pre-built environments)
Performance optimization (Python to C workflow)
Environment wrappers and best practices
Debugging and validation techniques
3. Vectorization and Performance Achieve maximum throughput with optimized parallel simulation.
import pufferlib
env = pufferlib.make('environment_name' , num_envs=256 , num_workers=8 )
Shared memory buffers for zero-copy observation passing
Busy-wait flags instead of pipes/queues
Surplus environments for async returns
Multiple environments per worker
For vectorization optimization , read references/vectorization.md for:
Architecture and performance characteristics
Worker and batch size configuration
Serial vs multiprocessing vs async modes
Shared memory and zero-copy patterns
Hierarchical vectorization for large scale
Multi-agent vectorization strategies
Performance profiling and troubleshooting
4. Policy Development Build policies as standard PyTorch modules with optional utilities.
import torch.nn as nn
from pufferlib.pytorch import layer_init
class Policy (nn.Module):
def __init__ (self, observation_space, action_space ):
super ().__init__()
self .encoder = nn.Sequential(
layer_init(nn.Linear(obs_dim, 256 )),
nn.ReLU(),
layer_init(nn.Linear(256 , 256 )),
nn.ReLU()
)
self .actor = layer_init(nn.Linear(256 , num_actions), std=0.01 )
self .critic = layer_init(nn.Linear(256 , 1 ), std=1.0 )
def forward (self, observations ):
features = self .encoder(observations)
return self .actor(features), self .critic(features)
For complete policy development , read references/policies.md for:
CNN policies for image observations
Recurrent policies with optimized LSTM (3x faster inference)
Multi-input policies for complex observations
Continuous action policies
Multi-agent policies (shared vs independent parameters)
Advanced architectures (attention, residual)
Observation normalization and gradient clipping
Policy debugging and testing
5. Environment Integration Seamlessly integrate environments from popular RL frameworks.
import gymnasium as gym
import pufferlib
gym_env = gym.make('CartPole-v1' )
env = pufferlib.emulate(gym_env, num_envs=256 )
env = pufferlib.make('gym-CartPole-v1' , num_envs=256 )
env = pufferlib.make('pettingzoo-knights-archers-zombies' , num_envs=128 )
Gymnasium / OpenAI Gym
PettingZoo (parallel and AEC)
Atari (ALE)
Procgen
NetHack / MiniHack
Minigrid
Neural MMO
Crafter
GPUDrive
MicroRTS
Griddly
And more...
For integration details , read references/integration.md for:
Complete integration examples for each framework
Custom wrappers (observation, reward, frame stacking, action repeat)
Space flattening and unflattening
Environment registration
Compatibility patterns
Performance considerations
Integration debugging
Quick Start Workflow
For Training Existing Environments
Choose environment from Ocean suite or compatible framework
Use scripts/train_template.py as starting point
Configure hyperparameters for your task
Run training with CLI or Python script
Monitor with Weights & Biases or Neptune
Refer to references/training.md for optimization
For Creating Custom Environments
Start with scripts/env_template.py
Define observation and action spaces
Implement reset() and step() methods
Test environment locally
Vectorize with pufferlib.emulate() or make()
Refer to references/environments.md for advanced patterns
Optimize with references/vectorization.md if needed
For Policy Development
Choose architecture based on observations:
Vector observations → MLP policy
Image observations → CNN policy
Sequential tasks → LSTM policy
Complex observations → Multi-input policy
Use layer_init for proper weight initialization
Follow patterns in references/policies.md
Test with environment before full training
For Performance Optimization
Profile current throughput (steps per second)
Check vectorization configuration (num_envs, num_workers)
Optimize environment code (in-place ops, numpy vectorization)
Consider C implementation for critical paths
Use references/vectorization.md for systematic optimization
Resources
scripts/ train_template.py - Complete training script template with:
Environment creation and configuration
Policy initialization
Logger integration (WandB, Neptune)
Training loop with checkpointing
Command-line argument parsing
Multi-GPU distributed training setup
env_template.py - Environment implementation templates:
Single-agent PufferEnv example (grid world)
Multi-agent PufferEnv example (cooperative navigation)
Multiple observation/action space patterns
Testing utilities
references/ training.md - Comprehensive training guide:
Training workflow and CLI options
Hyperparameter configuration
Distributed training (multi-GPU, multi-node)
Monitoring and logging
Checkpointing
Protein hyperparameter tuning
Performance optimization
Common training patterns
Troubleshooting
environments.md - Environment development guide:
PufferEnv API and characteristics
Observation and action spaces
Multi-agent environments
Ocean suite environments
Custom environment development workflow
Python to C optimization path
Third-party environment integration
Wrappers and best practices
Debugging
vectorization.md - Vectorization optimization:
Architecture and key optimizations
Vectorization modes (serial, multiprocessing, async)
Worker and batch configuration
Shared memory and zero-copy patterns
Advanced vectorization (hierarchical, custom)
Multi-agent vectorization
Performance monitoring and profiling
Troubleshooting and best practices
policies.md - Policy architecture guide:
Basic policy structure
CNN policies for images
LSTM policies with optimization
Multi-input policies
Continuous action policies
Multi-agent policies
Advanced architectures (attention, residual)
Observation processing and unflattening
Initialization and normalization
Debugging and testing
integration.md - Framework integration guide:
Gymnasium integration
PettingZoo integration (parallel and AEC)
Third-party environments (Procgen, NetHack, Minigrid, etc.)
Custom wrappers (observation, reward, frame stacking, etc.)
Space conversion and unflattening
Environment registration
Compatibility patterns
Performance considerations
Debugging integration
Tips for Success
Start simple : Begin with Ocean environments or Gymnasium integration before creating custom environments
Profile early : Measure steps per second from the start to identify bottlenecks
Use templates : scripts/train_template.py and scripts/env_template.py provide solid starting points
Read references as needed : Each reference file is self-contained and focused on a specific capability
Optimize progressively : Start with Python, profile, then optimize critical paths with C if needed
Leverage vectorization : PufferLib's vectorization is key to achieving high throughput
Monitor training : Use WandB or Neptune to track experiments and identify issues early
Test environments : Validate environment logic before scaling up training
Check existing environments : Ocean suite provides 20+ pre-built environments
Use proper initialization : Always use layer_init from pufferlib.pytorch for policies
Common Use Cases
Training on Standard Benchmarks
env = pufferlib.make('atari-pong' , num_envs=256 )
env = pufferlib.make('procgen-coinrun' , num_envs=256 )
env = pufferlib.make('minigrid-empty-8x8' , num_envs=256 )
Multi-Agent Learning
env = pufferlib.make('pettingzoo-pistonball' , num_envs=128 )
policy = create_policy(env.observation_space, env.action_space)
trainer = PuffeRL(env=env, policy=policy)
Custom Task Development
class MyTask (PufferEnv ):
env = pufferlib.emulate(MyTask, num_envs=256 )
trainer = PuffeRL(env=env, policy=my_policy)
High-Performance Optimization
env = pufferlib.make(
'my-env' ,
num_envs=1024 ,
num_workers=16 ,
envs_per_worker=64
)
Installation
Documentation 