gem-agentic-llm-environment

name	gem-agentic-llm-environment
title	GEM: A Gym for Agentic LLMs
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.01051
keywords	["environment","agent-training","RL-infrastructure","benchmark","LLM-agents"]
description	A standardized environment framework for training and evaluating LLM agents, providing 24+ tasks with asynchronous vectorized execution, extensible wrappers, and integration examples for five RL frameworks. Enables reproducible agent research and training at scale.

GEM: Unified Environment Framework for LLM Agents

Reinforcement learning frameworks like OpenAI Gym standardized RL research by providing consistent interfaces between agents and environments. The challenge is that LLM agents operate differently than traditional RL agents—they generate language, operate in long-horizon settings, and require different reward signals. GEM adapts the Gym paradigm to agentic LLMs.

Without standardized environments, every paper implements its own simulator, making comparison difficult and reproduction expensive. GEM solves this by providing a unified interface with diverse task environments, batch execution support, and integration examples for popular RL frameworks (GRPO, DPO, RL4LM, etc.).

Core Concept

GEM follows the OpenAI-Gym API but extends it for language agents:

• Step interface: (observation_text, available_actions) -> (action_text, reward, done, info)
• Vectorized execution: Asynchronous batching of multiple agent-environment interactions
• Task diversity: 24+ environments spanning web search, tool use, coding, reasoning, planning
• Extensibility: Custom wrapper system for adding new tasks or modifying existing ones
• Framework integration: Ready-to-run training examples for 5+ RL algorithms

Architecture Overview

• Core environment class: Implements standard Gym interface for text-based interactions
• Action space: Discrete or continuous (typically discrete with language actions)
• Observation space: Text descriptions + available action list
• Reward function: Task-specific (binary success/failure or dense intermediate rewards)
• Vectorized executor: Batch processing with async task scheduling
• Wrapper system: Composition-based customization (logging, preprocessing, reward shaping)

Implementation Steps

Create a basic GEM environment by extending the base class. Here's a minimal task (a reasoning problem):

from gem import BaseEnvironment
import numpy as np

class SimpleMathEnv(BaseEnvironment):
    """
    Minimal environment: solve arithmetic problems.
    """
     ():
        ().__init__()
        .problems = ._generate_problems(num_problems, problem_type)
        .current_idx = 
        .step_count = 
        .max_steps = 

     ():
        problems = []
         problem_type == :
             _  (num):
                a, b = np.random.randint(, , )
                problems.append({: a, : b, : a + b})
         problems

     ():
        
        .current_idx = np.random.randint(, (.problems))
        .step_count = 
        prob = .problems[.current_idx]
        observation = 
         observation, .get_available_actions()

     ():
        
        .step_count += 
        prob = .problems[.current_idx]

        :
            agent_answer = (action.strip())
            is_correct = agent_answer == prob[]
         ValueError:
            is_correct = 

        reward =   is_correct  
        done = is_correct  .step_count >= .max_steps

        feedback =   is_correct  
        observation = 

         observation, reward, done, {: is_correct}

     ():
        
        
         []

     ():
        prob = .problems[.current_idx]
        ()