agent-world-model-env-generation

name	agent-world-model-env-generation
title	Agent World Model: Infinity Synthetic Environments for Agentic RL
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.10090
keywords	["Synthetic Environments","Environment Generation","Database-Driven","Task Synthesis","Tool Use Training"]
description	Automatically synthesize executable RL training environments with database backends, Python tools, and task descriptions. Generate 1000+ diverse domains with 10K+ tasks enabling data-efficient tool-use agent training without manual scenario design.

Agent World Model: Automated Synthetic Environment Generation

Creating diverse training environments for tool-use agents is expensive and requires manual scenario design. Agent World Model (AWM) automates this through a multi-stage synthesis pipeline: starting from seed domain names, systematically generates scenarios, synthesizes executable databases and tool interfaces, and verifies task solvability. The result: 1000+ diverse environments with 35K+ tools across 10K+ tasks, enabling agents to train without human environment design.

Core Concept

Manual environment creation: expensive, limited diversity, slow iteration.

AWM approach: LLM-driven multi-stage synthesis:

1. Scenario Generation (1000 scenarios from 100 seeds via LLM expansion)
2. Task Synthesis (10 user-facing tasks per scenario)
3. Environment Synthesis (SQLite database + Python tool APIs)
4. Self-Correction (retry failed components up to 5 times)

Result: executable environments with verifiable success criteria, ready for RL training.

Architecture Overview

• Scenario Expansion: LLM generates diverse scenarios (e.g., e-commerce, banking, social) from domain seeds
• Task Generation: For each scenario, synthesize 10 user-facing goals (API-solvable)
• Database Schema: Auto-generate SQLite schema matching scenario (tables, columns, sample data)
• Tool API: Expose Python functions as Model Context Protocol (MCP) tools
• Task Verification: Code-augmented validation ensures tasks are solvable and rewards are meaningful
• Self-Correction Loop: Retry failed components with error feedback (up to 5 attempts)

Implementation

Implement scenario and task generation:

import json
import sqlite3
from typing import List, Dict

class EnvironmentGenerator:
    """Generates synthetic training environments."""

    def __init__(self, llm_model, num_scenarios=1000, tasks_per_scenario=):
        .llm = llm_model
        .num_scenarios = num_scenarios
        .tasks_per_scenario = tasks_per_scenario
        .generated_envs = []

     ():
        
        expansion_prompt = 

        response = .llm.generate(expansion_prompt, max_tokens=)
        scenarios = json.loads(response)

         scenarios

     () -> []:
        
        task_prompt = 

        response = .llm.generate(task_prompt, max_tokens=)
        tasks = json.loads(response)

         tasks

     () -> :
        
        schema_prompt = 

        response = .llm.generate(schema_prompt, max_tokens=)
        schema = json.loads(response)

         schema

     () -> :
        
        conn = sqlite3.connect()
        cursor = conn.cursor()

        
         table  schema[]:
            col_defs = .join([
                
                 col  table[]
            ])
            sql = 
            cursor.execute(sql)

        
         table_name, rows  schema.get(, {}).items():
             row  rows:
                cols = .join(row.keys())
                vals = .join([  (v, )  (v)  v  row.values()])
                sql = 
                cursor.execute(sql)

        conn.commit()
        conn.close()

         

     () -> :
        
        tool_prompt = 

        response = .llm.generate(tool_prompt, max_tokens=)
        tools = json.loads(response)

         tools

     ():
        
         attempt  (max_retries):
            :
                
                tasks = .synthesize_tasks(scenario, domain)

                
                schema = .generate_database_schema(scenario)
                db_path = .create_database(scenario.replace(, ), schema)

                
                tools = .synthesize_tool_apis(scenario, schema)

                
                success = ._verify_environment(scenario, tasks, db_path, tools)

                 success:
                     {
                        : scenario,
                        : domain,
                        : tasks,
                        : db_path,
                        : schema,
                        : tools
                    }

             Exception  e:
                ()
                 attempt == max_retries - :
                    ()
                     

         

     () -> :
        
        
         (tasks) >=   (tools.get(, [])) >=