code2worlds

name	code2worlds
title	Code2Worlds: Empowering Coding LLMs for 4D World Generation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.11757
keywords	["3D Generation","Physics Simulation","Code Generation","Dual-Stream Architecture","World Modeling"]
description	Generate physically grounded 4D scenes from natural language through dual-stream architecture separating object detail from scene orchestration. VLM-Motion Critic validates physics parameters iteratively, bridging semantic-physical execution gap.

Code2Worlds: 4D World Generation from Code

Problem Context

Generating realistic 3D environments with physics requires balancing two competing demands: high-fidelity individual objects and coherent global layouts. Open-loop code generation often produces "physical hallucinations" where visual structures don't align with physics. A single monolithic generator struggles with multi-scale context management. Existing methods sacrifice local detail for global structure or vice versa.

Core Concept

Code2Worlds uses a dual-stream architecture to decouple concerns in 4D world generation:

1. Object Stream: Generates high-fidelity individual objects through retrieval-augmented parameter generation
2. Scene Stream: Orchestrates global environment with hierarchical planning

Rather than open-loop generation, a VLM-Motion Critic validates rendered simulations and iteratively refines physics parameters, bridging the semantic-physical execution gap. The system generates executable procedural code that creates both geometry and physics-driven animations.

Architecture Overview

• Dual-Stream Generation: Object and scene streams with separate concerns
• Retrieval-Augmented Parameters: Library-based parameter selection for object fidelity
• Hierarchical Scene Planning: Multi-scale layout reasoning
• Closed-Loop Refinement: VLM-Motion Critic validates and refines
• Physics Parameter Inference: Automatic tuning of simulation parameters
• Temporal Coherence: Ensures physics-consistent evolution over time
• Executable Procedural Code: Generates Blender Python scripts or similar

Implementation

Dual-stream architecture for 4D generation:

class DualStreamWorldGenerator(nn.Module):
    """
    Generate 4D scenes through decoupled object and scene streams.
    Prevents compromise between local detail and global coherence.
    """

    def __init__(self, object_generator, scene_generator, vlm_critic):
        super().__init__()
        self.object_stream = object_generator
        self.scene_stream = scene_generator
        .vlm_critic = vlm_critic  

        
        .object_library = ObjectParameterLibrary()

     ():
        
        objects = []

         obj_idx  (num_objects):
            
            obj_spec = parse_object_spec(semantic_description, obj_idx)

            
            similar_params = .object_library.retrieve(
                obj_spec, top_k=)

            
            object_params = .object_stream(
                obj_spec, similar_params, embeddings=)

            
            obj = {
                : obj_spec[],
                : object_params,
                : generate_shape(object_params),
                : generate_material(object_params)
            }
            objects.append(obj)

         objects

     ():
        
        scene = {
            : objects,
            : []
        }

         level  (num_levels):
            
            
            

             level == :
                
                layout = .scene_stream.generate_layout(
                    semantic_description)
             level == :
                
                groups = .scene_stream.generate_groups(
                    objects, semantic_description)
            :
                
                positions = .scene_stream.generate_positions(
                    objects, groups)

            scene[].append({
                : level,
                : ,  
                : 
            })

         scene

     ():
        
        physics_params = {}

         obj  scene[]:
            
            size = obj[][]
            material = obj[][]
            mass = infer_mass_from_material(material, size)

            
            friction = material_friction_lookup(material)

            
            elasticity = infer_elasticity(material, scene)

            physics_params[obj[]] = {
                : mass,
                : friction,
                : elasticity,
                : 
            }

         physics_params

     ():
        
        code = 

        
         obj  scene[]:
            code += 

        
        code += 
         obj, params  physics_params.items():
            code += 

        
        code += 

         code