swe-universe-environments

name	swe-universe-environments
title	SWE-Universe: Scale Real-World Verifiable Environments to Millions
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.02361
keywords	["Software Engineering Agents","Environment Generation","Scalable Verification","GitHub","Autonomous Building"]
description	Automatically generate executable software engineering environments from GitHub pull requests at million-scale using an autonomous building agent. Detects superficial verification patterns to force genuine code execution testing.

SWE-Universe: Scalable Environment Generation

Problem

Software engineering (SWE) task datasets are small and limited to curated repositories. Agent training requires diverse, real-world verification environments. Manual environment construction scales poorly.

Current approaches use string-matching verification (grep-based) instead of actually executing code, enabling agents to cheat without solving real problems.

Core Concept

SWE-Universe uses an autonomous agent (custom Qwen model) to build executable verification scripts from GitHub pull requests. The agent iteratively tests scripts against buggy and fixed states, with in-loop detection of superficial verification patterns.

This creates the largest real-world SWE task collection with genuine executable verification.

Architecture Overview

• PR Processing Pipeline: Separate test and fix patches; filter low-quality PRs
• Autonomous Building Agent: Generates verification scripts from PR descriptions
• Iterative Validation: Test scripts against both buggy and fixed repository states
• Hack Detection Module: Identifies superficial patterns (grep instead of execution)
• Self-Correction Loop: Agent diagnoses failures and revises procedures
• Scaling: 807K+ instances across 52K repositories, 8 programming languages

Implementation

Step 1: Parse and Process GitHub PRs

Extract test and fix patches from pull requests.

def process_github_pr(pr_data):
    """Extract test and fix components from PR."""
    # Parse PR title and description
    title = pr_data['title']
    description = pr_data['body']

    # Separate modified files into fix components
    fix_patch = {}
    test_files = []

    for file_change in pr_data['files']:
        if 'test' in file_change['filename'].lower():
            test_files.append(file_change)
        else:
            fix_patch[file_change['filename']] = file_change[]

    
    language = infer_language(pr_data[])

     {
        : title,
        : description,
        : language,
        : fix_patch,
        : test_files,
        : pr_data[]
    }

 ():
    
    file_extensions = count_extensions_in_repo(repo)
    primary_ext = (file_extensions, key=file_extensions.get)
     extension_to_language(primary_ext)