swe-lego-issue-resolution

name	swe-lego-issue-resolution
title	SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2601.01426
keywords	["Software Engineering","Fine-tuning","Issue Resolution","Test-time Scaling","LLM Agents"]
description	Achieve state-of-the-art software issue resolution through lightweight supervised fine-tuning with structured datasets and curriculum learning, plus test-time scaling—reaching 52.6% on SWE-Bench Verified and 58.8% with test-time strategies, outperforming complex multi-stage training.

Overview

SWE-Lego demonstrates that sophisticated software engineering agent capabilities can be achieved through a carefully designed lightweight approach: supervised fine-tuning (SFT) only, without requiring complex multi-stage training pipelines (mid-training, SFT, RL combinations).

Core Insight: State-of-the-art software issue resolution doesn't require elaborate training paradigms. Instead, three key components—high-quality data, refined SFT procedure, and test-time scaling—deliver competitive performance at fraction of typical complexity.

Three Building Blocks

Block 1: SWE-Lego Dataset (32k + 18k trajectories)

A carefully curated dataset combining real and synthetic data for both quality and quantity:

Composition:

• 32,000 task instances - High-quality, diverse software engineering tasks
• 18,000 validated trajectories - Correct solution paths for training
• Real data - Authentic GitHub issues and PRs
• Synthetic data - Complementary examples filling capability gaps

Quality Control:

• Trajectory validation ensuring correctness
• Diversity across programming languages and issue types
• Difficulty stratification for curriculum learning
• Removal of low-quality or ambiguous examples

Key Advantage: Balanced coverage of both common patterns and edge cases through synthetic-real combination.

Block 2: Refined SFT Procedure

Enhanced training methodology improving action quality and overall performance:

Error Masking: Filter trajectories containing invalid tool calls or impossible actions. Prevents model learning degenerate patterns.

def mask_errors_in_trajectory(trajectory):
    """Filter out invalid actions from training trajectory."""
    valid_actions = []
    for step in trajectory:
        if is_valid_action(step.action, step.context):
            valid_actions.append(step)
        else:
            # Skip invalid action; continue from next valid state
            pass
    return valid_actions