unreasonable-scaling-computer-use-agents

name	unreasonable-scaling-computer-use-agents
title	Unreasonable Effectiveness of Scaling Computer Use Agents with Behavior Judgment
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.02250
keywords	["computer-use-agents","scaling","evaluation","trajectory-understanding"]
description	Scale computer-use agents from 30% to 72% success rate by generating parallel rollouts and selecting best trajectories through behavior narrative evaluation. Use when deploying desktop agents on complex, high-variance task scenarios.

Unreasonable Effectiveness of Scaling Computer-Use Agents with Behavior Judgment

This work identifies a fundamental bottleneck in scaling computer-use agents: "wide scaling exposes a fundamental bottleneck: evaluation." The solution converts dense agent trajectories into compact behavior narratives capturing task-relevant changes, enabling intelligent best-of-N selection via comparative evaluation.

Core Architecture

• Parallel trajectory execution: Generate multiple independent agent rollouts per task
• Behavior narrative generation: Compress trajectories into task-relevant summaries (10-20% original length)
• Comparative evaluation: Select best trajectory through structured pairwise comparison
• Cross-platform validation: Works on OSWorld (desktop), WindowsAgentArena (Windows), AndroidWorld (mobile)

Implementation Steps

Setup parallel agent executor with behavior narrative evaluation:

# Initialize parallel computer-use agent system
from bjudge import ParallelAgentExecutor, BehaviorNarrativeGenerator

executor = ParallelAgentExecutor(
    base_agent=your_computer_agent,
    num_rollouts=4,  # generate 4 parallel attempts per task
    executor="isolated_vms",  # each rollout in isolated environment
    snapshot_capability=True  # required for rollback
)

narrative_generator = BehaviorNarrativeGenerator(
    llm=your_evaluator_model,
    max_narrative_length=1000,  # tokens
    focus="task_relevant_changes"
)

Execute parallel agent execution with trajectory selection:

# Task execution with parallel rollouts
task = "Download Q1 financial reports from company website and extract revenue"

# Stage 1: Generate multiple rollouts in parallel
trajectories = executor.execute_parallel(
    task=task,
    num_rollouts=4,
    max_steps_per_rollout=50,
    environment_snapshots=True
)

# Stage 2: Convert trajectories to behavior narratives
narratives = []
 trajectory  trajectories:
    narrative = narrative_generator.generate(
        trajectory=trajectory,
        task_description=task,
        include_action_sequence=,
        include_state_changes=,
        summarization_ratio=  
    )
    narratives.append(narrative)


best_trajectory_idx = executor.select_best_trajectory(
    narratives=narratives,
    task=task,
    evaluation_criteria=[
        ,
        ,
        ,
        
    ]
)


best_result = trajectories[best_trajectory_idx]
()