vision-zero-self-play-vlm-improvement

name	vision-zero-self-play-vlm-improvement
title	Vision-Zero: Label-Free Self-Play for VLM Self-Improvement
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2509.25541
keywords	["VLM","self-play","RL","zero-label","multimodal-learning"]
description	Enable Vision-Language Models to improve without human annotations through competitive multi-agent self-play games (Who-Is-the-Spy format) combined with Iterative Self-Play Policy Optimization. Use when annotation costs limit multimodal dataset scaling or domain diversity.

Vision-Zero: Label-Free Self-Play for VLM Self-Improvement

Vision-Zero addresses the bottleneck of expensive multimodal annotation by introducing a label-free framework where VLMs competitively play games (Who-Is-the-Spy) against each other, generating training signals without human labels while developing strategic reasoning and inference optimization.

Core Architecture

• Multi-agent game dynamics: Asymmetric information game where spy identifies non-spy, non-spies identify spy
• Self-play loop: Agents alternate roles across episodes, learning both defensive and offensive visual reasoning
• Iterative-SPO: Policy optimization using game outcomes (win/loss) as sparse reward signals
• Role-based training: Asymmetric objectives shape distinct reasoning patterns for identification vs. deception

Implementation Steps

Initialize self-play framework with VLM baseline:

# Setup competitive game environment
from vision_zero import SelfPlayGame, IterativeSPO

# Create game with asymmetric roles
game = SelfPlayGame(
    model_name="Qwen2.5-VL-7B",
    num_agents=2,  # Spy vs. non-spies
    game_type="who_is_the_spy",
    max_rounds=5,
    image_domains=["synthetic", "chart", "real-world"]
)

# Initialize policy optimizer
optimizer = IterativeSPO(
    learning_rate=1e-5,
    policy_update_interval=100,  # update after every 100 games
    kl_coefficient=0.1,
    temperature=0.7
)

Execute self-play training with role switching:

# Run self-play iterations
for iteration in range(num_iterations):
    # Episode with current policy
    trajectories = game.play_episodes(
        num_episodes=100,
        policy=current_policy,
        role_switch=  
    )

    
    rewards = game.compute_rewards(trajectories)

    
    current_policy = optimizer.update(trajectories, rewards)