v-thinker-interactive-visual-reasoning

name	v-thinker-interactive-visual-reasoning
title	V-Thinker: Interactive Thinking with Images
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.04460
keywords	["Interactive Reasoning","Visual Tools","Vision-Language Models","Reinforcement Learning","Data Synthesis"]
description	Enable vision-centric interactive reasoning by synthesizing diverse reasoning datasets through co-evolutionary loops, then training models with progressive curriculum that starts with perception and advances to tool-based problem solving.

Title: Train Models to Think Visually Through Interactive Tool Use

Standard vision-language models can recognize objects but struggle with interactive visual reasoning—using tools like pointing, drawing, zooming to solve problems. V-Thinker bootstraps this capability through three key mechanisms: (1) data evolution flywheel that synthesizes high-quality reasoning datasets across dimensions of variety/quality/difficulty, (2) progressive training curriculum from perception to interaction, and (3) reward-shaped RL that incentivizes effective tool use.

The approach transforms VLMs from passive observers into active problem-solvers.

Core Concept

Data Evolution and Progressive Training for Visual Interaction:

• Data Flywheel: Knowledge concepts and tool use co-evolve, expanding diversity
• Quality Calibration: Checker-repairer loop maintains correctness
• Difficulty Progression: Systematically increase problem complexity
• Perception First: SFT on basic spatial understanding before tool use
• Interactive RL: GRPO optimizes for effective reasoning and tool application

Architecture Overview

• Data Synthesis Engine: Automatically generates varied, high-quality reasoning tasks
• V-Perception-40K Dataset: Foundational perception with point-level supervision
• V-Interaction-400K Dataset: Evolved interactive reasoning examples
• Progressive Curriculum: Two-stage training (perception → interaction)
• VTBench: 1,500 expert-reviewed problems for evaluation

Implementation Steps

1. Implement Data Evolution Flywheel

Create system that automatically synthesizes and improves training data.

class DataEvolutionFlywheel:
    def __init__(self, base_concepts, base_tools):
        self.concepts = set(base_concepts)
        self.tools = set(base_tools)

    def expand_concepts(self):
        
        
        
        new_concepts = []
         concept_pair  combinations(.concepts, ):
            hybrid = 
            new_concepts.append(hybrid)

        
         concept  (.concepts):
             modifier  [, , ]:
                new_concepts.append()

        .concepts.update(new_concepts)
         new_concepts

     ():
        
        
        new_tools = []

         concept  .concepts:
               concept     .tools:
                new_tools.append()
               concept     .tools:
                new_tools.append()
               concept     .tools:
                new_tools.append()

        .tools.update(new_tools)
         new_tools

     ():
        
        problems = []

         _  (num_problems):
            
            concept = random.choice((.concepts))
            required_tools = .tools_for_concept(concept)

            
            problem = {
                : concept,
                : required_tools,
                : .generate_problem_image(concept),
                : .generate_question(concept),
                :   
            }

            problems.append(problem)

         problems

     ():
        
        mapping = {
            : [, ],
            : [],
            : [],
            : [],
            : []
        }
         mapping.get(concept, [])