diffthinker-multimodal-reasoning

name	diffthinker-multimodal-reasoning
title	DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.24165
keywords	["diffusion models","multimodal reasoning","vision reasoning","sequential planning","spatial reasoning","MLLM"]
description	Apply diffusion models as native generative agents for vision-centric reasoning tasks (sequential planning, constraint satisfaction, spatial configuration) instead of text-based LLM chains. Achieves 3x+ improvements over GPT-5 and Gemini-3 on visual reasoning. Use when image-to-image generation better captures the reasoning constraints than text-based problem decomposition.

When to Use This Skill

• Sequential visual planning problems (step-by-step manipulation, assembly tasks)
• Combinatorial optimization with spatial constraints
• Constraint satisfaction problems visualizable as images
• Spatial configuration and layout reasoning tasks
• Problems where intermediate visual representations clarify the solution path

When NOT to Use This Skill

• Pure text reasoning tasks without visual grounding
• Tasks requiring guaranteed deterministic outputs
• Very large-scale problems (diffusion inference is iterative, not one-shot)
• Applications requiring real-time processing (<100ms latency)

Core Innovation

Traditional multimodal reasoning chains knowledge as text: LLMs decompose visual problems into linguistic steps, then reason through them sequentially. DiffThinker inverts this:

Instead of: Image → Extract facts → Reason in text → Generate image DiffThinker does: Image → Condition diffusion → Iteratively refine visual plan → Extract answer

This treats reasoning itself as an image-generation process where:

• Conditioning: Problem constraints come from the input image
• Iteration: Diffusion steps progressively refine the solution
• Extraction: The final image encodes the answer (trajectories, configurations, etc.)

Why Diffusion for Reasoning?

Diffusion models possess three properties that benefit multimodal reasoning:

1. Native parallelism: Multiple solution aspects evolve simultaneously (vs. sequential token generation)
2. Iterative refinement: Solutions improve gradually with feedback from intermediate states
3. Controllability: Fine-grained control over output structure via conditioning mechanisms

Architecture Pattern

# DiffThinker reasoning loop structure
class DiffusionReasoningAgent:
    def __init__(self, vision_encoder, diffusion_model, solution_decoder):
        self.encoder = vision_encoder  # Extract semantic features from input image
        self.diffusion = diffusion_model  
        .decoder = solution_decoder  

     ():
        
        
        problem_context = .encoder(input_image)

        
        x_t = torch.randn_like(input_image)

        
         step  (num_steps):
            
            denoised = .diffusion.denoise_step(
                x_t,
                context=problem_context,
                step=step,
                guidance_scale=  
            )
            
            x_t = .diffusion.reverse_step(denoised, x_t, step)

        
        solution = .decoder(x_t, problem_type)
         solution