cooper-spatial-intelligence

name	cooper-spatial-intelligence
title	COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.04563
keywords	["spatial reasoning","3D understanding","auxiliary modalities","multimodal LLMs","depth and segmentation"]
description	Enhance spatial reasoning in multimodal LLMs by integrating depth and segmentation as auxiliary modalities with adaptive reasoning strategies. COOPER achieves 6.91% improvement in spatial understanding—when you need 3D-aware vision-language capabilities.

Overview

COOPER unifies cooperative perception and reasoning through a two-stage training approach that develops both auxiliary modality generation and adaptive reasoning capabilities. Rather than treating perception and reasoning separately, the model learns to generate depth and segmentation maps while developing interleaved reasoning strategies.

When to Use

• Multimodal tasks requiring strong 3D spatial understanding
• Applications needing distance and size estimation from images
• Vision-language models that struggle with spatial relationships
• Scenarios requiring reasoning over spatial properties (volume, distance, orientation)
• Tasks involving scene understanding with geometric constraints

When NOT to Use

• 2D image analysis where depth adds no value
• Tasks not requiring spatial reasoning
• Models already achieving satisfactory spatial understanding
• Real-time applications where auxiliary modality generation adds latency
• Scenarios with limited 3D training data

Core Technique

Two-stage training developing auxiliary modality generation and adaptive reasoning:

# Unified spatial reasoning architecture
class CooperativeSpatialModel:
    def __init__(self, vllm_backbone):
        self.vllm = vllm_backbone

        # Auxiliary modality generators
        self.depth_generator = DepthDecoder()
        self.segmentation_generator = SegmentationDecoder()

        # Adaptive reasoning module
        self.reasoning_adapter = ReasoningAdapter()

    def forward(self, image, question):
        """
        Unified perception and reasoning for spatial intelligence.
        Generates auxiliary modalities and performs adaptive reasoning.
        """
        # Extract visual features
        features = self.vllm.encode_image(image)

        # Generate auxiliary modalities
        depth_map = .depth_generator(features)
        segmentation = .segmentation_generator(features)

        
        enhanced_features = .integrate_modalities(
            features, depth_map, segmentation
        )

        
        reasoning_path = .reasoning_adapter.compute_path(
            enhanced_features, question
        )

        
        answer = .vllm.decode_with_path(
            enhanced_features,
            question,
            reasoning_path
        )

         answer, depth_map, segmentation

     ():
        
        
        depth_features = .process_depth(depth)
        
        seg_features = .process_segmentation(segmentation)

        
        combined = torch.cat([visual, depth_features, seg_features], dim=-)
         combined

     ():
        
        complexity_score = .estimate_spatial_complexity(question)
         complexity_score > :
            
             .multi_step_reasoning(features, question)
        :
            
             .direct_reasoning(features, question)