| name | sensenovamars-multimodal-reasoning |
| title | SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2512.24330 |
| keywords | ["Multimodal Agents","Vision-Language Models","Agentic Reasoning","Tool Use","RL Training"] |
| description | Build vision-language agents that seamlessly integrate visual reasoning with dynamic tool manipulation (search, cropping) through reinforcement learning, achieving state-of-the-art performance on fine-grained visual understanding tasks—surpassing proprietary models like Gemini and GPT. |
Overview
SenseNova-MARS is a multimodal agentic framework enabling Vision-Language Models (VLMs) to perform sophisticated visual reasoning tasks through interleaved tool manipulation and continuous reasoning.
Core Innovation: While VLMs excel at text-oriented reasoning and isolated tool calls, they struggle with coordinated tool sequences on knowledge-intensive visual tasks. SenseNova-MARS enables human-like proficiency in dynamically invoking multiple tools (search, image cropping, text search) to solve complex visual understanding problems.
Architecture
Tool Integration
Three complementary tools enable sophisticated visual problem-solving:
Image Search Tool
- Retrieves semantically related images from knowledge bases
- Supports fine-grained visual understanding through reference images
- Enables comparative analysis across multiple images
Text Search Tool
- Queries knowledge bases for text-based information
- Supports factual grounding of visual analysis
- Bridges visual content with textual knowledge
Image Crop Tool
- Extracts regions of interest from high-resolution images
- Enables focus on fine-grained visual details
- Supports iterative refinement of understanding
Agentic Reasoning Loop
The agent operates through repeated cycles of:
- Perceive - Analyze current visual content and available information
- Reason - Determine which tools would advance understanding
- Act - Invoke appropriate tools (search, crop, etc.)
- Integrate - Synthesize results into improved understanding
RL Training: Batch-Normalized Group Sequence Policy Optimization
To enable reliable tool invocation and reasoning, SenseNova-MARS introduces BN-GSPO algorithm:
Key Components:
- Group Sequence Policy Optimization - Trains on sequences of decisions, not isolated actions
- Batch Normalization - Stabilizes training dynamics in high-variance visual environments
- Reinforcement Signal - Rewards both intermediate tool selections and final correctness
Training Stability Features:
- Handles variable-length action sequences
- Robust to initialization variance in visual encoders
