| name | sage-long-video-agents |
| title | SAGE: Smart Any-Horizon Agents with Reinforcement Learning for Long-Context Video Reasoning |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2512.13874 |
| keywords | ["video-understanding","agent-reasoning","reinforcement-learning","long-context","multi-turn-reasoning"] |
| description | Enable agents to reason over long videos through multi-turn reasoning for complex questions and single-turn answering for simpler queries. Equip orchestrator VLM with multiple tools (web search, speech transcription, temporal grounding). Use synthetic data generation and multi-reward RL achieving 6.1% improvements on open-ended tasks. |
Skill Summary
SAGE introduces an agent-based system for long video reasoning operating in two paradigms: multi-turn reasoning for complex questions and single-turn answering for simpler queries. The orchestrator VLM (SAGE-MM) is equipped with multiple tools including web search, speech transcription, and temporal grounding. Synthetic data generation leverages Gemini-2.5-Flash achieving nearly 100× cost and 10× time savings. Multi-reward RL approach combines step-level rewards with LLM-as-judge evaluation, achieving 6.1% improvements on open-ended tasks and 8.2% gains for videos exceeding 10 minutes.
When To Use
- Building systems for long-duration video understanding and reasoning
- Scenarios requiring adaptive reasoning strategies based on question complexity
- Projects where temporal grounding and external tools enhance reasoning
- Research on multi-turn video agents with RL optimization
When NOT To Use
- Short-form video tasks not benefiting from multi-turn complexity
- Applications without access to external tools or context-augmentation
- Scenarios where synthetic data quality is problematic
- Domains where simpler single-pass models suffice
Core Technique
Multiple innovations enable effective long video reasoning:
1. System Design
Use orchestrator VLM (SAGE-MM) equipped with multiple tools rather than relying solely on temporal grounding:
- Web search: external context
- Speech transcription: audio understanding
- Temporal grounding: video-specific localization
Implement two paradigms:
- Multi-turn reasoning: complex questions requiring multiple reasoning steps
- Single-turn answering: simpler queries with direct answers
2. Synthetic Data Generation
Leverage Gemini-2.5-Flash to generate high-quality QnA pairs efficiently, achieving "nearly 100× cost and 10× time savings compared to human annotation and subclip processing pipelines."
3. RL Post-Training Recipe
Implement multi-reward approach combining:
- Step-level rewards: format, reasonable tool use, argument validity
- LLM-as-judge evaluation: for open-ended problems avoiding string-matching metrics
4. SAGE-Bench Evaluation
Curated benchmark with "1744 manually verified samples spanning diverse durations" emphasizing open-ended questions from entertainment videos.
Results
- Open-ended tasks: 6.1% improvement
- Videos exceeding 10 minutes: 8.2% improvement
- Validates effectiveness of adaptive reasoning for variable-duration content
