primo-r1-vision-critic

name	primo-r1-vision-critic
title	From Passive Observer to Active Critic: RL Elicits Process Reasoning for Robotic Manipulation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.15600
keywords	["Visual Reasoning","Process Supervision","Robotic Control","Reinforcement Learning","Chain-of-Thought Video"]
description	Transform video multimodal models into active process critics for robotic tasks. Use RL to incentivize explicit reasoning about progress toward goals and anchor reasoning temporally between initial and current states.

PRIMO R1: Active Process Criticism for Robotic Manipulation

Video multimodal language models typically function as passive observers, recognizing events but not evaluatively assessing progress. PRIMO R1 transforms these models into active critics through two key mechanisms: (1) using reinforcement learning to incentivize explicit chain-of-thought reasoning about task progress, and (2) anchoring reasoning temporally by explicitly linking initial and current state images. The result is a 7B model that achieves 67.0% accuracy on robotic failure detection—surpassing closed-source 72B models by 6% and matching models like OpenAI o1 with 50% error reduction on manipulation benchmarks.

The approach combines outcome-based RL with structured temporal context, enabling models to develop genuine process supervision capabilities for long-horizon robot tasks.

Core Concept

PRIMO R1 transforms passive vision-language models through:

1. RL-Driven Reasoning — Use outcome rewards to train explicit chain-of-thought generation
2. Temporal Anchoring — Provide structured context linking sequence start to current frame
3. Progress Estimation — Train model to generate step-by-step assessments of task progress
4. Failure Detection — Use process reasoning to identify error conditions early

The key insight: explicit reasoning about progress, when incentivized by RL, enables models to develop genuine understanding of robot action effects rather than pattern matching.

Architecture Overview

• Video Encoder — Process video frames and extract temporal sequences
• Temporal Anchor Representation — Explicit structure linking initial and current states
• Process Reasoner — Generate intermediate reasoning steps about progress
• Progress Estimator — Predict task completion likelihood at each step
• Error Detector — Flag failure conditions based on reasoning
• Outcome Reward Model — Ground truth signal for RL training
• RL Optimizer — Policy gradient training for reasoning improvement

Implementation Steps

Start by implementing the temporal anchoring mechanism that structures context.

import torch
import torch.nn as nn
from typing import List

 :
    

     ():
        .embed_dim = embed_dim

        
        .initial_anchor = nn.Parameter(torch.randn(embed_dim))
        .current_anchor = nn.Parameter(torch.randn(embed_dim))
        .temporal_marker = nn.Parameter(torch.randn(embed_dim))

     ():
        
        batch_size = initial_frame.size()

        
        anchored = initial_frame + .initial_anchor.unsqueeze()

        
         intermediate_frames:
             i, frame  (intermediate_frames):
                
                temporal_pos = (i + ) / (intermediate_frames)
                pos_embed = torch.sin(torch.tensor([temporal_pos])) * .temporal_marker
                anchored = torch.cat([anchored, frame + pos_embed], dim=)

        
        anchored = torch.cat([anchored,
                             current_frame + .current_anchor.unsqueeze()],
                            dim=)

         anchored  

     ():
        
        
        initial = video_sequence[:, , :]  
        current = video_sequence[:, -, :]

        intermediate = [video_sequence[:, i, :]  i  (, video_sequence.size() - )]

        
        anchored = .create_anchored_context(initial, current, intermediate)

         anchored