roboalign-language-action-reasoning

name	roboalign-language-action-reasoning
title	RoboAlign: Learning Test-Time Reasoning for Language Action Alignment in Vision Language Action Models
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.21341
keywords	["VLA","Embodied AI","Language-Action Alignment","Test-Time Reasoning","Robot Control"]
description	Bridge the modality gap between language reasoning and low-level robot actions through two-stage alignment training. Use RL with action-accuracy rewards to ground linguistic reasoning in precise motor control, achieving 17.5%-106.6% improvements over SFT baselines.

Component ID

Two-Stage Language-Action Alignment Framework for VLAs

Motivation

Multimodal large language models (MLLMs) excel at embodied reasoning tasks, but these improvements don't consistently translate to better robot control. The gap exists because "modality gap between language and low-level actions"—language understanding and motor control are optimized separately. Reasoning improvements don't guarantee action accuracy.

The Modification

RoboAlign introduces two-stage training that directly aligns MLLM capabilities with robot action generation:

Stage 1: Supervised Fine-Tuning (SFT) - Train the model to generate low-level action tokens through reasoning. The model learns to emit action tokens as direct outcomes of spatial and temporal reasoning over visual inputs.

Stage 2: Reinforcement Learning (RL) - Apply RL with action-accuracy rewards to refine reasoning toward precise action execution. Instead of optimizing reasoning through language alone, sample action tokens as direct outcomes and evaluate reasoning quality through action accuracy feedback.

This grounds linguistic understanding in concrete motor control, bridging the modality gap.

How It Works

The mechanism operates by:

1. Sampling action tokens as concrete realizations of the model's reasoning (not as a separate decoding step)
2. Evaluating reasoning quality through action accuracy metrics (robot task success, trajectory closeness, etc.)
3. Backpropagating rewards to improve both the reasoning process and the action emission mechanism
4. Using minimal additional data for RL training (less than 1% of task trajectories)

The key insight is coupling language reasoning directly to motor control outcomes rather than optimizing them independently.

Ablation Results

The paper demonstrates substantial gains:

• LIBERO benchmark: 17.5% improvement over SFT baseline
• CALVIN benchmark: 18.9% improvement over SFT baseline
• Real-world environments: 106.6% improvement over SFT baseline
• Achieves these gains using less than 1% additional data for RL training
• Improvements hold across diverse robot embodiments and task families

Conditions

• Requires action-level ground truth or task success signals for RL reward computation
• Works best when task accuracy is measurable (robot manipulation, navigation)
• Assumes sufficient coverage of diverse reasoning scenarios in initial SFT data
• RL stage requires relatively small amounts of additional interaction data (< 1%)

Drop-In Checklist

• Implement SFT training on (image, reasoning, action token) triplets
• Design action-accuracy reward function appropriate for target tasks
• Integrate RL training loop with action token sampling
• Implement backpropagation of rewards to both reasoning and action generation
• Collect/prepare minimal RL training data (< 1% of total trajectories)
• Benchmark on LIBERO, CALVIN, and real-world robot tasks
• Validate that language reasoning improvements transfer to action accuracy