| name | roboalign-language-action-reasoning |
| title | RoboAlign: Learning Test-Time Reasoning for Language Action Alignment in Vision Language Action Models |
| version | 0.0.3 |
| engine | skillxiv-v0.0.3-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2603.21341 |
| keywords | ["VLA","Embodied AI","Language-Action Alignment","Test-Time Reasoning","Robot Control"] |
| description | Bridge the modality gap between language reasoning and low-level robot actions through two-stage alignment training. Use RL with action-accuracy rewards to ground linguistic reasoning in precise motor control, achieving 17.5%-106.6% improvements over SFT baselines. |
Component ID
Two-Stage Language-Action Alignment Framework for VLAs
Motivation
Multimodal large language models (MLLMs) excel at embodied reasoning tasks, but these improvements don't consistently translate to better robot control. The gap exists because "modality gap between language and low-level actions"—language understanding and motor control are optimized separately. Reasoning improvements don't guarantee action accuracy.
The Modification
RoboAlign introduces two-stage training that directly aligns MLLM capabilities with robot action generation:
Stage 1: Supervised Fine-Tuning (SFT) - Train the model to generate low-level action tokens through reasoning. The model learns to emit action tokens as direct outcomes of spatial and temporal reasoning over visual inputs.
Stage 2: Reinforcement Learning (RL) - Apply RL with action-accuracy rewards to refine reasoning toward precise action execution. Instead of optimizing reasoning through language alone, sample action tokens as direct outcomes and evaluate reasoning quality through action accuracy feedback.
This grounds linguistic understanding in concrete motor control, bridging the modality gap.
How It Works
The mechanism operates by:
- Sampling action tokens as concrete realizations of the model's reasoning (not as a separate decoding step)
- Evaluating reasoning quality through action accuracy metrics (robot task success, trajectory closeness, etc.)
- Backpropagating rewards to improve both the reasoning process and the action emission mechanism
- Using minimal additional data for RL training (less than 1% of task trajectories)
The key insight is coupling language reasoning directly to motor control outcomes rather than optimizing them independently.
Ablation Results
The paper demonstrates substantial gains:
- LIBERO benchmark: 17.5% improvement over SFT baseline
- CALVIN benchmark: 18.9% improvement over SFT baseline
- Real-world environments: 106.6% improvement over SFT baseline
- Achieves these gains using less than 1% additional data for RL training
- Improvements hold across diverse robot embodiments and task families
Conditions
- Requires action-level ground truth or task success signals for RL reward computation
- Works best when task accuracy is measurable (robot manipulation, navigation)
- Assumes sufficient coverage of diverse reasoning scenarios in initial SFT data
