Fine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads, LoRA adaptation, and FiLM conditioning on LIBERO simulation and ALOHA real-world setups. Use when reproducing OpenVLA-OFT paper results, training custom VLA action heads (L1 or diffusion), deploying server-client inference for ALOHA, or debugging normalization, LoRA merge, and cross-GPU issues.
Instrucciones de origen · Vista previa de solo lectura
name
fine-tuning-openvla-oft
description
Fine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads, LoRA adaptation, and FiLM conditioning on LIBERO simulation and ALOHA real-world setups. Use when reproducing OpenVLA-OFT paper results, training custom VLA action heads (L1 or diffusion), deploying server-client inference for ALOHA, or debugging normalization, LoRA merge, and cross-GPU issues.
Fine-tuning and evaluation workflows for OpenVLA-OFT and OpenVLA-OFT+ from the official openvla-oft codebase. Covers blank-machine setup plus LoRA-based adaptation of OpenVLA for robot action generation with continuous action prediction heads.
Quick start
Clone the public repo, follow the official setup, then evaluate a pretrained LIBERO checkpoint:
What OpenVLA-OFT changes: Standard OpenVLA tokenizes continuous actions into discrete bins, losing precision. OFT replaces this with dedicated continuous action heads (L1 regression or diffusion) while keeping the VLA backbone frozen and adapting via LoRA.
OFT vs OFT+ variants:
Variant
FiLM
Images
Typical use
OFT
Off
2 (front + wrist)
LIBERO simulation
OFT+
On
3 (high + left + right wrist)
ALOHA real-world
Key architecture choices:
LoRA adaptation: Rank-32 LoRA on VLA backbone (no full fine-tuning needed)
Continuous actions: L1 regression head (default) or diffusion head
FiLM conditioning: Feature-wise Linear Modulation for stronger language grounding in OFT+
Multi-image input: Configurable 2 or 3 camera streams via num_images_in_input
Compute requirements
Task
GPU
VRAM
Notes
LIBERO evaluation
1x A100/A40
~16 GB
Single GPU
ALOHA evaluation
1x A100/A40
~18 GB
Single GPU
LIBERO fine-tuning
8x A100
~27 GB/GPU
Paper default
ALOHA fine-tuning (OFT+)
8x A100
~35 GB/GPU
FiLM + 3 images
LoRA merge
1x any GPU
~16 GB
One-time step
Expected performance benchmarks
Official results (paper setup, seed=7, 50 trials per task):
Task Suite
Task-Specific
Combined Policy
Notes
LIBERO-Spatial
97.2%
96.8%
Easiest suite
LIBERO-Object
97.4%
97.0%
Object manipulation
LIBERO-Goal
95.8%
95.4%
May peak at 50k-100k steps
LIBERO-10
98.0%
98.0%
Long-horizon tasks
Average
97.1%
96.8%
Near-equivalent
Reproduction notes: results are tied to Python 3.10.14, PyTorch 2.2.0, NVIDIA A100, and custom Transformers fork.
When to use vs alternatives
Use OpenVLA-OFT when:
The target task is robot action generation with visual and language conditioning
LoRA-based adaptation of openvla/openvla-7b is preferred
You need official LIBERO or ALOHA workflows from the OpenVLA-OFT paper
You want continuous action heads (L1 regression or diffusion) instead of tokenized actions
Use alternatives when:
You need a different VLA architecture (use fine-tuning-serving-openpi for pi0/pi0.5 models)
You need the NVIDIA Cosmos Policy stack (use evaluating-cosmos-policy)
You need general LLM fine-tuning without robot action heads
Evaluate 50k, 100k, and 150k checkpoints — LIBERO-Goal may peak earlier than other suites. Keep best checkpoint per suite by actual task success, not only training loss.
Workflow 4: Train and evaluate OpenVLA-OFT+ on ALOHA
Detailed reference: See references/aloha-workflow.md for the full ALOHA server-client setup, data preprocessing, dataset registration, and troubleshooting.
ALOHA Progress:
- [ ] Step 1: Preprocess raw ALOHA demonstrations
- [ ] Step 2: Convert to RLDS and register dataset configs
- [ ] Step 3: Fine-tune OFT+ with FiLM and 3 images
- [ ] Step 4: Start VLA server on GPU machine
- [ ] Step 5: Run client-side robot evaluation
