x-vla-embodiment

name	x-vla-embodiment
title	X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment VLA Model
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.10274
keywords	["vla","cross-embodiment","soft-prompting","robot-learning","parameter-efficiency"]
description	Use soft-prompted transformer architecture for multi-robot learning. Add learnable embodiment-specific prompt embeddings to handle different robot types while maintaining single shared backbone. Scale to 0.9B parameters across 6 simulators and 3 real robots.

X-VLA: Efficient Cross-Embodiment Vision-Language-Action Learning

Different robot embodiments have different action spaces, dynamics, and sensor configurations. X-VLA handles this diversity through soft prompting: each embodiment gets learnable prompt embeddings that condition the shared transformer, enabling one model to work across diverse robotic systems without separate parameters per embodiment.

Core insight: embodiment diversity is largely a prompt-engineering problem. By treating embodiment as learnable conditioning information rather than fundamentally different models, you maintain parameter efficiency while achieving state-of-the-art multi-robot performance.

Core Concept

Soft Prompting for Embodiments: Each data source/embodiment gets learnable embedding that conditions model behavior without parameter multiplication.

Shared Backbone Architecture: Single transformer with minimal added parameters handles all embodiments through prompt variation.

Architecture Overview

• Shared Vision Transformer: Encodes images consistently
• Embodiment Prompts: Learnable per-embodiment conditioning vectors
• Prompt Fusion: Integrate embodiment prompts with visual/language features
• Action Decoder: Generates robot-specific actions

Implementation Steps

Stage 1: Embodiment-Specific Soft Prompts

Create learnable embodiment embeddings:

import torch
import torch.nn as nn
from transformers import AutoModel, AutoTokenizer

class EmbodimentPromptedVLA(nn.Module):
    def __init__(
        self,
        backbone_model='google/vit-base-patch16-224',
        num_embodiments=3,
        prompt_dim=768,
        vocab_size=1024
    ):
        """
        Multi-embodiment VLA with soft prompts.
        """

        super().__init__()

        # Shared vision backbone
        self.vision_encoder = AutoModel.from_pretrained(
            backbone_model
        )

        
        .num_embodiments = num_embodiments
        .embodiment_prompts = nn.ParameterList([
            nn.Parameter(torch.randn(, , prompt_dim))
             _  (num_embodiments)
        ])

        
         param  .embodiment_prompts:
            nn.init.normal_(param, std=)

        
        .fusion_transformer = nn.TransformerEncoder(
            encoder_layer=nn.TransformerEncoderLayer(
                d_model=prompt_dim,
                nhead=,
                dim_feedforward=,
                dropout=,
                batch_first=
            ),
            num_layers=
        )

        
        .action_decoder = nn.Sequential(
            nn.Linear(prompt_dim, ),
            nn.ReLU(),
            nn.Linear(, vocab_size)
        )

     ():
        

        
        image_features = .vision_encoder.forward_features(
            images
        )  

        batch_size = image_features.shape[]

        
        embodiment_prompt = .embodiment_prompts[embodiment_idx]
        embodiment_prompt = embodiment_prompt.expand(
            batch_size,
            -,
            -
        )

        
        fused_input = torch.cat(
            [image_features, embodiment_prompt, language_embeddings],
            dim=
        )

        
        fused_features = .fusion_transformer(fused_input)

        
        embodiment_conditioned = fused_features[:, image_features.shape[], :]

        
        action_logits = .action_decoder(
            embodiment_conditioned
        )

         action_logits