simart-articulated-asset-decomposition

name	simart-articulated-asset-decomposition
title	SIMART: Unified MLLM for Articulated Asset Decomposition via Sparse 3D VQ-VAE
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.23386
keywords	["3D VQ-VAE","Mesh Decomposition","Token Efficiency","Sparse Tokenization","MLLMs"]
description	Replace dense voxel tokenization with sparse 3D VQ-VAE to reduce token counts by 70% in multimodal 3D understanding. Enables efficient articulated asset decomposition for physics-based simulation. Works best for 3D generation tasks constrained by token budget. Trigger: When working with 3D mesh models and hitting token limits; want to decompose meshes into articulated parts with reduced memory footprint.
category	Component Innovation

What This Skill Does

Swap dense voxel-based 3D tokenization with sparse 3D Vector Quantized Variational Autoencoder (VQ-VAE) to achieve 70% reduction in token count while preserving mesh articulation quality for multimodal 3D understanding models.

Problem with Dense Voxel Tokenization

Standard 3D tokenization (voxel grids, point clouds) generates long token sequences that consume excessive memory. Dense representations encode every spatial position uniformly, creating redundant sequences where:

• Empty/sparse regions waste tokens
• Long sequences make complex articulated objects intractable for MLLMs
• High token count forces shorter context windows or model size reductions

The paper's insight: 3D shapes are inherently sparse (most of space is empty). Sparse tokenization that only encodes occupied regions reduces sequence length dramatically while preserving articulation structure.

The Swap: Dense Voxels → Sparse 3D VQ-VAE

Replace uniform voxel grids with learned sparse quantization:

# Dense voxel tokenization (baseline)
def dense_voxel_tokenize(mesh, grid_size=64):
    """Encode mesh on uniform voxel grid"""
    voxel_grid = mesh_to_voxels(mesh, resolution=grid_size)  # 64³ = 262,144 positions
    # All positions encoded, even empty ones
    tokens = voxel_grid.flatten()  # ~260K tokens for modest resolution
    return tokens

# Sparse 3D VQ-VAE (proposed)
def sparse_vq_vae_tokenize(mesh, latent_dim=256, num_codes=1024):
    """
    Encode only occupied regions via learned sparse representation.
    Key: VQ-VAE learns to compress sparse geometry into discrete codes.
    """
    # 1. Extract sparse geometry (octree or list of occupied voxels)
    occupied_voxels = mesh_to_sparse_voxels(mesh)  # Only ~5-10% of grid

    # 2. Encode occupied regions through learned VAE encoder
    z = vae_encoder(occupied_voxels)  # Continuous latent

    # 3. Quantize to nearest codebook vector (discrete)
    z_quantized, indices = vq_layer.encode(z)  

    
     indices