lattice-3d-generation

name	lattice-3d-generation
title	LATTICE: Democratizing High-Fidelity 3D Generation at Scale via VoxSet
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.03052
keywords	["3d-generation","diffusion-transformers","voxel-representations","scalable-generation","test-time-scaling"]
description	Semi-structured latent representation combining efficiency of VecSet with spatial structure guidance via voxel queries and rotary positional embeddings, enabling strong test-time scaling (6K to 30K tokens) and improved model scaling without sparse components.

Summary

LATTICE introduces VoxSet, a semi-structured latent representation that combines the efficiency of VecSet methods with spatial structure guidance from coarse voxel grids. The approach uses voxel-anchored queries with rotary positional embeddings in diffusion transformers, enabling high-fidelity 3D generation with strong test-time scaling and pure transformer architecture.

Core Technique

VoxSet Representation: Hybrid latent space balancing efficiency and structure:

• Query Positions: Voxel grid centers (coarse structure)
• Learned Features: Per-voxel latent codes (detail)
• Positional Encoding: Rotary embeddings anchoring queries to spatial positions

Two-Stage Pipeline:

1. Coarse Generation: Voxelize existing 3D model to initialize voxel structure
2. Fine Generation: Produce detailed VoxSet latents within sparse voxel structure

Test-Time Scaling: Increase token count during inference from 6K to 30K, leveraging learned spatial structure to maintain coherence.

Implementation

Voxel grid initialization:

def create_voxel_grid(coarse_geometry, voxel_size=0.05):
    # Convert mesh to voxels
    voxels = voxelize(coarse_geometry, voxel_size)

    # Extract voxel centers (query positions)
    voxel_centers = voxel_positions(voxels)

    return voxel_centers  # [num_voxels, 3]

VoxSet representation:

class VoxSetLatent:
    def __init__(self, num_voxels, latent_dim=64):
        # Query positions (fixed at voxel centers)
        self.positions = nn.Parameter(voxel_centers)  # [num_voxels, 3]

        # Learnable feature codes per voxel
        self.codes = nn.Parameter(torch.randn(num_voxels, latent_dim))

     ():
        
        positions_embedded = rotary_embed(.positions)
        features = .codes * positions_embedded
         features