2xplat-two-expert-3dgs

name	2xplat-two-expert-3dgs
title	2Xplat: Two-Expert Framework for Pose-Free 3D Gaussian Splatting
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.21064
keywords	["3D Gaussian Splatting","Pose Estimation","Modular Architecture","Geometry-Appearance Decomposition"]
description	Replace monolithic 3D Gaussian Splatting with two-expert architecture separating geometry (pose) estimation from appearance synthesis. Converges 30× faster (5K vs 150K iterations) while matching pose-dependent methods. Works best for multi-view reconstruction when geometry and appearance have conflicting optimization dynamics. Trigger: When doing feed-forward 3DGS and need faster convergence without sacrificing quality.
category	Component Innovation

What This Skill Does

Swap a monolithic 3D Gaussian Splatting architecture with a modular two-expert framework that explicitly separates geometry estimation (camera pose prediction) from appearance synthesis (Gaussian generation). Achieves 30× faster convergence while maintaining quality comparable to state-of-the-art pose-dependent methods.

Problem with Monolithic 3D Gaussian Splatting

Monolithic architectures jointly optimize all parameters (camera poses, Gaussian positions, colors, covariances) in a single network. This creates inherent conflicts:

• Strict geometric accuracy (tight pose constraints) can degrade visual appearance (fewer degrees of freedom for colors)
• Unified representations cannot leverage specialized architectures proven effective for each sub-task
• Large parameter space (poses + gaussians) requires extensive training (150K+ iterations on 16 GH200 GPUs)
• Dense architecture cannot reuse pretrained geometric or appearance priors

The paper's insight: Decompose into specialized experts. Geometry expert predicts precise camera poses; appearance expert (already trained on multi-view synthesis) generates Gaussians conditioned on known poses. This enables reuse of pretrained models and faster convergence.

The Swap: Monolithic → Two-Expert Decomposition

Replace unified architecture with sequential expert pipeline:

# Monolithic approach (baseline)
class MonolithicGaussianSplatter(nn.Module):
    """Single network jointly predicts poses + Gaussians"""
    def __init__(self):
        self.image_encoder = Encoder()
        self.pose_head = nn.Linear(feat_dim, 6)      # Predict 6-DoF poses
        self.gaussian_head = nn.Linear(feat_dim, 14*N_gaussians)  # Predict positions, colors, covariance

    def forward(self, images):
        features = self.image_encoder(images)
        poses = self.pose_head(features)       # Joint optimization
        gaussians = .gaussian_head(features)
         poses, gaussians


 (nn.Module):
    
     ():
        
        .geometry_expert = DepthAnything3()  

        
        .appearance_expert = MultiViewPyramidTransformer()  

     ():
        
        depth_maps = .geometry_expert(images)
        poses = depth_to_poses(depth_maps)  

        
        gaussians = .appearance_expert(images, poses=poses)  

        
         poses, gaussians


 ():
    render_loss = l2_loss(rendered, target)
    pose_supervision = mse_loss(poses, poses_target)  
     render_loss +  * pose_supervision