instruct-particulate-3d-articulation

name	instruct-particulate-3d-articulation
description	Instruct-Particulate methodology for feed-forward 3D object articulation reconstruction using kinematic control. Enables scalable recovery of articulated 3D structures from single images or multi-view inputs. Use when: 3D articulation recovery, feed-forward 3D reconstruction, kinematic control for 3D objects, articulated object modeling.
metadata	{"arxiv_id":"2606.14699","published":"2026-06-14","tags":["3d-vision","articulated-objects","kinematic-control","feed-forward-reconstruction","computer-graphics","robotics"]}

Instruct-Particulate: Feed-Forward 3D Articulation

Description

Feed-forward neural framework for reconstructing articulated 3D objects from images using kinematic control. Addresses the generalization gap in articulated 3D reconstruction by learning a scalable, instruction-conditioned model that predicts part-level kinematic structure. arXiv: 2606.14699

Activation Keywords

• 3d articulation recovery
• articulated object reconstruction
• feed-forward 3d reconstruction
• kinematic control 3d
• 3d object parts
• articulated mesh recovery
• 三维铰接对象重建

Core Concepts

Problem

Reconstructing articulated 3D objects (chairs with moving legs, laptops with hinges, etc.) from images requires understanding both the static geometry and the kinematic relationships between parts. Prior methods either:

1. Use optimization-based approaches (slow, per-instance)
2. Use neural networks that don't generalize well to novel object categories

Key Innovation

Instruct-Particulate uses a feed-forward neural network that:

• Takes an image (or multi-view images) as input
• Predicts the full articulated 3D structure: parts, joints, kinematic hierarchy
• Is instruction-conditioned: can be guided to recover specific articulation patterns
• Scales to diverse object categories through large-scale training

Architecture

Input Image(s) → Feature Encoder → Part Segmentation Head
                                    → Kinematic Parameter Head
                                    → 3D Geometry Head (per part)
                                    → Joint/Articulation Parameter Head
                                    → Assembled Articulated 3D Model

The model produces a complete articulated 3D representation including:

• Part decomposition: Semantic segmentation of object into rigid parts
• Kinematic parameters: Joint types (revolute, prismatic, fixed) and axes
• Per-part 3D geometry: Mesh or SDF representation for each rigid part
• Articulation hierarchy: Parent-child relationships between parts

Implementation Pattern

# Conceptual architecture
class InstructParticulate(nn.Module):
    """Feed-forward articulated 3D reconstruction."""
    
    def __init__(self, num_parts, instruction_dim):
        self.encoder = ImageEncoder()  # e.g., DINOv2 or ViT
        self.instruction_encoder = TextEncoder()  # instruction conditioning
        
        self.part_head = PartSegmentationHead(num_parts)
        self.kinematic_head = KinematicParameterHead()  # joint types, axes
        self.geometry_head = PerPartGeometryHead(num_parts)  # 3D geometry per part
        self.hierarchy_head = ArticulationHierarchyHead()  # parent-child links
    
    def forward(self, images, instructions=None):
        features = self.encoder(images)
        if instructions is not None:
            instr_embed = self.instruction_encoder(instructions)
            features = features + instr_embed
        
        parts = self.part_head(features)
        kinematics = self.kinematic_head(features)
        geometries = self.geometry_head(features)
        hierarchy = self.hierarchy_head(features)
        
        return ArticulatedModel(parts, kinematics, geometries, hierarchy)

Application Patterns

Pattern 1: Single-Image Articulation Recovery

Use for recovering articulated 3D structure from a single input image. The feed-forward nature means no per-instance optimization is needed.

Pattern 2: Multi-View Articulation Recovery

With multiple input views, the model produces more accurate 3D geometry and joint axis estimation.

Pattern 3: Instruction-Conditioned Recovery

Guide the model with text instructions (e.g., "recover the chair's leg articulation") to focus on specific articulation patterns.

When to Use

• 3D object reconstruction where parts move relative to each other
• Animation and gaming pipelines needing articulated 3D assets
• Robotic simulation requiring kinematic models of objects
• Computer graphics for automatic rigging of 3D models
• AR/VR applications needing interactive articulated objects

Pitfalls

• Generalization to unseen categories: Feed-forward models trained on specific categories may struggle with novel articulation types
• Joint type ambiguity: Distinguishing between similar joint types (e.g., revolute vs. prismatic) from static images is inherently ambiguous
• Occlusion: Heavily occluded objects make part-level reconstruction unreliable
• Instruction quality: Instruction-conditioning requires well-formed instructions to be effective

References

• arXiv: 2606.14699 - "Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control"
• Related: neural 3d reconstruction, articulated object modeling, kinematic estimation