gimbaldiffusion-camera-control

name	gimbaldiffusion-camera-control
title	GimbalDiffusion: Gravity-Aware Camera Control for Video Generation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.09112
keywords	["video generation","camera control","gravity-aware coordinates","text-to-video","spatial control"]
description	Control video camera motion using gravity-aligned absolute coordinates instead of relative trajectories. GimbalDiffusion enables precise camera control with null-pitch conditioning—ideal when you need interpretable, physics-aware camera motion in text-to-video.

Overview

GimbalDiffusion enables precise camera control in text-to-video generation by using gravity as a global reference point. The absolute coordinate system eliminates the need for relative frame references while null-pitch conditioning prevents conflicting text instructions.

When to Use

• Text-to-video with precise camera control
• Panoramic video generation from 360-degree data
• Scenarios requiring wide camera pitch variation
• Need for interpretable camera motion
• Avoiding relative trajectory ambiguity

When NOT to Use

• Simple camera-static videos
• Scenarios without camera control requirements
• Applications with limited training data

Core Technique

Gravity-aware camera coordinate system:

# GimbalDiffusion: Gravity-aware camera control
class GravityAwareCameraControl:
    def __init__(self):
        self.diffusion = DiffusionModel()

    def condition_on_camera_trajectory(self, camera_params):
        """
        Camera parameters in gravity-aligned coordinate system.
        pitch, yaw, roll relative to gravity vector (down).
        """
        # Extract camera parameters
        pitch = camera_params['pitch']  # Up/down (-90 to 90)
        yaw = camera_params['yaw']      # Left/right (0-360)
        roll = camera_params['roll']    # Rotation (0-360)

        # Gravity-aligned embedding
        gravity_vector = torch.tensor([0, -1, 0])  # Down

        # Create camera embedding from angles
        camera_embedding = self.encode_camera_angles(
            pitch, yaw, roll, gravity_vector
        )

        return camera_embedding

     ():
        
        
        scene_orientation = .analyze_text(text_instruction)

        
         .is_conflicting(camera_pitch, scene_orientation):
            
            
            camera_embedding = .apply_null_pitch_mask(
                camera_pitch
            )
        :
            camera_embedding = .condition_on_camera_trajectory(
                {: camera_pitch, : , : }
            )

         camera_embedding

     ():
        
        
        frames = []

         frame_idx, camera_params  (camera_trajectory):
            
            camera_cond = .apply_nullpitch_conditioning(
                prompt, camera_params[]
            )

            
            frame = .diffusion.sample(
                prompt=prompt,
                camera_conditioning=camera_cond
            )

            frames.append(frame)

         torch.stack(frames)

     ():
        
        
        rotation_matrix = .euler_to_rotation(pitch, yaw, roll)

        
        embedding = .rotate_embedding(gravity, rotation_matrix)

         embedding