wan-move-motion-control

name	wan-move-motion-control
title	Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.08765
keywords	["video generation","motion control","trajectory guidance","latent space","image-to-video"]
description	Enable precise motion control in video generation using dense point trajectories as latent space features. Wan-Move integrates with existing models without architecture changes—ideal when you need fine-grained scene control without auxiliary motion encoders.

Overview

Wan-Move introduces a framework for controlling motion in video generation models through latent trajectory guidance. Rather than coarse-grained motion guidance, the method enables precise control using dense point trajectories propagated through latent space without requiring architectural modifications to base models.

When to Use

• Video generation with precise object motion control
• Integrating motion control into existing image-to-video models
• Scenarios requiring dense, fine-grained trajectory specification
• Applications avoiding additional motion encoder overhead
• Need for diverse, complex motion patterns beyond standard guidance

When NOT to Use

• Simple global camera movement (simpler approaches sufficient)
• Models where architectural modification is acceptable
• Scenarios not requiring dense point-level control
• Real-time generation with strict latency requirements

Core Technique

Latent space trajectory propagation for motion-aware guidance:

# Motion-controllable video generation via latent trajectories
class WanMoveController:
    def __init__(self, base_video_model):
        """
        Wrap existing video generation model with motion control.
        No architectural modifications to base model.
        """
        self.video_model = base_video_model
        self.vae = base_video_model.vae  # Access VAE for latent space

    def parse_dense_point_trajectories(self, motion_specification):
        """
        Dense point trajectory representation for granular scene control.
        Objects represented as sets of point trajectories rather than
        coarse bounding boxes or simple vectors.
        """
        trajectories = []

        for obj in motion_specification['objects']:
            # Each object gets multiple tracking points
            # e.g., 16-64 points distributed across object
            point_trajectory = {
                : obj[],
                : obj[],  
                : obj[],      
                : motion_specification[]
            }

            
            full_trajectory = .interpolate_point_trajectory(
                point_trajectory
            )

            trajectories.append(full_trajectory)

         trajectories

     ():
        
        start = point_trajectory[]
        end = point_trajectory[]
        num_frames = point_trajectory[]

        
        timesteps = torch.linspace(, , num_frames)

        full_trajectory = []
         t  timesteps:
            frame_positions = ( - t) * start + t * end
            full_trajectory.append(frame_positions)

         torch.stack(full_trajectory)  

     ():
        
        
        first_latent = .vae.encode(first_frame_image)

        
        latent_trajectories = []

         trajectory  trajectories:
            
            latent_traj = []

             frame_idx, frame_points  (trajectory):
                
                latent_points = .project_pixels_to_latent(
                    frame_points,
                    first_latent,
                    .video_model.latent_scale
                )
                latent_traj.append(latent_points)

            latent_trajectories.append(torch.stack(latent_traj))

         latent_trajectories

     ():
        
        batch_size, latent_h, latent_w, latent_c = first_frame_latent.shape
        num_frames = latent_trajectories[].shape[]

        
        guidance_map = torch.zeros(
            batch_size,
            num_frames,
            latent_h,
            latent_w,
            latent_c
        )

        
         traj_idx, latent_traj  (latent_trajectories):
            
            start_positions = latent_traj[]  

             point_idx, pos  (start_positions):
                
                h, w = (pos[]), (pos[])
                h = torch.clamp(h, , latent_h - )
                w = torch.clamp(w, , latent_w - )

                start_feature = first_frame_latent[:, h, w, :]

                
                 frame_idx, frame_pos  (latent_traj):
                    pos = frame_pos[point_idx]
                    h, w = (pos[]), (pos[])
                    h = torch.clamp(h, , latent_h - )
                    w = torch.clamp(w, , latent_w - )

                    
                    guidance_map[:, frame_idx, h, w, :] += start_feature

        
        guidance_map = guidance_map / ((latent_trajectories) + )

         guidance_map

     ():
        
        
        parsed_trajectories = .parse_dense_point_trajectories(trajectories)

        
        first_latent = .vae.encode(first_frame)
        latent_trajectories = .project_trajectories_to_latent_space(
            parsed_trajectories,
            first_frame
        )

        
        guidance_map = .propagate_latent_features_along_trajectories(
            first_latent,
            latent_trajectories
        )

        
        
        conditioned_input = torch.cat([
            first_latent,
            guidance_map
        ], dim=-)

        
        generated_latent = .video_model.generate(
            prompt=prompt,
            initial_latent=conditioned_input,
            num_frames=guidance_map.shape[]
        )

        
        generated_video = .vae.decode(generated_latent)

         generated_video

     ():
        
        benchmark_data = {
            : videos,
            : {
                : [],
                : [],
                : []
            }
        }

         benchmark_data