flowblending-video-inference

name	flowblending-video-inference
title	FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.24724
keywords	["video generation","inference optimization","multi-model sampling","diffusion timesteps","computational efficiency","model scaling"]
description	Accelerate video generation by allocating smaller models to intermediate diffusion timesteps and larger models to capacity-critical early and late stages. Achieves 1.65x speedup and 57% FLOP reduction while maintaining visual quality. Use when video generation latency or computational cost is critical and you have multiple model sizes available.

When to Use This Skill

• Video generation inference where latency is critical (streaming, interactive applications)
• Scenarios with strict computational budgets (edge devices, cloud costs)
• Deployments with multiple model checkpoints (small, base, large)
• Batch processing where throughput matters more than per-sample latency
• Quality-conscious workflows where efficiency shouldn't hurt visual results

When NOT to Use This Skill

• Single-model deployments without size variants
• Real-time generation where step count itself is the bottleneck (use faster diffusion instead)
• Situations requiring deterministic, reproducible results per timestep
• Models where architecture significantly changes across sizes

Core Insight

Video diffusion models operate over many timesteps (typically 30-100). But not all timesteps are equal:

Diffusion timeline (noise → clean):
t=100 ──→ t=50 ──→ t=25 ──→ t=1

Stage mapping:
Early (t=100-75):    Capacity CRITICAL (removing large-scale corruption)
Middle (t=75-25):    Capacity NEGLIGIBLE (fine-tuning already-good samples)
Late (t=25-1):       Capacity CRITICAL (detail synthesis, temporal coherence)

Early and late stages solve hard problems (large-scale structure, fine details). Middle stages just refine—a small model does fine here.

Velocity-Divergence Analysis

The paper identifies capacity needs using velocity-divergence—a measure of how much the predicted velocity field changes in spatial regions:

• High divergence: Regions where predictions vary greatly across model capacities
  • Early stages: Large objects entering/leaving → divergence high
  • Late stages: Fine details → divergence high
• Low divergence: Predictions stable regardless of capacity
  • Middle stages: Content mostly determined, just denoising noise → divergence low

This analysis informs which stages can safely use smaller models.

Architecture Pattern

# FlowBlending multi-model inference strategy
class FlowBlendingVideoGenerator:
    def __init__(self, model_small, model_base, model_large, config):
        self.models = {
            'small': model_small,    
            : model_base,      
            : model_large     
        }
        .config = config

     ():
        
        x_t = torch.randn(batch_size, channels, height, width)

         step  (num_steps):
            
            stage = .identify_stage(step, num_steps)
            model = .select_model_for_stage(stage)

            
            noise_pred = model.predict_noise(
                x_t,
                timestep=step,
                prompt_embedding=.encode_prompt(prompt)
            )

            
            x_t = .diffusion_step(x_t, noise_pred, step)

         x_t

     ():
        
        progress = current_step / total_steps
         progress < :  
             
         progress < :  
             
        :  
             

     ():
        
         stage == :
             .models[]   
         stage == :
             .models[]   
        :  
             .models[]   

     ():
        
        alpha = .get_alpha(step)
        alpha_prev = .get_alpha(step - )
        sigma = .get_sigma(step)

        
        x_0_pred = (x_t - torch.sqrt( - alpha) * noise_pred) / torch.sqrt(alpha)

        
        x_prev = (torch.sqrt(alpha_prev) * x_0_pred +
                 torch.sqrt( - alpha_prev) * noise_pred)
         x_prev