seacache-spectral-evolution-diffusion

name	seacache-spectral-evolution-diffusion
title	SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.18993
keywords	["diffusion models","caching","acceleration","frequency domain","training-free"]
description	Accelerate diffusion models through spectral-evolution-aware caching. Exploit insight that early timesteps establish low-frequency structure while later timesteps refine high-frequency details. Apply FFT-based frequency filtering to feature cache decisions: preserve content-relevant frequency components while suppressing noise. Plug-and-play training-free enhancement achieving 1.5–2.5× speedup across FLUX, HunyuanVideo, Wan2.1 models.

SeaCache: Frequency-Aware Feature Reuse in Diffusion Models

Diffusion models generate high-quality images through iterative refinement: starting from noise, they gradually add structure across timesteps. However, each timestep requires expensive forward passes through large models. Naively, every step must compute all features, but adjacent timesteps often share similar feature patterns—if properly measured, cache decisions can reuse features from nearby timesteps.

The challenge is determining when features are similar "enough" to cache. Pixel-level differences are noisy (stochastic variations don't indicate meaningful changes); frequency-domain similarity is more stable. Early timesteps focus on low-frequency structure (edges, composition); late timesteps add high-frequency detail (texture, fine structure).

Core Concept

SeaCache leverages this frequency progression: early timesteps emphasize low-frequency components; late timesteps progressively incorporate higher frequencies. Rather than comparing raw features (noise-sensitive), the method:

1. Applies timestep-dependent frequency filters via FFT
2. Compares filtered features to identify meaningful similarity
3. Caches when features are similar in content-relevant frequency bands

The approach is training-free, plug-and-play, and applicable to any diffusion model.

Architecture Overview

• Spectral Evolution Analyzer: Characterize frequency content across timesteps using FFT
• SEA Filter Designer: Compute optimal frequency response for each timestep (emphasize low freq early, high freq late)
• Feature Transformer: Apply frequency filtering to features before similarity comparison
• Cache Decision: Compare filtered features; cache if distance below threshold
• Runtime Integration: Intercept feature computation, check cache with SEA filtering

Implementation

Analyze spectral evolution in diffusion to design optimal filters:

import numpy as np
from scipy import fft

def analyze_spectral_evolution(noisy_trajectory, clean_image):
    """
    Analyze how frequency content changes across diffusion timesteps.
    noisy_trajectory: list of (B, C, H, W) tensors across timesteps
    clean_image: target (B, C, H, W) tensor
    Returns: frequency response per timestep
    """
    num_timesteps = (noisy_trajectory)
    frequency_responses = []

     t  (num_timesteps):
        noisy = noisy_trajectory[t].detach().cpu().numpy()
        residual = clean_image.numpy() - noisy

        
        freq_domain = np.(fft.fft2(residual, axes=(, )))

        
        freq_magnitude = freq_domain.mean(axis=(, ))

        
        freq_radial = []
        h, w = freq_magnitude.shape
         r  ((h, w) // ):
            y, x = np.ogrid[:h, :w]
            mask = (x - w//)** + (y - h//)** <= (r+)**
            mask = mask & ((x - w//)** + (y - h//)** > r**)
             mask.() > :
                freq_radial.append(freq_magnitude[mask].mean())

        frequency_responses.append(freq_radial)

     np.array(frequency_responses)

 ():
    
    num_timesteps = frequency_responses.shape[]
    filters = np.zeros((num_timesteps, num_frequencies))

     t  (num_timesteps):
        
        normalized_t = t / (, num_timesteps - )

        
        
         freq_idx  (num_frequencies):
            
            normalized_freq = freq_idx / num_frequencies

            
            cutoff =  + normalized_t * 

            
            response = np.exp(-((normalized_freq - cutoff)**) / ( * **))
            filters[t, freq_idx] = response

     filters