| name | seacache-spectral-evolution-diffusion |
| title | SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2602.18993 |
| keywords | ["diffusion models","caching","acceleration","frequency domain","training-free"] |
| description | Accelerate diffusion models through spectral-evolution-aware caching. Exploit insight that early timesteps establish low-frequency structure while later timesteps refine high-frequency details. Apply FFT-based frequency filtering to feature cache decisions: preserve content-relevant frequency components while suppressing noise. Plug-and-play training-free enhancement achieving 1.5–2.5× speedup across FLUX, HunyuanVideo, Wan2.1 models. |
SeaCache: Frequency-Aware Feature Reuse in Diffusion Models
Diffusion models generate high-quality images through iterative refinement: starting from noise, they gradually add structure across timesteps. However, each timestep requires expensive forward passes through large models. Naively, every step must compute all features, but adjacent timesteps often share similar feature patterns—if properly measured, cache decisions can reuse features from nearby timesteps.
The challenge is determining when features are similar "enough" to cache. Pixel-level differences are noisy (stochastic variations don't indicate meaningful changes); frequency-domain similarity is more stable. Early timesteps focus on low-frequency structure (edges, composition); late timesteps add high-frequency detail (texture, fine structure).
Core Concept
SeaCache leverages this frequency progression: early timesteps emphasize low-frequency components; late timesteps progressively incorporate higher frequencies. Rather than comparing raw features (noise-sensitive), the method:
- Applies timestep-dependent frequency filters via FFT
- Compares filtered features to identify meaningful similarity
- Caches when features are similar in content-relevant frequency bands
The approach is training-free, plug-and-play, and applicable to any diffusion model.
Architecture Overview
- Spectral Evolution Analyzer: Characterize frequency content across timesteps using FFT
- SEA Filter Designer: Compute optimal frequency response for each timestep (emphasize low freq early, high freq late)
- Feature Transformer: Apply frequency filtering to features before similarity comparison
- Cache Decision: Compare filtered features; cache if distance below threshold
- Runtime Integration: Intercept feature computation, check cache with SEA filtering
Implementation
Analyze spectral evolution in diffusion to design optimal filters:
import numpy as np
from scipy import fft
def analyze_spectral_evolution(noisy_trajectory, clean_image):
"""
Analyze how frequency content changes across diffusion timesteps.
noisy_trajectory: list of (B, C, H, W) tensors across timesteps
clean_image: target (B, C, H, W) tensor
Returns: frequency response per timestep
"""
num_timesteps = (noisy_trajectory)
frequency_responses = []
t (num_timesteps):
noisy = noisy_trajectory[t].detach().cpu().numpy()
residual = clean_image.numpy() - noisy
freq_domain = np.(fft.fft2(residual, axes=(, )))
freq_magnitude = freq_domain.mean(axis=(, ))
freq_radial = []
h, w = freq_magnitude.shape
r ((h, w) // ):
y, x = np.ogrid[:h, :w]
mask = (x - w//)** + (y - h//)** <= (r+)**
mask = mask & ((x - w//)** + (y - h//)** > r**)
mask.() > :
freq_radial.append(freq_magnitude[mask].mean())
frequency_responses.append(freq_radial)
np.array(frequency_responses)
():
num_timesteps = frequency_responses.shape[]
filters = np.zeros((num_timesteps, num_frequencies))
t (num_timesteps):
normalized_t = t / (, num_timesteps - )
freq_idx (num_frequencies):
normalized_freq = freq_idx / num_frequencies
cutoff = + normalized_t *
response = np.exp(-((normalized_freq - cutoff)**) / ( * **))
filters[t, freq_idx] = response
filters
