菜单
Accelerate video diffusion generation by exploiting temporal attention sparsity—skip redundant attention tiles across denoising steps using persistent skip masks, achieving 40% speedup with quality retention.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill lite-attention-temporal-sparse命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
基于 SOC 职业分类
正在显示 SKILL.md
| name | lite-attention-temporal-sparse |
| title | LiteAttention: Temporal Sparse Attention for Diffusion Transformers |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2511.11062 |
| keywords | ["Sparse Attention","Diffusion","Video Generation","Temporal Coherence","Efficiency"] |
| description | Accelerate video diffusion generation by exploiting temporal attention sparsity—skip redundant attention tiles across denoising steps using persistent skip masks, achieving 40% speedup with quality retention. |
Video diffusion transformers spend significant computational budget computing self-attention across redundant spatial regions across denoising timesteps. LiteAttention exploits temporal coherence: tiles deemed non-essential at denoising step t remain non-essential at t+δ. By maintaining a persistent skip-mask and propagating skip decisions forward, LiteAttention reduces self-attention cost without dynamic recomputation at each step.
The key insight is that attention sparsity patterns are stable across adjacent denoising steps. Rather than recomputing which regions to skip (dynamic, expensive) or using fixed patterns throughout (inflexible), LiteAttention propagates early sparsity decisions, combining adaptivity with efficiency.
Video diffusion models iterate through T denoising steps, each computing full self-attention on spatial tiles (flattened image regions). Most tiles have negligible attention weights across most steps—they can be safely skipped. However, dynamically identifying skippable tiles at every step multiplies computational overhead.
LiteAttention maintains a Skip-Mask tracking which tiles are non-essential. The mask is computed once early (step 0 or 1) using QK-Skip: a simple condition on query-key dot products. Tiles with max local QK score significantly smaller than cumulative maximum are flagged as skippable. This mask propagates across timesteps, and only if attention patterns diverge significantly does the mask update.
Step 1: Sparsity Detection. Identify tiles with negligible QK scores early in denoising.
def compute_qk_skip_mask(queries, keys, threshold_ratio=0.1):
"""
Compute skip mask using QK-Skip condition.
Tiles with max QK << cumulative_max are marked as skippable.
queries, keys: (batch, num_tiles, head_dim)
threshold_ratio: tile is skipped if max_qk < threshold_ratio * cumulative_max
"""
qk_scores = queries @ keys.transpose(-, -)
skip_mask = []
b (qk_scores.shape[]):
local_maxs = qk_scores[b].(dim=-).values
cumulative_max = local_maxs.()
skip_threshold = threshold_ratio * cumulative_max
skippable = local_maxs < skip_threshold
skip_mask.append(skippable)
torch.stack(skip_mask)
Step 2: Mask Propagation. Maintain skip mask across denoising steps; update selectively.
class TemporalSkipAttention(nn.Module):
def __init__(self, num_tiles, update_frequency=5):
super().__init__()
self.num_tiles = num_tiles
self.skip_mask = None
self.update_frequency = update_frequency # recompute every N steps
def forward(self, queries, keys, values, timestep):
"""
Compute attention with persistent skip mask.
timestep: current denoising step (0 to T)
"""
# Recompute mask periodically or at first step
if self.skip_mask is None or timestep % self.update_frequency == 0:
self.skip_mask = compute_qk_skip_mask(queries, keys)
# Compute full attention scores
qk = queries @ keys.transpose(-1, -2)
qk = qk / math.sqrt(queries.shape[-1])
# Apply skip mask: set masked positions to -inf
qk[self.skip_mask] = float('-inf')
# Softmax and output
attn_weights = torch.softmax(qk, dim=-1)
attn_weights[self.skip_mask] = 0 # zero-out masked attention
output = attn_weights @ values
return output
Step 3: Error Calibration. Assign variable tolerance to timesteps based on impact on final output.
def calibrate_error_budgets(num_timesteps, early_step_ratio=0.3):
"""
Early denoising steps affect final output more; allocate smaller error budgets there.
Returns per-timestep error tolerance multiplier.
"""
error_budgets = []
for t in range(num_timesteps):
# Earlier steps get tighter error budgets (lower multiplier = smaller tolerance)
if t < int(num_timesteps * early_step_ratio):
# Scale from 0.5 to 1.0 across early steps
budget = 0.5 + 0.5 * (t / int(num_timesteps * early_step_ratio))
else:
# Later steps tolerate more error
budget = 1.0
error_budgets.append(budget)
return error_budgets
When to Use: Video diffusion generation where temporal consistency and computational cost matter. Works particularly well for hour-long videos or high-resolution frames where attention is prohibitive.
Hyperparameters:
Pitfalls:
When NOT to Use: Single-image generation (T=1, no temporal redundancy to exploit); inherently sparse-attention designs (local attention) already benefit from sparsity.
Integration: Compatible with any diffusion transformer; integrates into softmax pipeline without architecture changes. Pairs well with hierarchical attention and token merging for additional speedup.
Reference: https://arxiv.org/abs/2511.11062