メニュー
Accelerate video diffusion generation by exploiting temporal attention sparsity—skip redundant attention tiles across denoising steps using persistent skip masks, achieving 40% speedup with quality retention.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/ADu2021/skillXiv --skill lite-attention-temporal-sparseコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
SOC 職業分類に基づく
SKILL.md を表示中
| name | lite-attention-temporal-sparse |
| title | LiteAttention: Temporal Sparse Attention for Diffusion Transformers |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2511.11062 |
| keywords | ["Sparse Attention","Diffusion","Video Generation","Temporal Coherence","Efficiency"] |
| description | Accelerate video diffusion generation by exploiting temporal attention sparsity—skip redundant attention tiles across denoising steps using persistent skip masks, achieving 40% speedup with quality retention. |
Video diffusion transformers spend significant computational budget computing self-attention across redundant spatial regions across denoising timesteps. LiteAttention exploits temporal coherence: tiles deemed non-essential at denoising step t remain non-essential at t+δ. By maintaining a persistent skip-mask and propagating skip decisions forward, LiteAttention reduces self-attention cost without dynamic recomputation at each step.
The key insight is that attention sparsity patterns are stable across adjacent denoising steps. Rather than recomputing which regions to skip (dynamic, expensive) or using fixed patterns throughout (inflexible), LiteAttention propagates early sparsity decisions, combining adaptivity with efficiency.
Video diffusion models iterate through T denoising steps, each computing full self-attention on spatial tiles (flattened image regions). Most tiles have negligible attention weights across most steps—they can be safely skipped. However, dynamically identifying skippable tiles at every step multiplies computational overhead.
LiteAttention maintains a Skip-Mask tracking which tiles are non-essential. The mask is computed once early (step 0 or 1) using QK-Skip: a simple condition on query-key dot products. Tiles with max local QK score significantly smaller than cumulative maximum are flagged as skippable. This mask propagates across timesteps, and only if attention patterns diverge significantly does the mask update.
Step 1: Sparsity Detection. Identify tiles with negligible QK scores early in denoising.
def compute_qk_skip_mask(queries, keys, threshold_ratio=0.1):
"""
Compute skip mask using QK-Skip condition.
Tiles with max QK << cumulative_max are marked as skippable.
queries, keys: (batch, num_tiles, head_dim)
threshold_ratio: tile is skipped if max_qk < threshold_ratio * cumulative_max
"""
qk_scores = queries @ keys.transpose(-, -)
skip_mask = []
b (qk_scores.shape[]):
local_maxs = qk_scores[b].(dim=-).values
cumulative_max = local_maxs.()
skip_threshold = threshold_ratio * cumulative_max
skippable = local_maxs < skip_threshold
skip_mask.append(skippable)
torch.stack(skip_mask)
Step 2: Mask Propagation. Maintain skip mask across denoising steps; update selectively.
class TemporalSkipAttention(nn.Module):
def __init__(self, num_tiles, update_frequency=5):
super().__init__()
self.num_tiles = num_tiles
self.skip_mask = None
self.update_frequency = update_frequency # recompute every N steps
def forward(self, queries, keys, values, timestep):
"""
Compute attention with persistent skip mask.
timestep: current denoising step (0 to T)
"""
# Recompute mask periodically or at first step
if self.skip_mask is None or timestep % self.update_frequency == 0:
self.skip_mask = compute_qk_skip_mask(queries, keys)
# Compute full attention scores
qk = queries @ keys.transpose(-1, -2)
qk = qk / math.sqrt(queries.shape[-1])
# Apply skip mask: set masked positions to -inf
qk[self.skip_mask] = float('-inf')
# Softmax and output
attn_weights = torch.softmax(qk, dim=-1)
attn_weights[self.skip_mask] = 0 # zero-out masked attention
output = attn_weights @ values
return output
Step 3: Error Calibration. Assign variable tolerance to timesteps based on impact on final output.
def calibrate_error_budgets(num_timesteps, early_step_ratio=0.3):
"""
Early denoising steps affect final output more; allocate smaller error budgets there.
Returns per-timestep error tolerance multiplier.
"""
error_budgets = []
for t in range(num_timesteps):
# Earlier steps get tighter error budgets (lower multiplier = smaller tolerance)
if t < int(num_timesteps * early_step_ratio):
# Scale from 0.5 to 1.0 across early steps
budget = 0.5 + 0.5 * (t / int(num_timesteps * early_step_ratio))
else:
# Later steps tolerate more error
budget = 1.0
error_budgets.append(budget)
return error_budgets
When to Use: Video diffusion generation where temporal consistency and computational cost matter. Works particularly well for hour-long videos or high-resolution frames where attention is prohibitive.
Hyperparameters:
Pitfalls:
When NOT to Use: Single-image generation (T=1, no temporal redundancy to exploit); inherently sparse-attention designs (local attention) already benefit from sparsity.
Integration: Compatible with any diffusion transformer; integrates into softmax pipeline without architecture changes. Pairs well with hierarchical attention and token merging for additional speedup.
Reference: https://arxiv.org/abs/2511.11062