メニュー
Accelerate video diffusion transformer inference by 1.58-1.85× through discovering and exploiting sparse attention patterns that exhibit diagonal, multi-diagonal, and vertical-stripe structures.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
SOC 職業分類に基づく
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/ADu2021/skillXiv --skill sparse-vdit-video-accelerationコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SKILL.md を表示中
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
| name | sparse-vdit-video-acceleration |
| title | Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2506.03065 |
| keywords | ["video-diffusion","sparse-attention","acceleration","efficiency","inference-optimization"] |
| description | Accelerate video diffusion transformer inference by 1.58-1.85× through discovering and exploiting sparse attention patterns that exhibit diagonal, multi-diagonal, and vertical-stripe structures. |
Sparse-vDiT identifies a critical inefficiency in video diffusion transformers: attention mechanisms consume 77-81% of inference latency, yet their attention maps exhibit predictable sparse patterns (diagonal, multi-diagonal, vertical-stripe structures) that are largely independent of input content. Rather than computing dense attention, the framework systematically discovers and exploits these patterns per layer and head, achieving 1.58-1.85× actual speedup with minimal quality degradation.
The key insight: attention patterns correlate strongly with layer depth, not input variations, enabling offline optimization that generalizes across different inputs.
# Sparse pattern evaluation
def offline_sparse_search(layer_module, validation_data):
"""
For each layer and head, find optimal sparse attention pattern.
Searches five modes: dense, diagonal, multi-diagonal, vertical, mixed.
"""
modes = ['dense', 'diagonal', 'multi_diagonal', 'vertical_stripe', 'mixed']
best_patterns = {}
for head_idx in range(num_heads):
mode_scores = {}
for mode in modes:
sparse_attention = apply_pattern(layer_module, head_idx, mode)
output = compute_with_sparse_attention(sparse_attention, validation_data)
mse_loss = mean_squared_error(output, dense_output)
sparsity_penalty = lambda_param * compute_sparsity(sparse_attention)
mode_scores[mode] = mse_loss + sparsity_penalty
best_mode = (mode_scores, key=mode_scores.get)
best_patterns[head_idx] = best_mode
best_patterns
# Sparse attention patterns
def create_sparse_mask(seq_len, mode):
"""Define attention patterns with different sparsity structures."""
mask = torch.ones(seq_len, seq_len)
if mode == 'diagonal':
# Only attend to positions within local window
for i in range(seq_len):
for j in range(seq_len):
if abs(i - j) > window_size:
mask[i, j] = 0
elif mode == 'multi_diagonal':
# Multiple diagonal bands for increased expressivity
for band_offset in band_offsets:
for i in range(seq_len):
j = i + band_offset
if 0 <= j < seq_len:
mask[i, j] = 1
elif mode == 'vertical_stripe':
# Attend to specific column positions (useful for spatial patterns)
for i in range(seq_len):
for stripe_pos in stripe_positions:
mask[i, stripe_pos] = 1
elif mode == 'mixed':
# Combination of patterns for flexibility
mask = combine_patterns([diagonal, multi_diagonal, vertical])
return mask
# CUDA kernel invocation (pseudo-code)
def forward_sparse_attention(query, key, value, sparse_pattern, mode):
"""
Execute sparse attention using hardware-optimized kernels.
Pattern determines kernel selection.
"""
if mode == 'diagonal':
# Use diagonal band CUDA kernel
output = cuda_diagonal_attention(query, key, value, window_size)
elif mode == 'vertical_stripe':
# Use vertical stripe kernel with pre-computed stripe positions
output = cuda_stripe_attention(query, key, value, stripe_indices)
else:
# Fall back to generic sparse kernel
output = cuda_sparse_attention(query, key, value, sparse_pattern)
return output
def fuse_attention_heads(layer_module, patterns):
"""
Identify heads with identical sparse patterns and fuse computation.
Reduces kernel invocation overhead.
"""
pattern_groups = {}
for head_idx, pattern in patterns.items():
if pattern not in pattern_groups:
pattern_groups[pattern] = []
pattern_groups[pattern].append(head_idx)
fused_heads = {}
for pattern, head_indices in pattern_groups.items():
if len(head_indices) > 1:
# Fuse multiple heads into single computation
fused_heads[pattern] = head_indices
return fused_heads
When to Apply:
Implementation Prerequisites:
Performance Expectations:
Key Tuning Decisions:
Common Issues:
Demonstrated on CogVideoX1.5, HunyuanVideo, and Wan2.1. Achieves 2.09-2.38× theoretical speedup with 1.58-1.85× actual speedup through pattern-optimized sparse kernels and head fusion. Quality maintained across standard video quality metrics.