メニュー
Build hybrid Mamba-Transformer models combining efficient Mamba-2 layers with standard attention to achieve 6x higher inference throughput while maintaining reasoning accuracy on long-context tasks.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/ADu2021/skillXiv --skill nemotron-nano-hybrid-mamba-transformerコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
SOC 職業分類に基づく
SKILL.md を表示中
| name | nemotron-nano-hybrid-mamba-transformer |
| title | Nemotron Nano 2: Hybrid Mamba-Transformer Architecture for Efficient Reasoning |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2508.14444 |
| keywords | ["hybrid-architecture","mamba","transformer","model-compression","reasoning","throughput"] |
| description | Build hybrid Mamba-Transformer models combining efficient Mamba-2 layers with standard attention to achieve 6x higher inference throughput while maintaining reasoning accuracy on long-context tasks. |
Nemotron-Nano-9B-v2 achieves state-of-the-art accuracy with significantly improved inference throughput by replacing most self-attention layers in Transformers with Mamba-2 layers. This hybrid approach maintains the reasoning capabilities of full Transformers while gaining the linear-time complexity benefits of Mamba. The architecture processes extended reasoning traces efficiently, achieving up to 6x higher throughput on reasoning workloads while supporting 128k token context on a single GPU.
Determine which layers use Mamba vs. Transformer attention:
def create_hybrid_architecture(
total_layers: int,
mamba_ratio: float = 0.7,
attention_positions: list[int] = None
) -> list[str]:
"""
Design layer composition with Mamba and Transformer layers.
Default: ~70% Mamba for efficiency, ~30% attention for critical reasoning
Attention layers placed strategically at early, middle, and late stages
"""
if attention_positions is None:
# Default: preserve attention at key positions for information bottlenecks
attention_positions = [0, total_layers // 2, total_layers - 1]
layer_types = []
for i in range(total_layers):
i attention_positions:
layer_types.append()
i < (total_layers * mamba_ratio):
layer_types.append()
:
layer_types.append()
layer_types
Use selective state space models for efficient sequence processing:
def create_mamba2_layer(
hidden_size: int,
state_size: int = 16,
expand_factor: int = 2
) -> "Mamba2Layer":
"""
Create a Mamba-2 layer combining selective state space model with gating.
"""
class Mamba2Layer:
def __init__(self):
self.input_projection = Linear(hidden_size, hidden_size * expand_factor)
self.state_matrix = StateSpaceMatrix(hidden_size * expand_factor, state_size)
self.output_projection = Linear(hidden_size * expand_factor, hidden_size)
self.gate = Linear(hidden_size * expand_factor, hidden_size * expand_factor)
def forward(self, x):
# Project input
x_expanded = self.input_projection(x)
# Apply selective SSM
y = self.state_matrix(x_expanded)
# Apply gating mechanism
gated = y * sigmoid(self.gate(x_expanded))
# Project back to hidden size
output = self.output_projection(gated)
return output
return Mamba2Layer()
Maintain standard Transformer attention at strategic positions:
def create_attention_layer(
hidden_size: int,
num_heads: int = 8,
max_seq_length: int = 128000
) -> "AttentionLayer":
"""
Standard multi-head attention, optimized for long sequences.
"""
class AttentionLayer:
def __init__(self):
self.num_heads = num_heads
self.head_dim = hidden_size // num_heads
self.q_proj = Linear(hidden_size, hidden_size)
self.k_proj = Linear(hidden_size, hidden_size)
self.v_proj = Linear(hidden_size, hidden_size)
self.out_proj = Linear(hidden_size, hidden_size)
def forward(self, x, attention_mask=None):
batch_size, seq_len, _ = x.shape
# Project to Q, K, V
q = self.q_proj(x).view(batch_size, seq_len, self.num_heads, self.head_dim)
k = self.k_proj(x).view(batch_size, seq_len, self.num_heads, self.head_dim)
v = self.v_proj(x).view(batch_size, seq_len, self.num_heads, self.head_dim)
# Compute attention with optional long-context optimization
attention_scores = torch.matmul(q, k.transpose(-2, -1)) / sqrt(self.head_dim)
if attention_mask is not None:
attention_scores = attention_scores + attention_mask
attention_weights = softmax(attention_scores, dim=-1)
output = torch.matmul(attention_weights, v)
# Reshape and project
output = output.view(batch_size, seq_len, -1)
return self.out_proj(output)
return AttentionLayer()
Implement the Minitron compression strategy:
def pretrain_and_compress(
base_config: dict,
target_size: str = "9B",
pretrain_tokens: int = 20_000_000_000,
fp8_enabled: bool = True
) -> "CompressedModel":
"""
Pre-train large model then compress to target size using Minitron strategy.
"""
# Step 1: Pre-train larger base model
print(f"Pre-training on {pretrain_tokens:,} tokens...")
base_model = create_hybrid_model(
hidden_size=base_config["hidden_size"],
num_layers=base_config["num_layers"],
num_heads=base_config["num_heads"]
)
if fp8_enabled:
base_model = convert_to_fp8(base_model)
# Train on diverse data corpus
base_model = train_model(base_model, dataset, pretrain_tokens)
# Step 2: Apply Minitron compression
print(f"Compressing to {target_size}...")
target_hidden_size = map_size_to_hidden_dim(target_size)
target_num_layers = map_size_to_layers(target_size)
compressed = compress_model(
base_model,
target_hidden_size=target_hidden_size,
target_num_layers=target_num_layers,
method="layer_dropping_and_projection"
)
return compressed
Enable efficient processing of extended sequences:
def configure_long_context_inference(
model: "HybridModel",
max_tokens: int = 128000,
device: str = "cuda"
) -> dict:
"""
Set up inference for long-context processing on single GPU.
"""
config = {
"max_sequence_length": max_tokens,
"use_kv_cache": True,
"kv_cache_dtype": torch.float8_e4m3fn, # FP8 for memory efficiency
"attention_implementation": "flash_attention_2", # Optimized kernels
"mamba_scan_mode": "hardware_optimized",
"device": device,
"gradient_checkpointing": False # Only for training
}
# Apply configuration
model.config.update(config)
model = model.to(device)
if fp8_enabled:
model = quantize_model(model, "fp8")
return config
NVIDIA. (2024). NVIDIA Nemotron Nano 2: Hybrid Mamba-Transformer Reasoning Model. arXiv preprint arXiv:2508.14444.