菜单
NVIDIA CUTLASS CUDA Template Library - comprehensive reference for high-performance matrix multiplication (GEMM), convolution, tensor operations, and CuTe DSL across all GPU architectures (Volta through Blackwell)
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Comprehensive reference documentation and skill for xFormers, Facebook Research's toolbox to accelerate research on Transformers. Use this skill whenever the user mentions xformers, memory_efficient_attention, FMHA, flash attention, SwiGLU, RMSNorm, RoPE, rope_padded, 2:4 structured sparsity, sparsify24, sequence parallelism, fused all-gather/reduce-scatter, tiled matmul, block-sparse tensors, attention patterns, selective activation checkpointing, forward-backward overlap, tree attention, model parallel linear layers, xformers profiler, Triton kernels for transformers, CUTLASS attention, BlockDiagonalMask, LowerTriangularMask, merge_attentions, or xformers internals and build configuration.
Comprehensive reference documentation and skill for bitsandbytes, the k-bit quantization library for PyTorch enabling accessible large language models. Use this skill whenever the user mentions bitsandbytes, LLM.int8(), QLoRA, 4-bit quantization, 8-bit quantization, NF4, FP4, 8-bit optimizers, block-wise quantization, Int8Params, Params4bit, Linear8bitLt, Linear4bit, StableEmbedding, quantize_blockwise, dequantize_blockwise, quantize_4bit, dequantize_4bit, quantize_nf4, quantize_fp4, GlobalOptimManager, paged optimizers, FSDP integration with quantization, Triton kernels for quantization, CPU/XPU/MPS/HPU backends, CUDA kernels for quantized matmul, or bitsandbytes internals.
NVIDIA Megatron-LM & Megatron Core - GPU-optimized framework for training large language models with tensor parallelism, pipeline parallelism, data parallelism (DDP/FSDP), context parallelism, expert parallelism, FP8/FP4 quantization, CUDA graphs, MoE (Mixture of Experts), multimodal models, and TensorRT-LLM export. Supports GPT, BERT, T5, Mamba, LLaMA, Mixtral, DeepSeek-V3, Qwen3, and custom architectures from 2B to 462B parameters with up to 47% MFU on H100 GPUs.
Comprehensive reference documentation and skill for NVIDIA NCCL (Collective Communications Library), the GPU communication library for multi-GPU and multi-node collectives. Use this skill whenever the user mentions NCCL, all-reduce, all-gather, reduce-scatter, broadcast, gather/scatter, all-to-all, ncclSend/ncclRecv, communicator initialization, CUDA stream group semantics, distributed training communication, NVLink/NVSwitch/InfiniBand/TCP transport behavior, NCCL environment variables, debugging NCCL hangs or performance, NCCL plugins (net/tuner/profiler/env), Device API, GIN, LSA, symmetric memory, user buffer registration, RAS, or NCCL source-code internals.
Comprehensive reference documentation and skill for PyTorch - the GPU-accelerated tensor computation and deep learning framework. Covers tensor operations, automatic differentiation, neural network modules (nn), optimization, distributed training, CUDA support, automatic mixed precision (AMP), torch.compile/Dynamo, TorchScript, FX graph transformation, Inductor backend, ONNX export, quantization, profiling, data loading, probability distributions, FFT, linear algebra, sparse tensors, C++ API (libtorch), operator dispatch, custom operators, and deployment. Based on PyTorch source code analysis.
Comprehensive reference documentation and skill for SGLang - a high-performance serving framework for large language models and multimodal models. Covers SGLang architecture, ServerArgs configuration, OpenAI-compatible API server, native API, offline engine API, attention backends (FlashInfer, FlashAttention, Triton, FlashMLA, cutlass_mla), KV cache management with RadixAttention, paged attention, sampling and decoding (structured outputs, constrained decoding, speculative decoding with EAGLE/ngram/DFlash), distributed inference (tensor/pipeline/expert/data parallelism), PD disaggregation, EPD disaggregation, quantization (FP8, FP4/MXFP4, GPTQ, AWQ, INT4/INT8, Marlin, bitsandbytes, GGUF, modelopt), multi-LoRA batching, multimodal processing (image/audio/video), CUDA graphs, torch.compile, piecewise CUDA graphs, sgl-kernel, sgl-model-gateway (Rust), HiCache, HiSparse, RL/post-training support, checkpoint engine, diffusion models, observability, profiling, supported model architectures (200+ models), hardware p
| name | cutlass |
| description | NVIDIA CUTLASS CUDA Template Library - comprehensive reference for high-performance matrix multiplication (GEMM), convolution, tensor operations, and CuTe DSL across all GPU architectures (Volta through Blackwell) |
| version | 3.8 |
CUTLASS (CUDA Templates for Linear Algebra Subroutines) is NVIDIA's open-source C++ template library for implementing high-performance matrix-matrix multiplication (GEMM) and related computations at all levels and scales within CUDA. It provides near-optimal utilization of peak theoretical throughput on NVIDIA GPUs.
CUTLASS provides:
Use this skill when:
#include "cutlass/cutlass.h"
#include "cutlass/gemm/device/gemm_universal_adapter.h"
#include "cutlass/gemm/kernel/gemm_universal.h"
#include "cutlass/epilogue/collective/default_epilogue.hpp"
#include "cutlass/gemm/collective/collective_builder.hpp"
// Define types
using ElementA = cutlass::half_t;
using ElementB = cutlass::half_t;
using ElementC = float;
using ElementD = cutlass::half_t;
using LayoutA = cutlass::layout::RowMajor;
using LayoutB = cutlass::layout::ColumnMajor;
using LayoutC = cutlass::layout::RowMajor;
using LayoutD = cutlass::layout::RowMajor;
// Use CollectiveBuilder for convenient kernel assembly
using CollectiveOp = typename cutlass::gemm::collective::CollectiveBuilder<
cutlass::arch::Sm90, cutlass::arch::OpClassTensorOp,
ElementA, LayoutA, 8,
ElementB, LayoutB, 8,
ElementC,
cutlass::gemm::GemmShape<128, 128, 64>,
cutlass::gemm::collective::StageCountAutoCarveout<0>,
cutlass::gemm::collective::KernelScheduleAuto
>::CollectiveOp;
using EpilogueOp = cutlass::epilogue::collective::DefaultEpilogue<
LayoutD, LayoutC,
cutlass::epilogue::collective::EpilogueScheduleAuto
>;
using Kernel = cutlass::gemm::kernel::GemmUniversal<
cutlass::gemm::collective::CollectiveOp,
cutlass::epilogue::collective::EpilogueOp
>;
using Gemm = cutlass::gemm::device::GemmUniversalAdapter<Kernel>;
// Launch
Gemm gemm_op;
auto args = Gemm::Arguments{
{M, N, K},
{ptr_A, stride_A},
{ptr_B, stride_B},
{ptr_C, stride_C},
{ptr_D, stride_D},
{alpha, beta}
};
gemm_op(args);
#include "cutlass/gemm/device/gemm.h"
using Gemm = cutlass::gemm::device::Gemm<
cutlass::half_t, cutlass::layout::RowMajor, // ElementA, LayoutA
cutlass::half_t, cutlass::layout::ColumnMajor, // ElementB, LayoutB
float, cutlass::layout::RowMajor, // ElementC, LayoutC
float, // ElementAccumulator
cutlass::arch::OpClassTensorOp, // OpClass
cutlass::arch::Sm80, // ArchTag
cutlass::gemm::GemmShape<128, 128, 32>, // ThreadblockShape
cutlass::gemm::GemmShape<64, 64, 32>, // WarpShape
cutlass::gemm::GemmShape<16, 8, 16>, // InstructionShape
cutlass::epilogue::thread::LinearCombination<
float, 4, float, float>
>;
Gemm gemm_op;
gemm_op({M, N, K}, alpha, ptr_A, lda, ptr_B, ldb, beta, ptr_C, ldc, ptr_D, ldd);
| Architecture | SM Version | Key Features |
|---|---|---|
| Volta | SM70 | First Tensor Cores (FP16), WMMA API |
| Turing | SM75 | Tensor Cores (INT8/INT4/INT1), MMA instructions |
| Ampere | SM80/SM89 | TF32, BF16, async copy, cp.async |
| Hopper | SM90 | TMA, GMMA, WGMMA, thread block clusters, warp specialization |
| Blackwell | SM100+ | UMMA, block-scaled types (NVFP4/MXFP), distributed GEMM, green contexts |
Detailed reference documentation is organized in the references/ directory: