메뉴
NVIDIA CUTLASS CUDA Template Library - comprehensive reference for high-performance matrix multiplication (GEMM), convolution, tensor operations, and CuTe DSL across all GPU architectures (Volta through Blackwell)
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | cutlass |
| description | NVIDIA CUTLASS CUDA Template Library - comprehensive reference for high-performance matrix multiplication (GEMM), convolution, tensor operations, and CuTe DSL across all GPU architectures (Volta through Blackwell) |
| version | 3.8 |
CUTLASS (CUDA Templates for Linear Algebra Subroutines) is NVIDIA's open-source C++ template library for implementing high-performance matrix-matrix multiplication (GEMM) and related computations at all levels and scales within CUDA. It provides near-optimal utilization of peak theoretical throughput on NVIDIA GPUs.
CUTLASS provides:
Use this skill when:
#include "cutlass/cutlass.h"
#include "cutlass/gemm/device/gemm_universal_adapter.h"
#include "cutlass/gemm/kernel/gemm_universal.h"
#include "cutlass/epilogue/collective/default_epilogue.hpp"
#include "cutlass/gemm/collective/collective_builder.hpp"
// Define types
using ElementA = cutlass::half_t;
using ElementB = cutlass::half_t;
using ElementC = float;
using ElementD = cutlass::half_t;
using LayoutA = cutlass::layout::RowMajor;
using LayoutB = cutlass::layout::ColumnMajor;
using LayoutC = cutlass::layout::RowMajor;
using LayoutD = cutlass::layout::RowMajor;
// Use CollectiveBuilder for convenient kernel assembly
using CollectiveOp = typename cutlass::gemm::collective::CollectiveBuilder<
cutlass::arch::Sm90, cutlass::arch::OpClassTensorOp,
ElementA, LayoutA, 8,
ElementB, LayoutB, 8,
ElementC,
cutlass::gemm::GemmShape<128, 128, 64>,
cutlass::gemm::collective::StageCountAutoCarveout<0>,
cutlass::gemm::collective::KernelScheduleAuto
>::CollectiveOp;
using EpilogueOp = cutlass::epilogue::collective::DefaultEpilogue<
LayoutD, LayoutC,
cutlass::epilogue::collective::EpilogueScheduleAuto
>;
using Kernel = cutlass::gemm::kernel::GemmUniversal<
cutlass::gemm::collective::CollectiveOp,
cutlass::epilogue::collective::EpilogueOp
>;
using Gemm = cutlass::gemm::device::GemmUniversalAdapter<Kernel>;
// Launch
Gemm gemm_op;
auto args = Gemm::Arguments{
{M, N, K},
{ptr_A, stride_A},
{ptr_B, stride_B},
{ptr_C, stride_C},
{ptr_D, stride_D},
{alpha, beta}
};
gemm_op(args);
#include "cutlass/gemm/device/gemm.h"
using Gemm = cutlass::gemm::device::Gemm<
cutlass::half_t, cutlass::layout::RowMajor, // ElementA, LayoutA
cutlass::half_t, cutlass::layout::ColumnMajor, // ElementB, LayoutB
float, cutlass::layout::RowMajor, // ElementC, LayoutC
float, // ElementAccumulator
cutlass::arch::OpClassTensorOp, // OpClass
cutlass::arch::Sm80, // ArchTag
cutlass::gemm::GemmShape<128, 128, 32>, // ThreadblockShape
cutlass::gemm::GemmShape<64, 64, 32>, // WarpShape
cutlass::gemm::GemmShape<16, 8, 16>, // InstructionShape
cutlass::epilogue::thread::LinearCombination<
float, 4, float, float>
>;
Gemm gemm_op;
gemm_op({M, N, K}, alpha, ptr_A, lda, ptr_B, ldb, beta, ptr_C, ldc, ptr_D, ldd);
| Architecture | SM Version | Key Features |
|---|---|---|
| Volta | SM70 | First Tensor Cores (FP16), WMMA API |
| Turing | SM75 | Tensor Cores (INT8/INT4/INT1), MMA instructions |
| Ampere | SM80/SM89 | TF32, BF16, async copy, cp.async |
| Hopper | SM90 | TMA, GMMA, WGMMA, thread block clusters, warp specialization |
| Blackwell | SM100+ | UMMA, block-scaled types (NVFP4/MXFP), distributed GEMM, green contexts |
Detailed reference documentation is organized in the references/ directory:
Comprehensive reference documentation and skill for xFormers, Facebook Research's toolbox to accelerate research on Transformers. Use this skill whenever the user mentions xformers, memory_efficient_attention, FMHA, flash attention, SwiGLU, RMSNorm, RoPE, rope_padded, 2:4 structured sparsity, sparsify24, sequence parallelism, fused all-gather/reduce-scatter, tiled matmul, block-sparse tensors, attention patterns, selective activation checkpointing, forward-backward overlap, tree attention, model parallel linear layers, xformers profiler, Triton kernels for transformers, CUTLASS attention, BlockDiagonalMask, LowerTriangularMask, merge_attentions, or xformers internals and build configuration.
Comprehensive reference documentation and skill for bitsandbytes, the k-bit quantization library for PyTorch enabling accessible large language models. Use this skill whenever the user mentions bitsandbytes, LLM.int8(), QLoRA, 4-bit quantization, 8-bit quantization, NF4, FP4, 8-bit optimizers, block-wise quantization, Int8Params, Params4bit, Linear8bitLt, Linear4bit, StableEmbedding, quantize_blockwise, dequantize_blockwise, quantize_4bit, dequantize_4bit, quantize_nf4, quantize_fp4, GlobalOptimManager, paged optimizers, FSDP integration with quantization, Triton kernels for quantization, CPU/XPU/MPS/HPU backends, CUDA kernels for quantized matmul, or bitsandbytes internals.
NVIDIA Megatron-LM & Megatron Core - GPU-optimized framework for training large language models with tensor parallelism, pipeline parallelism, data parallelism (DDP/FSDP), context parallelism, expert parallelism, FP8/FP4 quantization, CUDA graphs, MoE (Mixture of Experts), multimodal models, and TensorRT-LLM export. Supports GPT, BERT, T5, Mamba, LLaMA, Mixtral, DeepSeek-V3, Qwen3, and custom architectures from 2B to 462B parameters with up to 47% MFU on H100 GPUs.
Comprehensive reference documentation and skill for NVIDIA NCCL (Collective Communications Library), the GPU communication library for multi-GPU and multi-node collectives. Use this skill whenever the user mentions NCCL, all-reduce, all-gather, reduce-scatter, broadcast, gather/scatter, all-to-all, ncclSend/ncclRecv, communicator initialization, CUDA stream group semantics, distributed training communication, NVLink/NVSwitch/InfiniBand/TCP transport behavior, NCCL environment variables, debugging NCCL hangs or performance, NCCL plugins (net/tuner/profiler/env), Device API, GIN, LSA, symmetric memory, user buffer registration, RAS, or NCCL source-code internals.
Comprehensive reference documentation and skill for PyTorch - the GPU-accelerated tensor computation and deep learning framework. Covers tensor operations, automatic differentiation, neural network modules (nn), optimization, distributed training, CUDA support, automatic mixed precision (AMP), torch.compile/Dynamo, TorchScript, FX graph transformation, Inductor backend, ONNX export, quantization, profiling, data loading, probability distributions, FFT, linear algebra, sparse tensors, C++ API (libtorch), operator dispatch, custom operators, and deployment. Based on PyTorch source code analysis.
Comprehensive reference documentation and skill for SGLang - a high-performance serving framework for large language models and multimodal models. Covers SGLang architecture, ServerArgs configuration, OpenAI-compatible API server, native API, offline engine API, attention backends (FlashInfer, FlashAttention, Triton, FlashMLA, cutlass_mla), KV cache management with RadixAttention, paged attention, sampling and decoding (structured outputs, constrained decoding, speculative decoding with EAGLE/ngram/DFlash), distributed inference (tensor/pipeline/expert/data parallelism), PD disaggregation, EPD disaggregation, quantization (FP8, FP4/MXFP4, GPTQ, AWQ, INT4/INT8, Marlin, bitsandbytes, GGUF, modelopt), multi-LoRA batching, multimodal processing (image/audio/video), CUDA graphs, torch.compile, piecewise CUDA graphs, sgl-kernel, sgl-model-gateway (Rust), HiCache, HiSparse, RL/post-training support, checkpoint engine, diffusion models, observability, profiling, supported model architectures (200+ models), hardware p