Comprehensive reference documentation and skill for xFormers, Facebook Research's toolbox to accelerate research on Transformers. Use this skill whenever the user mentions xformers, memory_efficient_attention, FMHA, flash attention, SwiGLU, RMSNorm, RoPE,…
Comprehensive reference documentation and skill for bitsandbytes, the k-bit quantization library for PyTorch enabling accessible large language models. Use this skill whenever the user mentions bitsandbytes, LLM.int8(), QLoRA, 4-bit quantization, 8-bit…
NVIDIA Megatron-LM & Megatron Core - GPU-optimized framework for training large language models with tensor parallelism, pipeline parallelism, data parallelism (DDP/FSDP), context parallelism, expert parallelism, FP8/FP4 quantization, CUDA graphs, MoE…
Comprehensive reference documentation and skill for NVIDIA NCCL (Collective Communications Library), the GPU communication library for multi-GPU and multi-node collectives. Use this skill whenever the user mentions NCCL, all-reduce, all-gather,…
Comprehensive reference documentation and skill for PyTorch - the GPU-accelerated tensor computation and deep learning framework. Covers tensor operations, automatic differentiation, neural network modules (nn), optimization, distributed training, CUDA…
Comprehensive reference documentation and skill for SGLang - a high-performance serving framework for large language models and multimodal models. Covers SGLang architecture, ServerArgs configuration, OpenAI-compatible API server, native API, offline engine…
Comprehensive reference documentation and skill for vLLM - a high-throughput and memory-efficient inference and serving engine for large language models (LLMs). Covers vLLM architecture (V0 and V1), engine APIs (LLMEngine, AsyncLLMEngine, LLM),…
Comprehensive reference documentation and skill for NVIDIA CUDA C++ - the parallel computing platform and programming model for GPU acceleration. Covers CUDA Programming Guide (Release 13.2) and CUDA C++ Best Practices Guide (Release 13.2). Includes…