Ejecuta cualquier Skill en Manus
con un clic

Ejecuta cualquier Skill en Manus con un clic

nccl

Estrellas2

Forks0

Actualizado7 de mayo de 2026 a las 11:27

Comprehensive reference documentation and skill for NVIDIA NCCL (Collective Communications Library), the GPU communication library for multi-GPU and multi-node collectives. Use this skill whenever the user mentions NCCL, all-reduce, all-gather, reduce-scatter, broadcast, gather/scatter, all-to-all, ncclSend/ncclRecv, communicator initialization, CUDA stream group semantics, distributed training communication, NVLink/NVSwitch/InfiniBand/TCP transport behavior, NCCL environment variables, debugging NCCL hangs or performance, NCCL plugins (net/tuner/profiler/env), Device API, GIN, LSA, symmetric memory, user buffer registration, RAS, or NCCL source-code internals.

Instalación

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Ejecutar en Manus

Fuente

jstzwj

jstzwj/ai-infra-plugins

Abrir repositorio de GitHub Ver repositorios del creador

Descarga

Ejecutar en Manus

Ocupaciones relacionadasSOC

Basado en la clasificación ocupacional SOC

Desarrolladores de softwareOcupaciones informáticas y matemáticas·SOC 15-1252

Explorador de archivos

50 archivos

SKILL.md

readonly

Más de este repositorio

mismo repositorio

xformers

jstzwj/ai-infra-plugins

Comprehensive reference documentation and skill for xFormers, Facebook Research's toolbox to accelerate research on Transformers. Use this skill whenever the user mentions xformers, memory_efficient_attention, FMHA, flash attention, SwiGLU, RMSNorm, RoPE, rope_padded, 2:4 structured sparsity, sparsify24, sequence parallelism, fused all-gather/reduce-scatter, tiled matmul, block-sparse tensors, attention patterns, selective activation checkpointing, forward-backward overlap, tree attention, model parallel linear layers, xformers profiler, Triton kernels for transformers, CUTLASS attention, BlockDiagonalMask, LowerTriangularMask, merge_attentions, or xformers internals and build configuration.

2026-05-072

bitsandbytes

jstzwj/ai-infra-plugins

Comprehensive reference documentation and skill for bitsandbytes, the k-bit quantization library for PyTorch enabling accessible large language models. Use this skill whenever the user mentions bitsandbytes, LLM.int8(), QLoRA, 4-bit quantization, 8-bit quantization, NF4, FP4, 8-bit optimizers, block-wise quantization, Int8Params, Params4bit, Linear8bitLt, Linear4bit, StableEmbedding, quantize_blockwise, dequantize_blockwise, quantize_4bit, dequantize_4bit, quantize_nf4, quantize_fp4, GlobalOptimManager, paged optimizers, FSDP integration with quantization, Triton kernels for quantization, CPU/XPU/MPS/HPU backends, CUDA kernels for quantized matmul, or bitsandbytes internals.

2026-05-072

megatron-lm

jstzwj/ai-infra-plugins

NVIDIA Megatron-LM & Megatron Core - GPU-optimized framework for training large language models with tensor parallelism, pipeline parallelism, data parallelism (DDP/FSDP), context parallelism, expert parallelism, FP8/FP4 quantization, CUDA graphs, MoE (Mixture of Experts), multimodal models, and TensorRT-LLM export. Supports GPT, BERT, T5, Mamba, LLaMA, Mixtral, DeepSeek-V3, Qwen3, and custom architectures from 2B to 462B parameters with up to 47% MFU on H100 GPUs.

2026-05-072

pytorch

jstzwj/ai-infra-plugins

Comprehensive reference documentation and skill for PyTorch - the GPU-accelerated tensor computation and deep learning framework. Covers tensor operations, automatic differentiation, neural network modules (nn), optimization, distributed training, CUDA support, automatic mixed precision (AMP), torch.compile/Dynamo, TorchScript, FX graph transformation, Inductor backend, ONNX export, quantization, profiling, data loading, probability distributions, FFT, linear algebra, sparse tensors, C++ API (libtorch), operator dispatch, custom operators, and deployment. Based on PyTorch source code analysis.

2026-05-072

sglang

jstzwj/ai-infra-plugins

Comprehensive reference documentation and skill for SGLang - a high-performance serving framework for large language models and multimodal models. Covers SGLang architecture, ServerArgs configuration, OpenAI-compatible API server, native API, offline engine API, attention backends (FlashInfer, FlashAttention, Triton, FlashMLA, cutlass_mla), KV cache management with RadixAttention, paged attention, sampling and decoding (structured outputs, constrained decoding, speculative decoding with EAGLE/ngram/DFlash), distributed inference (tensor/pipeline/expert/data parallelism), PD disaggregation, EPD disaggregation, quantization (FP8, FP4/MXFP4, GPTQ, AWQ, INT4/INT8, Marlin, bitsandbytes, GGUF, modelopt), multi-LoRA batching, multimodal processing (image/audio/video), CUDA graphs, torch.compile, piecewise CUDA graphs, sgl-kernel, sgl-model-gateway (Rust), HiCache, HiSparse, RL/post-training support, checkpoint engine, diffusion models, observability, profiling, supported model architectures (200+ models), hardware p

2026-05-072

vllm

jstzwj/ai-infra-plugins

Comprehensive reference documentation and skill for vLLM - a high-throughput and memory-efficient inference and serving engine for large language models (LLMs). Covers vLLM architecture (V0 and V1), engine APIs (LLMEngine, AsyncLLMEngine, LLM), OpenAI-compatible API server, configuration system, model executor and layers, attention mechanisms (PagedAttention, FlashAttention, MLA), KV cache management, sampling and decoding (beam search, speculative decoding, structured outputs), distributed inference (tensor/pipeline/data/expert parallelism), multimodal processing (image/audio/video), compilation and CUDA graphs, custom kernels, quantization (FP8, GPTQ, AWQ, INT4/INT8, etc.), LoRA and adapters, scheduling and memory management, supported model architectures (200+ models), speculative decoding (EAGLE, Medusa, n-gram), observability and profiling, and hardware platforms (NVIDIA GPU, AMD GPU, CPU, TPU, Intel GPU/XPU).

2026-05-072

name	nccl
description	Comprehensive reference documentation and skill for NVIDIA NCCL (Collective Communications Library), the GPU communication library for multi-GPU and multi-node collectives. Use this skill whenever the user mentions NCCL, all-reduce, all-gather, reduce-scatter, broadcast, gather/scatter, all-to-all, ncclSend/ncclRecv, communicator initialization, CUDA stream group semantics, distributed training communication, NVLink/NVSwitch/InfiniBand/TCP transport behavior, NCCL environment variables, debugging NCCL hangs or performance, NCCL plugins (net/tuner/profiler/env), Device API, GIN, LSA, symmetric memory, user buffer registration, RAS, or NCCL source-code internals.
version	2.30.4

NCCL - NVIDIA Collective Communications Library

How to use this skill

Use this skill as a code-aware NCCL reference manual. When a user asks about NCCL usage, debugging, performance, plugin development, or internals, first identify which layer they are working at:

Application API usage: communicator lifecycle, collectives, P2P, group semantics, errors.
Advanced user features: buffer registration, symmetric memory windows, one-sided RMA, Device API.
Runtime tuning/debugging: environment variables, topology, algorithms/protocols, transport choice.
Plugin development: net, tuner, profiler, env, GIN/plugin loading.
Source-code internals: init, group/enqueue/scheduler, device kernels, topology graph, proxy/transport.
Bindings/contrib: nccl4py and NCCL EP for MoE expert parallel communication.

When the answer needs precise behavior, cite the source file path from sources/nccl and, if you read current files, cite path:line. If a reference here names a symbol and the user is about to modify code, verify the current source first because NCCL internals change quickly.

NCCL in one page

NCCL is a standalone GPU communication library implementing collective and point-to-point primitives optimized for PCIe, NVLink, NVSwitch, InfiniBand Verbs, and TCP/IP sockets. It supports arbitrary GPU counts in a single node or across multiple nodes, and can be used from single-process, multi-threaded, or multi-process/MPI applications.

Core public API families:

Communicator lifecycle: ncclGetUniqueId, ncclCommInitRank, ncclCommInitAll, ncclCommInitRankConfig, ncclCommInitRankScalable, ncclCommFinalize, ncclCommDestroy, ncclCommAbort, ncclCommSplit, ncclCommShrink, ncclCommGrow, ncclCommRevoke.
Collectives: ncclReduce, ncclBroadcast, ncclAllReduce, ncclReduceScatter, ncclAllGather, ncclAlltoAll, ncclGather, ncclScatter.
P2P and one-sided: ncclSend, ncclRecv, ncclPutSignal, ncclSignal, ncclWaitSignal.
Group semantics: ncclGroupStart, ncclGroupEnd, ncclGroupSimulateEnd.
Memory/registration: ncclMemAlloc, ncclMemFree, ncclCommRegister, ncclCommDeregister, ncclCommWindowRegister, ncclCommWindowDeregister, ncclWinGetUserPtr.
Device API: ncclCommQueryProperties, ncclDevCommCreate, ncclDevCommDestroy, LSA/GIN pointer, team, barrier, and device communication helpers from nccl_device headers.
Parameters: ncclParamBind, typed ncclParamGet*, ncclParamGetParameter, ncclParamGetAllParameterKeys, ncclParamDumpAll.

Quick usage patterns

Single-process multi-GPU allreduce

int ndev = 0;
cudaGetDeviceCount(&ndev);
std::vector<ncclComm_t> comms(ndev);
ncclCommInitAll(comms.data(), ndev, nullptr);

ncclGroupStart();
for (int r = 0; r < ndev; ++r) {
  cudaSetDevice(r);
  ncclAllReduce(send[r], recv[r], count, ncclFloat32, ncclSum, comms[r], streams[r]);
}
ncclGroupEnd();

for (int r = 0; r < ndev; ++r) cudaStreamSynchronize(streams[r]);
for (int r = 0; r < ndev; ++r) {
  ncclCommFinalize(comms[r]);
  ncclCommDestroy(comms[r]);
}

Multi-process initialization shape

ncclUniqueId id;
if (rank == 0) ncclGetUniqueId(&id);
// Broadcast id with MPI, sockets, shared memory, or another launcher mechanism.
cudaSetDevice(local_device);
ncclComm_t comm;
ncclCommInitRank(&comm, world_size, id, rank);

P2P ring pattern must use grouping

ncclGroupStart();
ncclSend(sendbuf, count, ncclFloat32, next_rank, comm, stream);
ncclRecv(recvbuf, count, ncclFloat32, prev_rank, comm, stream);
ncclGroupEnd();

Debugging first line

NCCL_DEBUG=INFO NCCL_DEBUG_SUBSYS=INIT,GRAPH,NET,TUNING ./your_app

Code Examples

Official NCCL performance tests from NVIDIA/nccl-tests, covering all collective operations:

Example	Collective Operation
`src/all_reduce.cu`	AllReduce — sum/avg/min/max across all ranks
`src/all_gather.cu`	AllGather — gather data from all ranks
`src/alltoall.cu`	All-to-All — personalized communication
`src/broadcast.cu`	Broadcast — one-to-all distribution
`src/reduce.cu`	Reduce — reduce to single rank
`src/reduce_scatter.cu`	ReduceScatter — scatter reduced data
`src/gather.cu`	Gather — collect to single rank
`src/scatter.cu`	Scatter — distribute from single rank
`src/sendrecv.cu`	Send/Recv — point-to-point communication
`src/hypercube.cu`	Hypercube collective algorithm
`src/common.cu/h`	Common infrastructure: timing, data validation
`verifiable/`	Verifiable NCCL operations with correctness checks

Each test supports configurable message sizes, data types, and iteration counts for benchmarking NCCL performance.

Documentation map

User-facing API and usage

01 - Overview, Build, Install: what NCCL is, build/package/test flow, examples layout.
02 - Public API and Communicators: public types, errors, config, initialization, lifecycle, split/shrink/grow.
03 - Collectives, P2P, and Groups: operation semantics, in-place rules, count/layout rules, P2P deadlocks, grouping.
04 - Memory Registration, Symmetric Windows, RMA: ncclMemAlloc, buffer registration, symmetric windows, suspend/resume, signals.
05 - Device API, LSA, GIN: device communicators, teams, windows, LSA/GPU-Initiated Networking, examples.
06 - Example Patterns: single process, pthread, MPI, AllReduce, ring P2P, advanced examples.

Runtime configuration and performance

07 - Environment Variables and Parameters: source-derived parameter map and public parameter API.
08 - Debugging, Troubleshooting, Profiling: hangs, async errors, transport visibility, profiler plugins, logging.
09 - Algorithms, Protocols, Tuning: Ring/Tree/CollNet/NVLS/PAT, LL/LL128/SIMPLE, cost model and tuner plugin hooks.

Source internals

10 - Source Architecture and Communicator Lifecycle: key source files, ncclComm, init pipeline.
11 - Group, Enqueue, Scheduler, Launch Pipeline: ncclInfo → tasks → kernel plans → CUDA launch/proxy ops.
12 - Device Kernels and Protocol Primitives: generated kernels, RunWork*, SIMPLE/LL/LL128 primitives.
13 - Topology, Graph Search, Channels: topology discovery, ring/tree/NVLS/CollNet graph search, channel connection.
14 - Transports, Proxy, Networking: P2P/SHM/NET/CollNet/NVLS transports, proxy progress, sockets/IB.

Extension surfaces

15 - Plugin Development: net, tuner, profiler, env, mixed plugins, ABI/versioning/load names.
16 - RAS and Fault Handling: RAS thread, peers, keepalive, communicator revoke/shrink patterns.
17 - Python Bindings and NCCL EP: nccl4py, NCCL EP dispatch/combine for MoE.
18 - Source File Index: source-code map by module for fast navigation.

Answering rules for NCCL tasks

For API questions, explain both host-side enqueue semantics and CUDA stream completion semantics. NCCL calls usually return after enqueueing work, not after communication finishes.
For P2P questions, check whether multiple sends/recvs must progress concurrently. If yes, recommend ncclGroupStart/End.
For multi-GPU single-thread code, default to grouping per-device NCCL calls.
For multi-process code, make cudaSetDevice(localRank) before communicator init explicit.
For hangs, ask for NCCL_DEBUG=INFO, rank count, launcher, device mapping, network interface/NIC, and the last log lines per rank.
For performance, distinguish topology/algorithm/protocol/channel issues from application stream synchronization and buffer registration issues.
For source changes, start from sources/nccl/src/nccl.h.in, collectives.cc, group.cc, enqueue.cc, init.cc, and src/include/comm.h depending on the layer.