ai-dynamo/nixl

Última atividade de origem registrada: 26 de maio de 2026 às 18:20

Catálogo do SkillsMP atualizado: 12 de novembro de 2026 às 12:52

skills coletadas: 1

Estrelas no GitHub: 1.363
Forks no GitHub: 520

Instalar com um assistente de IA

Copie este prompt para o assistente de IA que você usa.

Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/ai-dynamo/nixl.

README do repositório

Arquivo de origem do README: README.md

Ler o README original

Expandir README completo

# NVIDIA Inference Xfer Library (NIXL) NVIDIA Inference Xfer Library (NIXL) is targeted for accelerating point to point communications in AI inference frameworks such as NVIDIA Dynamo, while providing an abstraction over various types of memory (e.g., CPU and GPU) and storage (e.g., file, block and object store) through a modular plug-in architecture. [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![GitHub Release](https://img.shields.io/github/v/release/ai-dynamo/nixl)](https://github.com/ai-dynamo/nixl/releases/latest) ## Documentation and Resources * [NIXL overview](https://github.com/ai-dynamo/nixl/blob/main/docs/nixl.md) - Core concepts/architecture overview (`docs/nixl.md`) * [Python API](https://github.com/ai-dynamo/nixl/blob/main/docs/python_api.md) - Python API usage and examples (`docs/python_api.md`) * [Backend guide](https://github.com/ai-dynamo/nixl/blob/main/docs/BackendGuide.md) - Backend/plugin development guide (`docs/BackendGuide.md`) * [Telemetry](https://github.com/ai-dynamo/nixl/blob/main/docs/telemetry.md) - Observability and telemetry details (`docs/telemetry.md`) * [Doxygen guide](https://github.com/ai-dynamo/nixl/blob/main/docs/doxygen/nixl_doxygen.md) - API/class diagrams overview (`docs/doxygen/nixl_doxygen.md`) * [Doxygen images](https://github.com/ai-dynamo/nixl/tree/main/docs/doxygen) - Diagram assets (`docs/doxygen/`) * [NIXLBench docs](https://github.com/ai-dynamo/nixl/blob/main/benchmark/nixlbench/README.md) - Benchmark usage guide (`benchmark/nixlbench/README.md`) * [KVBench docs](https://github.com/ai-dynamo/nixl/tree/main/benchmark/kvbench/docs) - KVBench workflows and tutorials (`benchmark/kvbench/docs/`) * [Use cases](https://github.com/ai-dynamo/nixl/blob/main/docs/uses.md) - NIXL usage examples and integrations (`docs/uses.md`) ## Supported Platforms NIXL is supported on a Linux environment only. It is tested on Ubuntu (22.04/24.04) and Fedora. macOS and Windows are not currently supported; use a Linux host or container/VM. ## Pre-build Distributions ### PyPI Wheel The nixl python API and libraries, including UCX, are available directly through PyPI. For example, if you have a GPU running on a Linux host, container, or VM, you can do the following install: Install with: ```bash pip install nixl ``` This installs both CUDA 12 and CUDA 13 backends. At runtime, the correct backend is selected automatically based on the CUDA version reported by PyTorch. ## Prerequisites for source build (Linux) NIXL requires a C++20 compatible compiler (GCC >= 11 or Clang >= 14). ### Ubuntu: `sudo apt install build-essential cmake pkg-config` ### Fedora: `sudo dnf install gcc-c++ cmake pkg-config` ### Python `pip3 install meson ninja pybind11 tomlkit` ### UCX NIXL was tested with UCX version 1.23.x. [GDRCopy](https://github.com/NVIDIA/gdrcopy) is available on Github and is necessary for maximum performance, but UCX and NIXL will work without it. ``` git clone https://github.com/openucx/ucx.git cd ucx git checkout v1.23.x ./autogen.sh ./contrib/configure-release-mt \ --enable-shared \ --disable-static \ --disable-doxygen-doc \ --enable-optimizations \ --without-avx \ --enable-cma \ --enable-devel-headers \ --with-cuda=<cuda install> \ --with-verbs \ --with-dm \ --with-gdrcopy=<gdrcopy install> make -j make -j install-strip ldconfig ``` ### ETCD (Optional) NIXL can use ETCD for metadata distribution and coordination between nodes in distributed environments. To use ETCD with NIXL: #### ETCD Server and Client ``` sudo apt install etcd etcd-server etcd-client # Or use Docker docker run -d -p 2379:2379 \ --name etcd-server \ -e ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379 \ -e ETCD_ADVERTISE_CLIENT_URLS=http://0.0.0.0:2379 \ quay.io/coreos/etcd:v3.5.1 ``` #### ETCD CPP API Installed from https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3 ``` sudo apt install libgrpc-dev libgrpc++-dev libprotobuf-dev protobuf-compiler-grpc sudo apt install libcpprest-dev git clone https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3.git cd etcd-cpp-apiv3 mkdir build && cd build cmake .. make -j$(nproc) && make install ``` ### Additional plugins Some plugins may have additional build requirements, see them here: - [Mooncake](src/plugins/mooncake/README.md) - [POSIX](src/plugins/posix/README.md) - [GDS](src/plugins/cuda_gds/README.md) ## Getting started ### Build & install ``` meson setup <name_of_build_dir> cd <name_of_build_dir> ninja ninja install ``` ### Build Options #### Release build (default) ```bash meson setup <name_of_build_dir> ``` #### Debug build ```bash meson setup <name_of_build_dir> --buildtype=debug ``` #### NIXL-specific build options ```bash # Example with custom options meson setup <name_of_build_dir> \ -Dbuild_docs=true \ # Build Doxygen documentation -Ducx_path=/path/to/ucx \ # Custom UCX installation path -Dinstall_headers=true \ # Install development headers -Ddisable_gds_backend=false # Enable GDS backend ``` Common build options: - `build_docs`: Build Doxygen documentation (default: false) - `ucx_path`: Path to UCX installation (default: system path) - `install_headers`: Install development headers (default: true for direct Meson builds; disabled for Python wheels) - `disable_gds_backend`: Disable GDS backend (default: false) - `cudapath_inc`, `cudapath_lib`: Custom CUDA paths - `static_plugins`: Comma-separated list of plugins to build statically - `enable_plugins`: Comma-separated list of plugins to build (e.g. `-Denable_plugins=UCX,POSIX`). Cannot be used with `disable_plugins`. - `disable_plugins`: Comma-separated list of plugins to exclude (e.g. `-Ddisable_plugins=GDS`). Cannot be used with `enable_plugins`. - `wheel_variant`: Override the Python wheel variant suffix (e.g. `-Dwheel_variant=rocm` yields `nixl_rocm`). Empty (default) = autodetect from the CUDA major version. #### Building for AMD ROCm NIXL itself builds vendor-neutrally; CPU-side hardware detection (`hwInfo::numAmdGpus`) discovers AMD GPUs via PCI vendor `0x1002` whether or not a ROCm toolchain is present. GPU-side ROCm/HIP build support is available for nixlbench and UCX plugin unit tests. When packaging a ROCm wheel, pass `-Dwheel_variant=rocm` so the wheel is named `nixl_rocm`. **Building with ROCm support:** ```bash # For UCX unit tests with ROCm meson setup build -Drocm_path=/opt/rocm # Or specify a custom ROCm path meson setup build -Drocm_path=/custom/path/to/rocm ``` **Plugins on ROCm hosts (CUDA toolchain absent):** - `UCX` — primary transport for AMD GPU memory (requires UCX built with `--with-rocm`). - `POSIX`, `OBJ`, `AZURE_BLOB`, `HF3FS`, `GUSLI`, `UCCL` — vendor-neutral; build unchanged. - `MOONCAKE` — needs a Transfer Engine built with `-DUSE_HIP=ON` for AMD GPU memory. - `GDS` / `GDS_MT`, `GPUNETIO`, `LIBFABRIC` (with `-DHAVE_CUDA`) — skip automatically because their CUDA / cuFile / DOCA dependencies are not found. **Known gaps (will be addressed in follow-up PRs):** - `LIBFABRIC` plugin disabled on ROCm pending header refactor. - No NVSHMEM-equivalent backend yet (rocSHMEM analog is a candidate for a future plugin). #### Environment Variables There are a few environment variables that can be set to configure the build: - `NIXL_NO_STUBS_FALLBACK`: If not set or 0, build NIXL stub library if the library build fails ### Building Documentation If you have Doxygen installed, you can build the documentation: ```bash # Configure with documentation enabled meson setup <name_of_build_dir> -Dbuild_docs=true cd <name_of_build_dir> ninja # Documentation will be generated in <name_of_build_dir>/html # After installation (ninja install), documentation will be available in <prefix>/share/doc/nixl/ ``` ### Python Interface NIXL provides Python bindings through pybind11. For detailed Python API documentation, see [docs/python_api.md](docs/python_api.md). The preferred way to install the Python bindings is through pip from PyPI: ```bash pip install nixl ``` This installs both CUDA 12 and CUDA 13 backends. At runtime, the correct backend is selected automatically based on the CUDA version reported by PyTorch. #### Installation from source Prerequisites: - `uv`: https://docs.astral.sh/uv/getting-started/installation/ - `tomlkit`: https://pypi.org/project/tomlkit/ - `PyTorch`: https://pytorch.org/get-started/locally/ `uv` is always required *even if* you have another kind of Python virtual environment manager or if you are using a system-wide Python installation without using a virtual environment. Example with `uv` Python virtual environment: ``` curl -LsSf https://astral.sh/uv/install.sh | sh export PATH="$HOME/.local/bin:${PATH}" uv venv .venv --python 3.12 source .venv/bin/activate uv pip install tomlkit ``` Example with python-virtualenv: ``` curl -LsSf https://astral.sh/uv/install.sh | sh export PATH="$HOME/.local/bin:${PATH}" python3 -m venv .venv source .venv/bin/activate pip install tomlkit ``` Example with system-wide Python installation without using a virtual environment: ``` curl -LsSf https://astral.sh/uv/install.sh | sh export PATH="$HOME/.local/bin:${PATH}" pip install tomlkit ``` Then install PyTorch following the instructions on the PyTorch website: https://pytorch.org/get-started/locally/ After installing the prerequisites, you can build and install the NIXL binaries and the Python bindings from source. You have to: 1. Build NIXL binaries and install them 2. Build and install the CUDA platform-specific package (`nixl-cu12` or `nixl-cu13`) 3. Build and install the `nixl` meta-package **For CUDA 12:** ``` pip install . meson setup build ninja -C build install pip install build/src/bindings/python/nixl-meta/nixl-*-py3-none-any.whl ``` **For CUDA 13:** ``` pip install . ./contrib/tomlutil.py --wheel-name nixl-cu13 pyproject.toml meson setup build ninja -C build install pip install build/src/bindings/python/nixl-meta/nixl-*-py3-none-any.whl ``` To check if the installation is successful, you can run the following command: ``` python3 -c "import nixl; agent = nixl.nixl_agent('agent1')" ``` which should print: ``` 2026-01-08 13:36:27 NIXL INFO _api.py:363 Backend UCX was instantiated 2026-01-08 13:36:27 NIXL INFO _api.py:253 Initialized NIXL agent: agent1 ``` You can also run a complete Python example to test the installation: ``` python3 examples/python/expanded_two_peers.py --mode=target --use_cuda=true --ip=127.0.0.1 --port=4242 & sleep 5 python3 examples/python/expanded_two_peers.py --mode=initiator --use_cuda=true --ip=127.0.0.1 --port=4242 ``` For more Python examples, see [examples/python/](examples/python/). ### Rust Bindings #### Build - Use `-Drust=true` meson option to build rust bindings. - Use `--buildtype=debug` for a debug build (default is release). - Or build manually: ```bash cargo build --release ``` #### Install The bindings will be installed under `nixl-sys` in the configured installation prefix. Can be done using ninja, from project build directory: ```bash ninja install ``` #### Test ``` # Rust bindings tests cargo test ``` Use in your project by adding to `Cargo.toml`: ```toml [dependencies] nixl-sys = { path = "path/to/nixl/bindings/rust" } ``` ### Other build options See [contrib/README.md](contrib/README.md) for more build options. ### Building Docker container To build the docker container, first clone the current repository. Also make sure you are able to pull docker images to your machine before attempting to build the container. Run the following from the root folder of the cloned NIXL repository: ``` ./contrib/build-container.sh ``` By default, the container is built with Ubuntu 24.04. To build a container for Ubuntu 22.04 use the --os option as follows: ``` ./contrib/build-container.sh --os ubuntu22 ``` To see all the options supported by the container use: ``` ./contrib/build-container.sh -h ``` The container has the NIXL python bindings preinstalled (built from source against the container's own PyTorch). For a redistributable python wheel, use the wheel build script below or install the published `nixl` package. ### Building the python wheel Python wheel builds set `-Dinstall_headers=false` in [pyproject.toml](pyproject.toml), so installing a wheel does not install the NIXL C++ development headers. This overrides the default for direct Meson builds listed in [Build Options](#build-options). For C++ development, follow [Build & install](#build--install) with `install_headers` enabled. Headers are installed under `<prefix>/include` (`/opt/nvidia/nvda_nixl/include` by default); use Meson's `--prefix` option to choose a different installation prefix. The contrib folder also includes a script to build the python wheel with the UCX dependencies. Note, that UCX and other NIXL dependencies are required to be installed. ``` ./contrib/build-wheel.sh ``` ## Running with ETCD NIXL can use ETCD for metadata exchange between distributed nodes. This is especially useful in containerized or cloud-native environments. The examples below require NIXL built with ETCD support. Install the [ETCD CPP API prerequisites](#etcd-cpp-api) before building NIXL. ### Environment Setup To use ETCD with NIXL, set the following environment variables: ```bash # Set ETCD endpoints (required) - replace localhost with the hostname of the etcd server export NIXL_ETCD_ENDPOINTS="http://localhost:2379" # Set ETCD namespace (optional, defaults to /nixl/agents/) export NIXL_ETCD_NAMESPACE="/nixl/agents/" ``` ### Running the ETCD Examples NIXL includes C++ and Python examples demonstrating metadata exchange and data transfer using ETCD: ```bash # Start an ETCD server as described in "ETCD Server and Client" above # Set the ETCD env variables as above # Run the example. The two agents in the example will exchange metadata through ETCD # and perform data transfers ./<nixl_build_path>/examples/cpp/nixl_etcd_example ``` To run the Python example from the repository root after installing the Python bindings: ```bash python3 examples/python/nixl_etcd_example.py ``` ### nixlbench Benchmark For more comprehensive testing, the nixlbench benchmarking tool supports ETCD for worker coordination: ```bash # Build nixlbench (see benchmark/nixlbench/README.md for details) cd benchmark/nixlbench meson setup build && cd build && ninja # Run benchmark with ETCD ./nixlbench --etcd-endpoints http://localhost:2379 --backend UCX --initiator_seg_type VRAM ``` ## Code Examples * [C++ examples](https://github.com/ai-dynamo/nixl/tree/main/examples/cpp) * [Python examples](https://github.com/ai-dynamo/nixl/tree/main/examples/python) ## Contributing For contribution guidelines, see [CONTRIBUTING.md](https://github.com/ai-dynamo/nixl/blob/main/CONTRIBUTING.md) (`CONTRIBUTING.md`). ## Third-Party Components This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use. NIXL Python wheels bundle NVIDIA modules (`libuct_ib_mlx5_ext.so`, `libuct_ib_mlx5_gda.so`, `libuct_ib_mlx5_gdp.so`) licensed under the [NVIDIA Proprietary License](licenses/NVIDIA-proprietary-LICENSE.txt) (`LicenseRef-NvidiaProprietary`).

Carregando o README do repositório…

Skills neste repositório

Desenvolvedores de software

1 categorias ocupacionais · 100% classificado

skill

ocupação

descrição

atualizado

nixl-python-api

Desenvolvedores de software

Use for source-matched NIXL Python API help on agents, descriptors, metadata, transfers, polling, or cleanup. Do NOT use for install/framework setup.

Idioma do texto original: inglês

2026-05-26

Mostrando 1 de 1 skills coletadas.