| license | Apache-2.0 |
| name | doca-gpunetio-ib-write-lat |
| description | Use this skill when the user is measuring GPU-kernel-initiated RDMA WRITE latency through doca-gpunetio — building and running the `gpunetio_ib_write_lat` client + server pair under `doca/tools/gpunetio_ib_write_lat/`, checking GPU-NIC pairing, reading the half-iter / full-iter / CUDA-side usec columns, characterizing median / p99 / jitter for a real-time control loop, picking GPUNetIO vs GPI vs CPU-initiated `perftest`, or weighing the latency-vs-batching trade-off. Trigger even without 'GPUNetIO' or 'ib_write_lat': 'GPU kernel RDMA latency benchmark', 'how fast can a CUDA kernel post a WRITE', 'p99 RDMA latency on H100 + ConnectX', 'kernel-launched WR tail latency', or 'compare GPU-init vs CPU-init perftest'. Route elsewhere for bandwidth runs (doca-gpunetio-ib-write-bw), the GPI surface (doca-gpi), library debugging (doca-gpunetio), or DOCA install.
|
| metadata | {"kind":"tool"} |
| compatibility | Requires DOCA SDK installed at /opt/mellanox/doca on Linux (Ubuntu 22.04/24.04 or RHEL/SLES) with an InfiniBand-capable ConnectX or BlueField RNIC. Requires NVIDIA GPU with CUDA Toolkit and `nvidia_peermem` loaded; client and server hosts each need a GPU-NIC pair on a common PCIe / NVLink fabric. Reads pkg-config doca-gpunetio / doca-rdma / doca-common and builds from the source tree at /opt/mellanox/doca/tools/gpunetio_ib_write_lat against the installed DOCA.
|
DOCA GPUNetIO ib_write_lat
Where to start: This is a tool skill for the GPUNetIO-
flavored ib_write_lat benchmark shipped under
doca/tools/gpunetio_ib_write_lat/ (a client + server pair,
built from source against the installed DOCA via meson).
It measures the latency of an RDMA WRITE work request when
the WR is posted from a CUDA kernel through the
doca-gpunetio device-side surface, in a ping-pong cadence.
Open TASKS.md and start at
## configure for the GPU-NIC pairing
precondition and the build pattern; jump to
## run for the single-iteration smoke
flow. Open CAPABILITIES.md when the
question is what this tool actually measures, how it
differs from the GPI sister tool on the same physical
operation, or how to interpret the half-iter / full-iter
/ CUDA-side usec output and the median / p99 / jitter
characterization. If DOCA is not installed yet, route to
doca-setup first; if the
user is still deciding between GPUNetIO and GPI as a
programming surface, the picture in
../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes
and
../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes
is the first stop.
Example questions this skill answers well
The CLASSES of doca-gpunetio-ib-write-lat questions this
skill is built to answer, each with one worked example. The
class is the load-bearing piece; the worked example is one
instance.
- "What GPU-init RDMA-WRITE latency / jitter can the
GPUNetIO path deliver for a real-time / control-loop
workload?" — worked example: "measure per-iteration
WRITE latency between two hosts with an H100 +
ConnectX-7 on each side, target the median and the p99
separately". Answered by the GPU-NIC pairing
precondition in
CAPABILITIES.md ## Capabilities and modes
- "This is the GPUNetIO tool — how does the latency
number differ from the GPI programming surface?" —
worked example: . Answered by the
rule in
