GPU compute in Python with NVIDIA's cuda-python bindings, CuPy (drop-in NumPy on GPU), Numba @cuda.jit, and PyCUDA. Specializes in the optional-dependency pattern — runtime CUDA detection with graceful CPU fallback so code runs on machines without GPU. Use when cuda, gpu, nvidia, cupy, numba, @cuda.jit, pycuda, cuda-python, cuBLAS, cuDNN, device detection, GPU/CPU fallback, optional GPU acceleration, torch.cuda.is_available, ImportError cuda, CUDA toolkit version mismatch, illegal memory access, multi-GPU, kernel fusion, RawKernel.
Instrucciones de origen · Vista previa de solo lectura
name
cuda-python
description
GPU compute in Python with NVIDIA's cuda-python bindings, CuPy (drop-in NumPy on GPU), Numba @cuda.jit, and PyCUDA. Specializes in the optional-dependency pattern — runtime CUDA detection with graceful CPU fallback so code runs on machines without GPU. Use when cuda, gpu, nvidia, cupy, numba, @cuda.jit, pycuda, cuda-python, cuBLAS, cuDNN, device detection, GPU/CPU fallback, optional GPU acceleration, torch.cuda.is_available, ImportError cuda, CUDA toolkit version mismatch, illegal memory access, multi-GPU, kernel fusion, RawKernel.
Loaded automatically when its description matches the active task. This skill is the source of truth for CUDA environment setup and the optional-dependency pattern — other Python skills (pandas, polars, pytorch) that may opt into GPU acceleration delegate the routing/fallback logic here.
Use this skill when
Numerical compute on GPU from Python — array math, linear algebra, custom kernels
Drop-in NumPy speedup with CuPy (cp.asarray(...) instead of np.asarray(...))
Writing custom CUDA kernels with @cuda.jit or cupy.RawKernel
Optional GPU acceleration with CPU fallback — code must run with or without CUDA installed
Direct CUDA driver/runtime API access via cuda.bindings.driver / cuda.bindings.runtime
Diagnosing CUDA environment issues — toolkit vs driver vs library version mismatch
Memory management problems — out-of-memory, fragmentation, pinned vs pageable transfer
Interop between CuPy, PyTorch, NumPy, DLPack consumers (Polars/Arrow, JAX, TensorFlow)
CI without GPU — running CUDA tests via NUMBA_ENABLE_CUDASIM=1 or mocked detection
Do not use this skill when
Deep learning model training/inference workflows — use pytorch (but route here for torch.cuda environment diagnostics, driver issues, container CUDA setup)
Building ML pipelines, optimizers, losses — use pytorch
Pure CPU-only numerical code with no GPU concern — use python / pandas / polars directly
Picking a GPU cloud instance or provisioning the host — use linux-sysadmin
AMD/ROCm acceleration — out of scope (CuPy ROCm is experimental, not covered here)
Purpose
GPU compute in Python is fragmented across four libraries with different abstraction levels — cupy (high-level drop-in NumPy), numba.cuda (JIT-compiled Python kernels), pycuda (raw CUDA C), and cuda-python (NVIDIA's official low-level bindings). Picking the wrong one wastes weeks. This skill exists to make the routing obvious and the optional-dep pattern bulletproof.
The differentiating artifact is references/optional-dep-pattern.md — a complete reusable module that detects CUDA at runtime, exposes a uniform xp namespace (CuPy if available, NumPy otherwise), and never crashes on import in a CPU-only environment. Every Python skill that touches GPU code should import this pattern instead of re-implementing try: import cupy; except: ....
Secondary purpose: act as the CUDA environment expert for sibling skills. When pytorch users hit CUDA error: no kernel image is available for execution on the device, this skill owns the diagnosis path (driver vs toolkit vs library compatibility, container setup, multi-CUDA environments via uv/conda).
Capabilities
Library decision matrix
Four libraries cover the same space at different levels. Choosing wrong is the most common mistake:
CuPy — Drop-in NumPy/SciPy replacement. Pick this 80% of the time. Most array math, FFT, linear algebra, ufuncs work unchanged via cp.asarray(). Custom kernels via RawKernel/ElementwiseKernel/ReductionKernel when needed.
Numba @cuda.jit — JIT-compile Python functions to CUDA kernels. Pick when you need a custom kernel but want to stay in Python syntax. Excellent for prototyping and NUMBA_ENABLE_CUDASIM lets you debug on CPU.
PyCUDA — Low-level, raw CUDA C via SourceModule. Pick only when you have existing .cu source you must wrap. Mostly legacy now — CuPy RawKernel covers the same need with less ceremony.
cuda-python (NVIDIA official) — Direct bindings to driver/runtime/nvrtc/nvjitlink/nvml APIs. Pick when you need control over streams, contexts, primary context attach/detach, or library-version diagnostics from Python. Not for everyday compute.
from your_pkg.gpu import xp, is_cuda_available, cuda_device_count, to_cpu
if is_cuda_available():
x = xp.zeros(N) # CuPy if GPU presentelse:
x = xp.zeros(N) # NumPy fallback
result = to_cpu(x) # uniform device→host transfer
The pattern handles: ImportError (cupy not installed), RuntimeError (cupy installed but driver mismatch), cupy.cuda.runtime.getDeviceCount() == 0 (no GPU on host), env-var override FORCE_CPU=1 for CI/testing. See references/optional-dep-pattern.md for the full module + decorator pattern + pytest fixture for mocking GPU presence.
Out-of-memory, version mismatch, illegal memory access, no kernel image, slow first call (JIT compile cost), multiprocessing+CUDA fork errors, container Docker NVIDIA toolkit issues. See references/troubleshooting.md.
Recommended defaults
Memory pool config, block/grid sizing rules of thumb, when streams help vs hurt, profiling intervals, when to set CUPY_ACCELERATORS. See references/recommended-defaults.md.
Behavioral Traits
Always calls cupy.cuda.runtime.getDeviceCount() or equivalent inside a try/except before any CuPy operation — never assumes the device is present
Always provides a CPU fallback path when GPU is described as "optional acceleration"
Imports CuPy lazily (inside a function or try block at module top) — never at the top of a module that must import on CPU-only hosts
Pins cupy-cudaXxx to match the installed CUDA Toolkit major; documents the supported range in pyproject.toml
Uses the xp namespace pattern (xp = cupy if has_cuda else numpy) to keep call sites identical
For Numba, ships tests that run under NUMBA_ENABLE_CUDASIM=1 so CI without GPU can exercise kernel logic
Reads nvidia-smi and nvcc --version before debugging — driver and toolkit version drift is the #1 root cause
Uses mempool.free_all_blocks() between major phases of long-running scripts to surface real memory needs
Prefers DLPack over __cuda_array_interface__ when both sides support it — DLPack handles stream safety and lifetime
Important Constraints
NEVER assume a GPU exists. Always guard with cuda_is_available() before any cupy/torch.cuda/numba.cuda call
NEVER import cupy, pycuda, or numba.cuda at the top of a module that must run on CPU-only hosts — use lazy import inside the optional-dep wrapper
NEVER call torch.cuda.<anything> or cupy.<op> without an is_available guard — they raise RuntimeError (not ImportError) when driver is broken
NEVER mix CUDA Toolkit majors in one environment. cupy-cuda12x + a CUDA 13 PyTorch wheel = silent corruption or no kernel image errors
ALWAYS document the supported CUDA Toolkit version range in pyproject.toml (e.g., cupy-cuda12x>=13.0,<14) and in README
NEVER allocate inside a hot loop. Pre-allocate outputs with cp.empty(shape) outside the loop and pass them in
NEVER transfer host↔device inside a hot loop — batch transfers, keep data resident on device
NEVER call cp.cuda.Stream.synchronize() after every op — it kills async benefits; use only at output boundaries
NEVER swallow cupy.cuda.runtime.CUDARuntimeError — surface it with the original error code; the code is the diagnostic key
ALWAYS check nvidia-smi shows the GPU before debugging Python — if driver is missing, no Python code can fix it
NEVER use multiprocessing.fork() after initializing a CUDA context — fork-after-init corrupts the child's context; use spawn instead
Related Skills
Python stack (siblings — opt-into GPU through this skill's optional-dep pattern)
✓ python — base Python 3.14 tooling, packaging via uv, pyproject.toml
✓ pytorch — deep learning framework, owns torch.cuda.is_available(); route here for environment diagnostics
✓ pandas — CPU DataFrames; cudf is a separate sibling out of scope here
✓ polars — fast DataFrames; can consume CuPy via DLPack/Arrow
✓ scikit-learn — classical ML; route here for GPU-accelerated inference fallback patterns
How to use: open the specific topic file. The optional-dep-pattern is the differentiating artifact — read it first if your job is to write CPU/GPU-portable code.
