Menu
Compile TensorRT-LLM on a compute node inside a Docker container. Use this when already on a compute node with GPUs visible.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | exec-local-compile |
| description | Compile TensorRT-LLM on a compute node inside a Docker container. Use this when already on a compute node with GPUs visible. |
| license | Apache-2.0 |
| metadata | {"author":"NVIDIA Corporation"} |
Compile TensorRT-LLM from source on a compute node inside a Docker container.
| Scenario | Use This Skill? |
|---|---|
On a compute node with GPUs visible (nvidia-smi works) | Yes |
| On a SLURM login node (no GPUs) | No โ use exec-slurm-compile instead |
nvidia-smi succeeds (GPUs visible)/usr/local/tensorrt exists (TensorRT installation in the container)Run nvidia-smi to confirm you are on a compute node with GPU access.
cd to the TensorRT-LLM repository. If the path is not provided by the user, ask for it.
If the user specifies a branch (e.g., "compile ToT"), checkout and pull:
git checkout main && git pull
Run the build command (incremental by default โ omit -c/--clean unless explicitly requested or the incremental build fails):
./scripts/build_wheel.py --trt_root /usr/local/tensorrt --benchmarks --use_ccache -a "<arch>" -f --nvtx
Replace <arch> with the target GPU architecture (see Architecture Reference below). If not specified by the user, auto-detect from nvidia-smi.
pip install -e .[devel]
python3 -c "import tensorrt_llm; print(tensorrt_llm.__version__)"
| Flag | Description |
|---|---|
--trt_root /usr/local/tensorrt | TensorRT installation path (standard in NVIDIA containers) |
--benchmarks | Build the C++ benchmarks |
-a "<arch>" | Target GPU architecture(s) |
--nvtx | Enable NVTX markers for profiling |
--use_ccache | Use ccache for faster recompilation |
-f / --fast_build | Skip some kernels for faster dev compilation. Always use for dev builds. |
-c / --clean | Clean build directory before building. Only when needed (see below). |
--skip_building_wheel | Build in-place without creating a wheel file |
--no-venv | Skip virtual environment creation |
| Value | GPU Family |
|---|---|
"100-real" | Blackwell (B200, GB200) |
"90-real" | Hopper (H100, H200) |
"89-real" | Ada Lovelace (L40S) |
"80-real" | Ampere (A100) |
"90;100-real" | Multiple architectures |
Default to incremental builds โ CMake only recompiles changed files, saving significant time.
Use a clean build (-c) only when:
CMakeLists.txt, *.cmake)Onboard a HuggingFace multimodal model (vision/audio/video + text) to the TensorRT-LLM PyTorch backend. Use when writing a new `tensorrt_llm/_torch/models/modeling_<vlm>.py` plus its input processor and weight mapper, or extending an existing VLM. Not for AutoDeploy โ use `ad-model-onboard` for that path.
Adds sharding-aware IR hints (op substitutions, sharding kwargs, all_reduce insertions) directly into an existing AutoDeploy custom model (modeling_*.py). Edits the file in place โ no separate _ir.py copy. Validates with apply_sharding_hints and end-to-end multi-GPU runs.
Review, design, and refactor TensorRT-LLM PyTorch MoE code for architecture fit, clean code, maintainability, and testability. Always use for any modification, review, refactor, or design planning that touches MoE modules, including tensorrt_llm/_torch/modules/fused_moe, ConfigurableMoE, MoE backends, MoEScheduler/moe_scheduler.py, forward execution/chunking, communication strategies, EPLB, quantization/weight handling, routing, factories, MoE docs, or MoE tests. Also use when the user asks whether a MoE design follows the current architecture or whether a MoE refactor is reasonable.
Compile TensorRT-LLM on a SLURM cluster. Covers submitting a batch job with a container image, monitoring the job, and verifying the build. Use when the user wants to compile TRT-LLM remotely via SLURM rather than on a local compute node.
Translates a HuggingFace model into a prefill-only AutoDeploy custom model using reference custom ops, validates with hierarchical equivalence tests.
Debug AutoDeploy accuracy regressions vs a reference score (PyTorch backend or published baseline). Use when an AutoDeploy model's eval score is significantly below the reference and the root cause is unknown.