메뉴
Compile TensorRT-LLM on a compute node inside a Docker container. Use this when already on a compute node with GPUs visible.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Onboard a HuggingFace multimodal model (vision/audio/video + text) to the TensorRT-LLM PyTorch backend. Use when writing a new `tensorrt_llm/_torch/models/modeling_<vlm>.py` plus its input processor and weight mapper, or extending an existing VLM. Not for AutoDeploy — use `ad-model-onboard` for that path.
Adds sharding-aware IR hints (op substitutions, sharding kwargs, all_reduce insertions) directly into an existing AutoDeploy custom model (modeling_*.py). Edits the file in place — no separate _ir.py copy. Validates with apply_sharding_hints and end-to-end multi-GPU runs.
Review, design, and refactor TensorRT-LLM PyTorch MoE code for architecture fit, clean code, maintainability, and testability. Always use for any modification, review, refactor, or design planning that touches MoE modules, including tensorrt_llm/_torch/modules/fused_moe, ConfigurableMoE, MoE backends, MoEScheduler/moe_scheduler.py, forward execution/chunking, communication strategies, EPLB, quantization/weight handling, routing, factories, MoE docs, or MoE tests. Also use when the user asks whether a MoE design follows the current architecture or whether a MoE refactor is reasonable.
Compile TensorRT-LLM on a SLURM cluster. Covers submitting a batch job with a container image, monitoring the job, and verifying the build. Use when the user wants to compile TRT-LLM remotely via SLURM rather than on a local compute node.
Translates a HuggingFace model into a prefill-only AutoDeploy custom model using reference custom ops, validates with hierarchical equivalence tests.
Debug AutoDeploy accuracy regressions vs a reference score (PyTorch backend or published baseline). Use when an AutoDeploy model's eval score is significantly below the reference and the root cause is unknown.
| name | exec-local-compile |
| description | Compile TensorRT-LLM on a compute node inside a Docker container. Use this when already on a compute node with GPUs visible. |
| license | Apache-2.0 |
| metadata | {"author":"NVIDIA Corporation"} |
Compile TensorRT-LLM from source on a compute node inside a Docker container.
| Scenario | Use This Skill? |
|---|---|
On a compute node with GPUs visible (nvidia-smi works) | Yes |
| On a SLURM login node (no GPUs) | No — use exec-slurm-compile instead |
nvidia-smi succeeds (GPUs visible)/usr/local/tensorrt exists (TensorRT installation in the container)Run nvidia-smi to confirm you are on a compute node with GPU access.
cd to the TensorRT-LLM repository. If the path is not provided by the user, ask for it.
If the user specifies a branch (e.g., "compile ToT"), checkout and pull:
git checkout main && git pull
Run the build command (incremental by default — omit -c/--clean unless explicitly requested or the incremental build fails):
./scripts/build_wheel.py --trt_root /usr/local/tensorrt --benchmarks --use_ccache -a "<arch>" -f --nvtx
Replace <arch> with the target GPU architecture (see Architecture Reference below). If not specified by the user, auto-detect from nvidia-smi.
pip install -e .[devel]
python3 -c "import tensorrt_llm; print(tensorrt_llm.__version__)"
| Flag | Description |
|---|---|
--trt_root /usr/local/tensorrt | TensorRT installation path (standard in NVIDIA containers) |
--benchmarks | Build the C++ benchmarks |
-a "<arch>" | Target GPU architecture(s) |
--nvtx | Enable NVTX markers for profiling |
--use_ccache | Use ccache for faster recompilation |
-f / --fast_build | Skip some kernels for faster dev compilation. Always use for dev builds. |
-c / --clean | Clean build directory before building. Only when needed (see below). |
--skip_building_wheel | Build in-place without creating a wheel file |
--no-venv | Skip virtual environment creation |
| Value | GPU Family |
|---|---|
"100-real" | Blackwell (B200, GB200) |
"90-real" | Hopper (H100, H200) |
"89-real" | Ada Lovelace (L40S) |
"80-real" | Ampere (A100) |
"90;100-real" | Multiple architectures |
Default to incremental builds — CMake only recompiles changed files, saving significant time.
Use a clean build (-c) only when:
CMakeLists.txt, *.cmake)