القائمة
جمع SkillsMP عدد ٣٦ من skills من NVIDIA/TensorRT-LLM. افتح أي skill لمراجعة مصدره وتفاصيله.
عرض ٣٦ من أصل ٣٦ skills مجمعة.
Casebook of past successful and classic TensorRT-LLM optimizations (runtime/execution and kernel level) recorded as reusable decision precedents. Consult when deciding which optimization to apply for a classified bottleneck or a given config/model/hardware,…
لغة النص الأصلي: الإنجليزية
Review, design, and refactor TensorRT-LLM PyTorch MoE code for architecture fit, clean code, maintainability, and testability. Always use for any modification, review, refactor, or design planning that touches MoE modules, including…
لغة النص الأصلي: الإنجليزية
Write and implement GPU kernels using NVIDIA CuTe DSL (CUTLASS 4.x Python API) — NOT for Triton, CUDA C++, or conceptual explanations. Trigger only when the user wants to write or implement a kernel, not when asking questions about CuTe DSL concepts or…
لغة النص الأصلي: الإنجليزية
Check the local execution environment for GPU availability, Docker support, and Slurm access. Returns the execution scenario (`satisfied, local, docker`, `satisfied, local, direct`, `satisfied, slurm, local`, or `not_satisfied`), the number of available GPUs,…
لغة النص الأصلي: الإنجليزية
Execute a TensorRT-LLM workload locally in Docker. Runs a fully-resolved Docker command in background, monitors completion, reads logs, and reports results. Workflow-agnostic — does not need to know if the workload is pytest, eval, benchmark, or a custom…
لغة النص الأصلي: الإنجليزية
Submit and monitor a Slurm job on a local cluster. Supports two modes: (1) Persistent allocation (default) — allocates nodes once via nohup salloc, imports the container once, installs once, and reuses across runs by setting SLURM env vars and running the…
لغة النص الأصلي: الإنجليزية
Remote SLURM cluster development via SSH. Use when running jobs, profiling, or developing on a remote SLURM cluster with pyxis/enroot containers. Covers SSH connection management, srun/sbatch/salloc job patterns, tmux-based allocation persistence, file…
لغة النص الأصلي: الإنجليزية
Compile TensorRT-LLM on a SLURM cluster. Covers submitting a batch job with a container image, monitoring the job, and verifying the build. Use when the user wants to compile TRT-LLM remotely via SLURM rather than on a local compute node.
لغة النص الأصلي: الإنجليزية
Performance analysis coordination workflow. Guides profiling delegation, bottleneck classification (compute/memory/launch/communication/sync), and structured report generation. Use when the user asks to analyze performance, profile a workload, check MFU/SOL,…
لغة النص الأصلي: الإنجليزية
Performance optimization coordination playbook. Contains specialist routing table, TileIR two-step pipeline, kernel generation specialist selection, prioritization criteria, and safe modification workflow. Use when the user asks to apply optimizations, write…
لغة النص الأصلي: الإنجليزية
Run TensorRT-LLM test cases, benchmarks, evaluations, or custom scripts by checking the environment (local GPU or Slurm), selecting the appropriate Docker image, and executing either locally or via Slurm job submission. Accepts pre-built command strings —…
لغة النص الأصلي: الإنجليزية
Build Slurm scripts or Docker commands for TensorRT-LLM workloads. Resolves all parameters (docker image, mounts, parallelism, MPI mode), generates the complete script from Category templates, and writes both the script and a job_spec.json manifest to the…
لغة النص الأصلي: الإنجليزية
Runs model-level and module-level tests for TensorRT-LLM. First classifies the test scope (module test or model test), then dispatches to the appropriate workflow. Model tests are further classified by type (functionality/smoke test, benchmark, or…
لغة النص الأصلي: الإنجليزية
Onboard a HuggingFace multimodal model (vision/audio/video + text) to the TensorRT-LLM PyTorch backend. Use when writing a new `tensorrt_llm/_torch/models/modeling_<vlm>.py` plus its input processor and weight mapper, or extending an existing VLM. Not for…
لغة النص الأصلي: الإنجليزية
Compile TensorRT-LLM on a compute node inside a Docker container. Use this when already on a compute node with GPUs visible.
لغة النص الأصلي: الإنجليزية
Adds sharding-aware IR hints (op substitutions, sharding kwargs, all_reduce insertions) directly into an existing AutoDeploy custom model (modeling_*.py). Edits the file in place — no separate _ir.py copy. Validates with apply_sharding_hints and end-to-end…
لغة النص الأصلي: الإنجليزية
Translates a HuggingFace model into a prefill-only AutoDeploy custom model using reference custom ops, validates with hierarchical equivalence tests.
لغة النص الأصلي: الإنجليزية
Debug AutoDeploy accuracy regressions vs a reference score (PyTorch backend or published baseline). Use when an AutoDeploy model's eval score is significantly below the reference and the root cause is unknown.
لغة النص الأصلي: الإنجليزية
Claude Code skill (trtllm-agent-toolkit): implement or extend TensorRT-LLM AutoDeploy fusion transforms under transform/library/ in a TensorRT-LLM checkout. Prefer existing kernels and custom ops; use Triton only when no viable existing-kernel path exists.…
لغة النص الأصلي: الإنجليزية
Enable and interpret TensorRT-LLM AutoDeploy FX graph text dumps via AD_DUMP_GRAPHS_DIR. Use when you need before/after graphs per transform, to locate subgraphs, or to confirm a rewrite ran. Paths and behavior are grounded in tensorrt_llm/_torch/auto_deploy…
لغة النص الأصلي: الإنجليزية
Visualize a specific transformer decoder layer from an AutoDeploy FX graph text dump as a hierarchical DOT/PNG diagram. Optionally annotate nodes with actual GPU kernel names and durations from an nsys trace. Use when the user wants to visualize, inspect, or…
لغة النص الأصلي: الإنجليزية
Optimize existing Triton kernels for NVIDIA TileIR backend on Blackwell GPUs (sm_100+). Adds TileIR-specific autotune configs: occupancy, num_ctas, TMA descriptors. Covers kernel classification (dot-related, norm-like, elementwise, reduction), type-specific…
لغة النص الأصلي: الإنجليزية
ONLY for OpenAI Triton (@triton.jit) kernel development. NEVER use for CUDA C++ kernels, TileIR, or profiling tools (ncu, nsys). The user's request must involve Triton explicitly. Covers Triton-specific patterns: fused elementwise, reductions (softmax,…
لغة النص الأصلي: الإنجليزية
Analyze host/CPU overhead in TensorRT-LLM inference from nsys traces. Detect whether host overhead is the bottleneck using GPU idle ratio, host prep exposed ratio, and per-phase evidence. For regressions, isolate forward steps via allreduce/NVTX patterns,…
لغة النص الأصلي: الإنجليزية
Profiles and optimizes TensorRT-LLM host/CPU overhead using line_profiler (with nsys support planned). Runs iterative profile-analyze-optimize-validate rounds. Use when GPU utilization is low or optimizing PyExecutor throughput.
لغة النص الأصلي: الإنجليزية
Apply CUDA Graphs to PyTorch workloads — API selection (torch.compile, PyTorch make_graphed_callables, TE make_graphed_callables, MCore CudaGraphManager, FullCudaGraphWrapper, manual torch.cuda.graph), code compatibility, capture workflows, dynamic pattern…
لغة النص الأصلي: الإنجليزية
Identify and eliminate host-device synchronizations in PyTorch code. Detects sync points (.item(), .cpu(), boolean indexing, torch.tensor on CUDA), classifies false vs true dependencies, provides sync-free alternatives. Triggers: sync-free, synchronization,…
لغة النص الأصلي: الإنجليزية
Upgrade flashinfer-python version in TensorRT-LLM. Fetches the latest releases from GitHub (stable and nightly), compares with the current pinned version, lets the user pick a target version, and updates all version references across the repo. Use when the…
لغة النص الأصلي: الإنجليزية
Generate a source-backed starting `trtllm-serve --config` YAML for basic aggregate single-node PyTorch serving, aligned with checked-in TensorRT-LLM configs and deployment docs. Preserves explicit latency / balanced / throughput objectives. Excludes…
لغة النص الأصلي: الإنجليزية
Check whether AutoDeploy YAML configs were actually applied by analyzing server logs and optionally graph dumps (AD_DUMP_GRAPHS_DIR). Use when the user wants to verify config application, debug config issues, or check if AutoDeploy transforms (piecewise CUDA…
لغة النص الأصلي: الإنجليزية
Translates a HuggingFace model into a prefill-only AutoDeploy custom model using reference custom ops, validates with hierarchical equivalence tests.
لغة النص الأصلي: الإنجليزية
Analyze ncu (NVIDIA Nsight Compute) profiling output: SOL% bottleneck classification, roofline analysis, occupancy diagnosis, memory hierarchy analysis, warp stall analysis, metric interpretation, and programmatic .ncu-rep report analysis. NOT for kernel…
لغة النص الأصلي: الإنجليزية
Nsight Systems (nsys) CLI for system-level timeline profiling. Use when the user wants to run nsys profile, analyze .nsys-rep reports, use nsys stats/analyze/recipe commands, diagnose GPU idle time from timeline traces, or profile distributed training with…
لغة النص الأصلي: الإنجليزية
Code instrumentation for timing workloads. Two scenarios: (1) Training loop — inject manual timing to report per-iteration latency, throughput (samples/sec), and data load time. (2) Standalone kernel/op — write CUDA event timing code with warmup,…
لغة النص الأصلي: الإنجليزية
Best practices for contributing code to TensorRT-LLM. Covers the official contribution process (issue tracking, fork workflow, DCO signing), coding guidelines, implementation workflow, common mistakes, testing strategy, commit hygiene, and review readiness.…
لغة النص الأصلي: الإنجليزية
Systematic approach to exploring the TensorRT-LLM codebase before implementing new features or optimizations. Teaches how to discover existing infrastructure, trace code paths, and avoid reimplementing what already exists. Derived from real mistakes where…
لغة النص الأصلي: الإنجليزية