Skip to main content

NVIDIA/TensorRT-Edge-LLM

最近记录的来源活动
SkillsMP 收录数据更新
已收集 skills
15
GitHub 星标
631
GitHub Forks
153

用 AI 助手安装

复制这段 Prompt,发给你正在使用的 AI 助手。

正在加载仓库 README…

这个仓库中的 skills

职业分类待补全

已展示 15 / 15 个已收集 Skill。

职业分类
未分类
描述

Localize a TensorRT Edge-LLM accuracy mismatch against the model provider's PyTorch reference after few-layer validation identifies a failing component or layer.

原文语言:英语

更新
职业分类
未分类
描述

Implement TensorRT Edge-LLM support for a new Hugging Face checkpoint or model family. Use for a new `model_type`, architecture, text/vision/audio/action component, checkpoint tensor layout, speculative draft, model-specific custom operation, or runtime…

原文语言:英语

更新
职业分类
未分类
描述

Measure dataset accuracy (MMLU and other NeMo Evaluator benchmarks) of a TensorRT Edge-LLM deployment through its OpenAI-compatible chat endpoint with NeMo Evaluator. Use for "benchmark accuracy", "run MMLU/GSM8K/IFEval on this checkpoint", comparing a…

原文语言:英语

更新
职业分类
未分类
描述

Benchmark TensorRT Edge-LLM serving performance (TTFT, inter-token latency, request latency, output throughput, concurrency scaling) through the OpenAI-compatible server with AIPerf. Use for "benchmark performance", "measure TTFT/ITL/tokens per second",…

原文语言:英语

更新
职业分类
未分类
描述

Convert and run one supported Hugging Face checkpoint with TensorRT Edge-LLM. Use for quantization, checkpoint-to-ONNX export, `llm_build`/`visual_build`/`audio_build`, C++ `llm_inference`, the experimental checkpoint-direct `trtedge build`, engine profile…

原文语言:英语

更新
职业分类
未分类
描述

Write or change a C++ test under unittests/. Covers the BDD comment contract, mutation testing, and the substitute engines that drive LLMInferenceRuntime without a serialized engine. Apply whenever adding, migrating, or repairing a GTest case in this…

原文语言:英语

更新
职业分类
未分类
描述

Add or change a TensorRT Edge-LLM CuTe DSL kernel end to end, from the Python kernel source through the KERNEL_VARIANTS registry in kernelSrcs/build_cutedsl.py, GPU-free AOT artifact generation (native or AArch64 cross build), the metadata-driven…

原文语言:英语

更新
职业分类
未分类
描述

Write, edit, or review TensorRT Edge-LLM documentation using the Google developer documentation style guide. Apply to READMEs, tutorials, how-to guides, conceptual and design documents, API and CLI references, and release notes; not ordinary code changes or…

原文语言:英语

更新
职业分类
未分类
描述

Write, select, run, and verify TensorRT Edge-LLM end-to-end model tests. Use when adding or changing a model case, running one model deployment instead of an entire CI job, interpreting model accuracy or baseline failures, or selecting sufficient tests for…

原文语言:英语

更新
职业分类
未分类
描述

List the supported combinations a change reaches and verify them before the change is submitted. Use before coding a change to quantization or checkpoint parsing, checkpoint loading, the lm_head, KV cache, a speculative decoding method or draft, shared linear…

原文语言:英语

更新
职业分类
未分类
描述

Add or repair a few-layer numeric validation case, which compares the first N decoder layers of a checkpoint against a PyTorch golden. Covers when a change needs one, choosing the checkpoint, setting a cosine gate that is measured rather than tuned, and…

原文语言:英语

更新
职业分类
未分类
描述

Select, install, verify, update, or remove TensorRT Edge-LLM, and install this repository's agent skills into Claude Code or Codex. Inspects revision, CPU architecture, GPU or SoC, JetPack/DriveOS, CUDA, TensorRT, and Python, rejects unsupported combinations…

原文语言:英语

更新
职业分类
未分类
描述

Add or change a TensorRT Edge-LLM operation (TensorRT IPluginV3 plugin, native TensorRT layer composition, or runtime-only operation) so one tensor, dtype, shape, layout, serialization, and weight-layout contract holds across the ONNX export route, the…

原文语言:英语

更新
职业分类
未分类
描述

Launch and verify TensorRT Edge-LLM's experimental OpenAI-compatible server for one Hugging Face checkpoint or local checkpoint. Use for `trtedge serve`, OpenAI API requests, server sizing, streaming, tools, media, batching, speculative decoding, or a server…

原文语言:英语

更新
职业分类
未分类
描述

Migrate an existing TensorRT Edge-LLM deployment, integration, or downstream patch between releases, branches, TensorRT/CUDA/JetPack/DriveOS stacks, or checkpoint-family versions; also backport fixes to a release branch. Use when commands, package extras,…

原文语言:英语

更新
已展示 15 / 15 个已收集 Skill。