edge-llm-benchmark-performance
Benchmark TensorRT Edge-LLM serving performance (TTFT, inter-token latency, request latency, output throughput, concurrency scaling) through the OpenAI-compatible server with AIPerf. Use for "benchmark performance", "measure TTFT/ITL/tokens per second", prefill or decode sweeps over input and output lengths, in-flight-batching concurrency scaling, or comparing two deployments (quantization, speculative decoding, engine profile, another runtime). Do not use to launch a server only (edge-llm-serve), to score dataset accuracy (edge-llm-benchmark-accuracy), or to profile kernels.
Source facts
- Repository
- NVIDIA/TensorRT-Edge-LLM
- Last source activity
- November 13, 2026 at 06:10
- Detected SKILL.md language
- English
- Stars
- 631
- Forks
- 153
Install options
The review-first prompt is selected by default. You can switch to a direct command or download a local copy.
Review the source files
Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.
Showing SKILL.md
| name | edge-llm-benchmark-performance |
|---|---|
| description | Benchmark TensorRT Edge-LLM serving performance (TTFT, inter-token latency, request latency, output throughput, concurrency scaling) through the OpenAI-compatible server with AIPerf. Use for "benchmark performance", "measure TTFT/ITL/tokens per second", prefill or decode sweeps over input and output lengths, in-flight-batching concurrency scaling, or comparing two deployments (quantization, speculative decoding, engine profile, another runtime). Do not use to launch a server only (edge-llm-serve), to score dataset accuracy (edge-llm-benchmark-accuracy), or to profile kernels. |