vllm-project/vllm-skills

vllm-bench-random-synthetic

Run vLLM performance benchmark using synthetic random data to measure throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and other key performance metrics. Use when the user wants to quickly test vLLM serving performance without downloading…

لغة النص الأصلي: الإنجليزية

vllm-bench-serve

Benchmark vLLM or OpenAI-compatible serving endpoints using vllm bench serve. Supports multiple datasets (random, sharegpt, sonnet, HF), backends (openai, openai-chat, vllm-pooling, embeddings), throughput/latency testing with request-rate control, and result…

لغة النص الأصلي: الإنجليزية

vllm-deploy-docker

Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.

لغة النص الأصلي: الإنجليزية

vllm-deploy-k8s

Deploy vLLM to Kubernetes (K8s) with GPU support, health probes, and OpenAI-compatible API endpoint. Use this skill whenever the user wants to deploy, run, or serve vLLM on a Kubernetes cluster, including creating deployments, services, checking existing…

لغة النص الأصلي: الإنجليزية

vllm-deploy-simple

Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.

لغة النص الأصلي: الإنجليزية

vllm-prefix-cache-bench

This is a skill for benchmarking the efficiency of automatic prefix caching in vLLM using fixed prompts, real-world datasets, or synthetic prefix/suffix patterns. Use when the user asks to benchmark prefix caching hit rate, caching efficiency, or…

لغة النص الأصلي: الإنجليزية