serving-llms-vllm

name	serving-llms-vllm
description	Serves LLMs with high throughput using vLLM's PagedAttention and continuous...
version	1.0.0
triggers	["serving llms vllm","serving-llms-vllm","vllm","模型部署","推理加速","LLM推理"]
author	Orchestra Research
license	MIT
dependencies	["vllm","torch","transformers"]
metadata	{"hermes":{"tags":["vLLM","Inference Serving","PagedAttention","Continuous Batching","High Throughput","Production","OpenAI API","Quantization","Tensor Parallelism"]}}
category	mlops

vllm

Serves LLMs with high throughput using vLLM's PagedAttention and continuous...