메뉴
Serve MiniCPM5-1B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for "vLLM", "OpenAI server", "REST API for MiniCPM5", or "production deployment".
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | minicpm5-deploy-vllm |
| description | Serve MiniCPM5-1B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for "vLLM", "OpenAI server", "REST API for MiniCPM5", or "production deployment". |
OpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B checkpoint.
| Var | Example | Default |
|---|---|---|
MODEL_PATH | openbmb/MiniCPM5-1B | required |
PORT | 8000 | 8000 |
GPU_ID | 0 | 0 |
CTX_LEN | 131072 (128 K) | 131072; lower if VRAM tight |
MEM_FRAC | 0.85 | 0.85; lower on shared GPUs |
pip install "vllm>=0.21" # latest (CUDA 13.x driver hosts)
# pip install "vllm==0.10.1.1" # fallback for CUDA 12.x driver hosts
CUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve "${MODEL_PATH}" \
--served-model-name MiniCPM5-1B \
--dtype bfloat16 \
--max-model-len ${CTX_LEN} \
--gpu-memory-utilization ${MEM_FRAC} \
--port ${PORT}
Wait for Application startup complete in the log.
curl http://localhost:${PORT}/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "MiniCPM5-1B",
"messages": [{"role": "user", "content": "1+1=?"}],
"temperature": 0.7, "top_p": 0.95, "max_tokens": 64,
"chat_template_kwargs": {"enable_thinking": false}
}'
Expected: choices[0].message.content contains "2". If you see <think>...</think>, you forgot chat_template_kwargs.enable_thinking=false.
{"temperature": 0.9, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": true}} // think
{"temperature": 0.7, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": false}} // nothink
(free / total) < MEM_FRAC hard error: lower --gpu-memory-utilization (e.g. 0.5 on a shared GPU).--max-model-len to 32768 or 8192.The vLLM-side MiniCPM5 XML parser (PR #43175) merged to main on 2026-05-27 but is not in any pip release yet (v0.22.0 was cut before the merge). Use the bridge plugin shipped at tool_parsers/minicpm5xml_tool_parser.py in this repo:
vllm serve "${MODEL_PATH}" \
--served-model-name MiniCPM5-1B \
--dtype bfloat16 --max-model-len ${CTX_LEN} --port ${PORT} \
--enable-auto-tool-choice \
--tool-parser-plugin /path/to/MiniCPM/tool_parsers/minicpm5xml_tool_parser.py \
--tool-call-parser minicpm5
Drop --tool-parser-plugin once vLLM ships a release containing the parser natively.
minicpm5-deploy-transformersminicpm5-deploy-llama-cpp / minicpm5-deploy-mlxminicpm5-deploy-sglangRun MiniCPM5-1B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for "llama.cpp", "llama-cli", "llama-server", "GGUF", or has no Python available.
Fine-tune MiniCPM5-1B into a LoRA adapter and convert it to a GGUF adapter that loads directly into llama.cpp / llama-server and the MiniCPM Desk Pet app's custom-LoRA upload. Use when the user wants "GGUF LoRA", "convert LoRA to GGUF", "convert_lora_to_gguf", a custom persona/skin for the desktop pet, "桌宠自定义 LoRA", "上传 LoRA 到桌宠", or asks how to take a trained adapter and run it on a GGUF base.
Pick the right fine-tuning framework for a MiniCPM5-1B base checkpoint and route to a framework-specific cookbook skill. Use when the user wants to SFT / LoRA / DPO / continue-pretrain MiniCPM5 and has not yet committed to a specific framework, or when they say "fine-tune MiniCPM5", "train MiniCPM5", "MiniCPM5 微调", "LoRA MiniCPM5", "继续训练 MiniCPM5".
Run MiniCPM5-1B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions "LM Studio", desktop GUI inference, "lms" CLI, or wants a no-code chat UI for MiniCPM5.
Run MiniCPM5-1B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for "MLX", "mlx_lm", "mlx_lm.convert", "mlx_lm.generate", or wants the fastest path on Apple Silicon.
Serve MiniCPM5-1B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for "SGLang", "RadixAttention", "prefix cache", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.