メニュー
Deploy and serve LLMs with vLLM behind an OpenAI-compatible endpoint, with tool calling enabled for agent workloads.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/Prism-Shadow/penguin-harness --skill vllmコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SKILL.md を表示中
Use when developing PenguinHarness itself — changing packages/{core,server,web,cli,desktop,landing,docs,skills}, the built-in model catalog, the installers or the release workflow; writing or auditing changelog entries; writing a blog post or capturing release screenshots; deciding what to do about data already on disk; or auditing prose that reads like a leaked authoring session. Covers the two-repo symlink layout, the CI-parity verification chain, the record-and-ship contract, where blog media is hosted, and the seams that are intentional.
Use whenever the user wants to build an agent application — their own program with an embedded agent, such as an AI app, an agentic app or a RAG app. This is writing application code on the Penguin Harness SDK, not configuring an Agent State inside PenguinHarness. Covers self-contained projects, the createSession/run streaming loop with thinking and image messages, wiring the user's existing tools in as CLI commands, and a complete RAG recipe that ingests documents into a knowledge base and answers with citations behind a web UI.
Drive PenguinHarness itself from a shell — list and create agents and sessions, send and steer messages mid-flight, and query costs and scheduled tasks via the penguin CLI over the local server.
| name | vllm |
| description | Deploy and serve LLMs with vLLM behind an OpenAI-compatible endpoint, with tool calling enabled for agent workloads. |
| short_description | Serve models locally with vLLM. |
| short_description_zh | 用 vLLM 部署本地模型服务。 |
| version | 2 |
| updated | 2026-08-23T15:42:44.000Z |
vLLM serves open-weight LLMs on local GPUs with high-throughput inference behind an OpenAI-compatible API, ready for chat and agent workloads.
If the user's message only invokes this skill (e.g. "use vllm skill") without a concrete request, ask the user what they want. Do not run any command until the goal is clear.
Ask the user which model to serve; if they have no preference, recommend the small default Qwen/Qwen3.5-0.8B. Also ask what context length the workload needs.
vLLM needs an NVIDIA or AMD GPU. Engine choice follows the user's preference: Ollama also runs on GPUs and is the simpler default — pick vLLM for high-throughput serving, and Ollama on macOS or CPU-only machines, which vLLM does not serve. Confirm the hardware first:
nvidia-smi # NVIDIA: GPU model and free VRAM (AMD ROCm: rocm-smi)
python3 --version # a recent Python is required
The model must fit the available VRAM — model size and context length drive the serve flags below.
curl http://localhost:8000/v1/models.penguin config model add ... --client-type openai-chat --base-url http://localhost:8000/v1 — a served model is not visible to Penguin until added.penguin config model list.Use a fresh virtual environment (or uv):
python3 -m venv .venv && source .venv/bin/activate
pip install vllm
vllm serve Qwen/Qwen3.5-0.8B --port 8000
This exposes an OpenAI-compatible API at http://localhost:8000/v1. Key flags:
--served-model-name <name> — the model id clients request (defaults to the model path).--api-key <key> — require this bearer token on every request.--max-model-len <n> — context window; agent sessions need a large one.--gpu-memory-utilization <0..1> — fraction of VRAM to claim (default 0.9).--tensor-parallel-size <n> — shard across n GPUs.--dtype <auto|bfloat16|float16> and --quantization <awq|gptq|fp8> — precision and quantized weights.If the port is taken, pick a free one — never kill a process already listening on it.
Agent harnesses (PenguinHarness included) send tools with their requests. vLLM must opt in at startup:
vllm serve Qwen/Qwen3.5-0.8B --enable-auto-tool-choice --tool-call-parser hermes
Choose the parser for the model family — e.g. hermes for Qwen models, llama3_json for Llama models. Without these flags, requests that set tool_choice fail with 400 "auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set.
curl http://localhost:8000/v1/models
Model configuration is the penguin CLI's job — penguin config model add registers an endpoint and penguin config model list shows what has been registered. A served model is not visible to Penguin until you add it:
penguin config model add --provider custom --client-type openai-chat \
--base-url http://localhost:8000/v1 --model-id <served-model-name> --api-key <key>
penguin config model list # the new entry should now be listed
--gpu-memory-utilization or --max-model-len, or serve a quantized model.--max-model-len (bounded by VRAM).400 on tool calls: restart the server with the tool-calling flags above.