ollama

name	ollama
description	Deploy and serve local models with Ollama — pull and run them, then expose the OpenAI-compatible endpoint to apps and agents.
short_description	Run local models with Ollama.
short_description_zh	用 Ollama 运行本地模型。
version	2
updated	2026-08-23T15:42:44.000Z

Ollama Serving

Ollama runs open-weight models locally with automatic GPU detection and an OpenAI-compatible API on http://localhost:11434.

Before you start

If the user's message only invokes this skill (e.g. "use ollama skill") without a concrete request, ask the user what they want. Do not run any command until the goal is clear.

Ask the user which model to run; if they have no preference, recommend the small default Qwen/Qwen3.5-0.8B (ollama pull qwen3.5:0.8b). The model must fit the machine's RAM/VRAM.

Ollama runs everywhere — macOS, Linux and Windows, on CPUs as well as NVIDIA/AMD GPUs — so engine choice follows the user's preference: Ollama is the simple default, while vLLM targets high-throughput GPU serving. Check the current state first:

ollama --version   # is Ollama installed?
ollama ps          # is the service already serving models?

If port 11434 is already serving, reuse that instance — never kill an existing Ollama process.

Suggested workflow

1. Ask the user which model to run; with no preference, recommend Qwen/Qwen3.5-0.8B (qwen3.5:0.8b).
2. Pick the engine the user prefers: Ollama is the default; vLLM covers high-throughput GPU serving.
3. Install Ollama if missing, then ollama pull qwen3.5:0.8b.
4. Verify with curl http://localhost:11434/v1/models.
5. Register the endpoint: penguin config model add ... --client-type openai-chat --base-url http://localhost:11434/v1 — a pulled Ollama model is not visible to Penguin until added.
6. Confirm the new entry with penguin config model list.

Install

curl -fsSL https://ollama.com/install.sh | sh   # Linux; macOS/Windows use the desktop app

The service then listens on http://localhost:11434.

Pull and run

ollama pull qwen3.5:0.8b   # download a model
ollama run qwen3.5:0.8b    # interactive chat (pulls first if missing)
ollama list                # downloaded models
ollama ps                  # models loaded in memory
ollama stop qwen3.5:0.8b   # unload a model

OpenAI-compatible endpoint

The endpoint is http://localhost:11434/v1; any non-empty API key is accepted (conventionally ollama):

curl http://localhost:11434/v1/models

Context length

The default context window is small, and agent sessions need a large one. Raise it in the server's environment:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve   # systemd service: set it via `systemctl edit ollama`

Or bake it into a model variant with a Modelfile:

FROM qwen3.5:0.8b
PARAMETER num_ctx 32768

ollama create qwen3.5-32k -f Modelfile

Register with PenguinHarness

Model configuration is the penguin CLI's job — penguin config model add registers an endpoint and penguin config model list shows what has been registered. A pulled Ollama model is not visible to Penguin until you add it:

penguin config model add --provider custom --client-type openai-chat \
  --base-url http://localhost:11434/v1 --model-id qwen3.5:0.8b --api-key ollama
penguin config model list   # the new entry should now be listed