| name | ollama |
| description | Local LLM runner. One-command setup for Llama, Mistral, Gemma, Qwen, DeepSeek, Phi, and 100+ models. OpenAI-compatible API, model management, GPU acceleration, and custom Modelfile creation. |
| tags | ["ollama","local-llm","llm","inference","api","modelfile","zorai"] |
Overview
Ollama runs LLMs locally with a single command. Supports Llama 3, Mistral, Gemma, Qwen 2.5, DeepSeek, Phi, and 100+ models with GPU acceleration (CUDA/Metal), OpenAI-compatible API, and custom Modelfiles for configuration.
Installation
curl -fsSL https://ollama.com/install.sh | sh
Basic Usage
ollama pull llama3.1:8b
ollama run llama3.1:8b "Explain quantum computing"
Python API
import openai
client = openai.OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
resp = client.chat.completions.create(
model="llama3.1:8b",
messages=[{"role": "user", "content": "What is ML?"}],
)
print(resp.choices[0].message.content)
Custom Modelfile
FROM llama3.1:8b
PARAMETER temperature 0.3
SYSTEM "You are a medical coding assistant."
ollama create my-coder -f Modelfile
References