ollama-herd

Gemma 3 by Google — run Gemma 3 (4B, 12B, 27B) across your local device fleet. Google's most capable open model with 128K context, strong coding, and multilingual support. Fleet-routed to the best available machine via Ollama Herd. Cross-platform (macOS, Linux, Windows). Zero cloud costs.

Mac Studio AI — run LLMs, image generation, speech-to-text, and embeddings on your Mac Studio. M2 Ultra (192GB), M3 Ultra (512GB), M4 Max (128GB), and M4 Ultra (256GB) make the Mac Studio the most powerful local AI device. Load 120B+ models in Mac Studio unified memory. Route across multiple Mac Studios automatically. Mac Studio本地AI推理。Mac Studio IA local.

Mistral and Codestral — run Mistral Large, Mistral-Nemo, Codestral, and Mistral-Small locally. Mistral AI's open-source LLMs for code generation and reasoning. Codestral by Mistral trained on 80+ languages. Mistral routed across your fleet. Mistral本地推理。Mistral IA local. Codestral código local.

MLX-powered local AI — run LLMs, Stable Diffusion, speech-to-text, and embeddings natively on Apple Silicon via MLX. Ollama uses MLX for LLM inference, mflux uses MLX for Flux image generation, DiffusionKit uses MLX for Stable Diffusion 3, and Qwen3-ASR uses MLX for transcription. One fleet router coordinates all four across Mac Studio, Mac Mini, MacBook Pro.

Ollama proxy — one endpoint that routes to multiple Ollama instances. Drop-in Ollama proxy replacement for localhost:11434. Same Ollama API, same model names, but the Ollama proxy routes requests to the best device. Auto-discovers Ollama nodes, scores on 7 signals, retries on failure. Works with Open WebUI, LangChain, Aider. Ollama代理 | proxy Ollama

Private AI — run LLMs, image generation, speech-to-text, and embeddings on your own hardware. Private AI keeps all data on your network. No cloud APIs, no telemetry, no third-party access. Offline AI and air-gapped AI compatible. On-premise AI for privacy, compliance, data sovereignty. HIPAA-friendly private AI, GDPR-ready private AI. 私有AI离线推理。IA privada sin nube.

Qwen 3.5 by Alibaba — run Qwen 3.5 (the latest and most capable Qwen model) across your local device fleet. Qwen 3.5 rivals GPT-4o and Claude 3.5 on reasoning benchmarks. Plus Qwen3-Coder for code generation and Qwen3-ASR for speech-to-text. Fleet-routed to the best available machine via Ollama Herd. Zero cloud costs.

Self-hosted AI — run your own LLM inference, image generation, speech-to-text, and embeddings. No cloud APIs, no SaaS subscriptions, no data leaving your network. Self-hosted alternative to OpenAI, DALL-E, Whisper API, and cloud embedding services. Route across macOS, Linux, and Windows machines. 自托管AI本地推理平台。IA autoalojada sin dependencias en la nube.

Stable Diffusion 3 and SD3.5 Large on Apple Silicon — generate Stable Diffusion images locally with DiffusionKit's MLX-native backend. SD3 Medium for fast Stable Diffusion generation, SD3.5 Large for highest quality. Plus Flux models via mflux and Ollama native image gen. All routed across your device fleet. No cloud APIs, no DALL-E costs. 稳定扩散SD3本地图像生成。Difusion estable SD3 para generacion de imagenes local.

Windows AI — run local AI on Windows with LLM inference, image generation, and embeddings. Windows AI server for Llama, Qwen, DeepSeek, Phi, Mistral. Turn Windows PCs into a Windows AI cluster. No cloud APIs, no subscriptions — Windows AI runs entirely on your hardware. Windows AI本地推理。Windows IA local sin dependencias en la nube.

Apple Silicon AI — run LLMs, image generation, speech-to-text, and embeddings on Mac Studio, Mac Mini, MacBook Pro, and Mac Pro. Turn your Apple Silicon devices into a local AI fleet. M1, M2, M3, M4 Max and Ultra chips with unified memory make these machines ideal for local inference. No cloud APIs, no GPU rentals — your Macs are the cluster. 苹果芯片AI本地推理集群。IA Apple Silicon para inferencia local.

CUDA Ollama — route Ollama LLM inference across NVIDIA GPUs with automatic CUDA load balancing. CUDA Ollama cluster for RTX 4090, RTX 4080, A100, L40S, H100. NVIDIA CUDA Ollama fleet routing with 7-signal scoring, vRAM-aware fallback, and auto-retry. Run Llama, Qwen, DeepSeek, Phi, Mistral on NVIDIA CUDA GPUs. CUDA Ollama本地推理路由。CUDA Ollama enrutador IA NVIDIA.

Home lab AI — turn your spare machines into a local AI home lab cluster. LLM inference, image generation, speech-to-text, and embeddings across macOS, Linux, and Windows devices. Zero-config mDNS discovery, real-time dashboard, 7-signal scoring. No cloud, no Docker, no Kubernetes. The home lab AI setup that just works. 家庭实验室AI本地推理集群。Laboratorio IA para inferencia local en casa.

Linux AI Server — turn Linux servers into a local AI inference cluster. Headless Linux AI with systemd, NVIDIA CUDA, and zero GUI overhead. Linux AI server for Llama, Qwen, DeepSeek, Phi, Mistral. Run a Linux AI server cluster on Ubuntu, Debian, RHEL, Fedora. Linux AI服务器本地推理。Servidor Linux IA para inferencia local.

Linux Ollama — run Ollama on Linux with fleet routing across multiple Linux machines. Linux Ollama setup for Llama, Qwen, DeepSeek, Phi, Mistral. Route Ollama inference across Linux servers, desktops, and edge devices. Linux Ollama load balancing with systemd integration. Linux Ollama本地推理。Linux Ollama enrutador IA.

Llama 3 by Meta — run Llama 3.3, Llama 3.2, and Llama 3.1 across your local device fleet. The most popular open-source LLM family routed to the best available machine. 8B for fast responses, 70B for quality, 405B for frontier performance. OpenAI-compatible API, Cross-platform (macOS, Linux, Windows). Zero cloud costs.

Ollama multimodal model router for Llama, Qwen, DeepSeek, Phi, and Mistral — plus mflux image generation, speech-to-text, and embeddings. Self-hosted Ollama local AI (macOS, Linux, Windows) with 7-signal scoring, Ollama queue management, real-time dashboard, and Ollama health monitoring. Routes Ollama LLM, image, STT, and embedding requests across macOS, Linux, and Windows devices. Ollama本地推理路由 | Ollama enrutador IA local. Use when the user asks about their Ollama fleet, Ollama inference routing, Ollama node status, or Ollama fleet performance.

Ollama load balancer for Llama, Qwen, DeepSeek, and Mistral inference across multiple machines. Load balancing with auto-discovery via mDNS, health checks, queue management, automatic failover, retry on node failure, and zombie request cleanup. Zero configuration. 负载均衡Ollama推理分发。Balanceador de carga Ollama para inferencia distribuida.

Ubuntu AI — build a local AI platform on Ubuntu and Debian. Ubuntu AI across x86 desktops, ARM edge devices, Raspberry Pi, Jetson Orin, and cloud VMs. Heterogeneous Ubuntu AI cluster with NVIDIA CUDA, AMD ROCm, and CPU inference. LLM, image generation, and embeddings on Ubuntu. Debian AI with the same stack. Ubuntu AI本地推理平台。Ubuntu IA plataforma local.

Ubuntu Ollama — run Ollama on Ubuntu with fleet routing across multiple Ubuntu machines. Ubuntu Ollama setup with apt, systemd, and NVIDIA CUDA. Route Ollama inference across Ubuntu servers and desktops. Ubuntu Ollama load balancing, auto-discovery, and health monitoring. Ubuntu Ollama本地推理。Ubuntu Ollama enrutador IA.

RTX Local AI — turn your gaming PC into a local AI server. RTX 4090, RTX 4080, RTX 4070, RTX 3090 run Llama, Qwen, DeepSeek, Phi, Mistral locally. Gaming PC AI inference with NVIDIA RTX GPUs via Ollama Herd. No cloud costs — your RTX GPU is the AI server. RTX本地AI推理。RTX IA local en tu PC gaming.

Windows Ollama — run Ollama on Windows with fleet routing across multiple Windows PCs. Windows Ollama setup for Llama, Qwen, DeepSeek, Phi, Mistral. Route Ollama inference across Windows machines with NVIDIA RTX GPUs. Windows Ollama load balancing, health monitoring, and real-time dashboard. Windows Ollama本地推理。Windows Ollama enrutador IA.

WSL2 Local AI — run LLMs on Windows via WSL2 with NVIDIA GPU passthrough. WSL2 AI development with Ollama, CUDA, and Docker. WSL2 Ollama fleet routing for Windows developers. Build AI apps on WSL2 with full Linux performance and Windows convenience. WSL2本地AI开发。WSL2 IA local para desarrolladores Windows.

DevOps observability toolkit for local AI fleet operations. DevOps traces, DevOps health checks, DevOps latency monitoring, DevOps capacity planning, and DevOps analytics — all backed by SQLite. No Prometheus, no Grafana. DevOps运维工具 | herramientas DevOps

DeepSeek DeepSeek-Coder — run DeepSeek-V3, DeepSeek-R1, DeepSeek-Coder across your local fleet. 7-signal scoring routes every request to the best device. Cross-platform (macOS, Linux, Windows). Zero cloud costs via Ollama Herd.

Distributed inference for Llama, Qwen, DeepSeek across heterogeneous hardware. Self-hosted distributed inference — scatter requests across macOS, Linux, Windows, and any machine running Ollama. Thermal-aware distributed inference scheduling, 7-signal distributed inference scoring, adaptive capacity learning, context-aware model placement. No orchestration layer, no container runtime — just HTTP and mDNS. 分布式推理 across local devices | Inferencia distribuida en hardware local.

Embeddings with nomic-embed-text, mxbai-embed, and snowflake-arctic-embed across your device fleet. Fleet-routed via Ollama for RAG, semantic search, and vector similarity. Batch embed thousands of documents across nodes instead of bottlenecking on one machine. Use when the user needs to create embeddings, build a knowledge base, or set up semantic search.

GPU cluster manager for local AI — run Llama, Qwen, DeepSeek, and Phi across macOS, Linux, and Windows devices with one endpoint. Self-hosted local AI GPU cluster. Auto-discovers machines via mDNS, routes to the best device, manages queues. Zero config, zero Docker. GPU集群管理本地AI推理。Clúster GPU para inferencia IA local.

Local coding assistant — run DeepSeek-Coder, Codestral, StarCoder, and Qwen-Coder across your device fleet. Code generation, review, refactoring, and debugging routed to the best available machine. Works with Aider, Continue.dev, Cline, and any OpenAI-compatible coding tool. No cloud API costs, all code stays local.

Local LLM model router for Llama, Qwen, DeepSeek, Phi, Mistral, and Gemma across multiple devices. Self-hosted local LLM inference routing on macOS, Linux, and Windows. Local LLM 7-signal scoring engine picks the optimal machine for every local LLM request. OpenAI-compatible local LLM API with context protection, VRAM-aware fallback, and auto-retry. 本地LLM路由 inference router | LLM local enrutador de inferencia. Use when the user wants to optimize local LLM routing, reduce local LLM latency, or load balance local LLM across machines.

Local speech-to-text transcription with Qwen ASR — transcription routed across your Apple Silicon fleet. Transcribe meetings, voice notes, podcasts with local speech-to-text. Works like Whisper but runs locally via MLX. Fleet-routed transcription with queue management and dashboard visibility. 语音转文字 | transcripción de voz

Local mflux image generation on Apple Silicon — mflux routes Z-Image-Turbo, Flux Dev, Flux Schnell across your Mac fleet. mflux is MLX-native for Mac Studio, Mac Mini, MacBook Pro. mflux generates images in ~7s at 512px, ~18s at 1024px. Fleet-routed mflux with queue management. mflux图像生成 | generación de imágenes mflux

Manage Ollama models — Ollama Llama, Qwen, DeepSeek, Phi, Mistral — across your machines. See what's loaded in Ollama, what's eating disk, what's never used. Pull, delete, and organize Ollama models from one place. AI-powered Ollama recommendations for the optimal model mix based on your hardware. Ollama本地模型管理 | Ollama gestor de modelos IA.

Ollama Ollama Herd — multimodal Ollama model router that herds your Ollama LLMs into one smart Ollama endpoint. Route Ollama Llama, Qwen, DeepSeek, Phi, Mistral across macOS, Linux, and Windows devices. Self-hosted Ollama local AI with 7-signal Ollama scoring, Ollama auto-retry, VRAM-aware Ollama fallback. Plus Ollama image generation, speech-to-text, and embeddings. Drop-in OpenAI SDK compatible. Ollama本地推理路由 | Ollama enrutador IA local.

Phi 4 by Microsoft — small but powerful LLMs that run on minimal hardware. Phi-4 (14B), Phi-4-mini (3.8B), and Phi-3.5 across your device fleet. Perfect for low-RAM devices on any platform. State-of-the-art reasoning in a tiny footprint. Zero cloud costs.

Qwen Qwen3 — run Qwen3.5, Qwen3, Qwen3-Coder, Qwen2.5-Coder, and Qwen3-ASR across your local fleet. LLM inference, code generation, and speech-to-text from Alibaba's Qwen family via Ollama Herd. Cross-platform (macOS, Linux, Windows). Zero cloud costs.

Mac Mini AI — run LLMs, image generation, speech-to-text, and embeddings on your Mac Mini. M4 (16-32GB) and M4 Pro (24-64GB) configurations make the Mac Mini the most affordable entry point for local AI. Stack multiple Mac Minis into a fleet for the cost of one cloud GPU. Route requests across all your Mac Minis automatically.