菜单
Discover, audit and use local LLM servers (LM Studio, Ollama, LocalAI, vLLM, llama.cpp) on this machine or the network to offload work from the cloud and save tokens. Use when the user mentions local models, LM Studio, Ollama, "run it locally", token savings via local hardware, or wants to know what models their machine can run.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Memory as a learnable skill — store, recall, compress, and consolidate agent memories. Inspired by Stanford AutoMem.
Auto-decide whether a task runs on a LOCAL model or a CLOUD model (DeepSeek, GLM, Nemotron, Grok, Gemma, …) from an automatic effort estimate, and run multi-model fusion (cascade, draft→refine, vote). Use when the user wants automatic local-vs-cloud routing, to add cloud LLM providers, to make local and cloud models collaborate, or mentions effort-based routing, model fusion/ensemble, OpenRouter, DeepSeek, GLM, Nemotron, or Grok.
Fenêtres de contexte pour boucles rétroactives — charge seulement les deltas.
Optimisations DAG/RAG — waves, pruning, memoization, routing.
Generate one self-contained, timestamped HTML dashboard of the system's cost picture — routing savings (control loop), metric trends, current metrics, and the cost of outstanding fixes. Also renders as a live ANSI terminal view (--tui, --watch) and serves a live HTTP API (api.py). Use when the user wants a single visual view of cost/savings/health over time, or a live terminal view they don't have to open a browser for.
Append-only JSONL decision log (.botte/events.jsonl) that every filter in the belt writes to — routing, cache hits, escalations, micro-NN outputs. The single source of truth demo mode, the live dashboard, and session replay all read from. Use when you want to see or emit a live feed of routing/cache/escalation decisions, or when building a tool that needs to watch the belt work in real time.
| name | llm_backends |
| description | Discover, audit and use local LLM servers (LM Studio, Ollama, LocalAI, vLLM, llama.cpp) on this machine or the network to offload work from the cloud and save tokens. Use when the user mentions local models, LM Studio, Ollama, "run it locally", token savings via local hardware, or wants to know what models their machine can run. |
Turn idle local hardware into a token-saving tier. Every task served by a local model is a task not billed to the cloud.
# Discover + register backends (writes configs/llm-endpoints.json)
python -m skills.llm_backends.cli scan # localhost only
python -m skills.llm_backends.cli scan --subnet # sweep local /24
python -m skills.llm_backends.cli scan 192.168.1.47 # specific host(s)
# What's registered?
python -m skills.llm_backends.cli list
# Audit: are local models used? what can this machine run? next steps?
python -m skills.llm_backends.cli audit --fresh
# Run a prompt locally (0 cloud tokens)
python -m skills.llm_backends.cli chat "classify: bug or feature?" --max-tokens 128
# Suggest a local model for a project (adaptive per project type)
python -m skills.llm_backends.cli profile ~/my-project
from skills.llm_backends import registry, quick_chat, audit
registry.refresh() # discover + persist
best = registry.best_chat_backend() # lowest-latency chat backend
model = registry.preferred_model(best) # coder/instruct over voice/reasoning
res = quick_chat("summarize in 1 line: ...", max_tokens=200)
print(res.text, res.total_tokens) # all local — no cloud cost
| Backend | Default port | API |
|---|---|---|
| LM Studio | 1234 | OpenAI /v1 |
| Ollama | 11434 | native /api/tags + OpenAI /v1 |
| LocalAI | 8080 | OpenAI /v1 |
| vLLM | 8000 | OpenAI /v1 |
| Jan / KoboldCpp / text-gen-webui | 1337 / 5001 / 5000 | OpenAI /v1 |
| ComfyUI | 8188 | image gen |
| Qdrant | 6333 | vector search |
tiered_router L0/L1) to a local model.Related: [[llm_mcp]] (MCP tools for agents), tiered_router (cost tiers),
local_router (task→backend mapping), response_cache (skip repeated calls).