lmswitch-recipes

name	lmswitch-recipes
description	Author lmswitch model recipes (ai-models/*.yaml) for serving local LLMs via llama-server (GGUF) or vLLM (Docker). Use when adding a new model to lmswitch, editing a recipe, or passing custom serving flags.

Writing lmswitch recipes

lmswitch serves local LLMs from one YAML file per model in ai-models/. The filename minus .yaml is the model id — it's also the --alias / --served-model-name and (for vLLM) the container name vllm-<id>. Drop a file in, run lmswitch on <id>. No code changes needed.

Two runtimes:

• runtime: llama (default) → GGUF via llama-server, background process.
• runtime: vllm → safetensors/quantized via vllm-openai Docker container.

model: is always a path relative to MODELS_DIR (default ~/models, override in ai-models/.lmswitch). For GGUF it points at the .gguf file (or the -00001-of-000NN.gguf first shard of a split); for vLLM it points at the model directory.

Minimal recipes

GGUF (ai-models/my-model.yaml):

runtime: llama
model: "unsloth/Qwen3-4B-GGUF/Qwen3-4B-Q4_K_M.gguf"
port: 8085
ctx: 65536
display_name: