# llm-skills
Agent skills for choosing, running, connecting, improving and operating LLMs and generative-AI models: on a laptop, on your own GPU servers (NVIDIA DGX Spark first-class), or in the cloud. Each skill teaches an AI coding agent to inspect the live system first, follow verified upstream documentation, never print secrets, and measure before and after every change. The skills work with any agent that supports the [Agent Skills](https://agentskills.io/specification) format (Cursor, Claude Code, Codex, and others) and install with the [skills CLI](https://github.com/vercel-labs/skills).
## Install
```bash
npx skills add ScaleDockHQ/llm-skills
```
The CLI lists every skill in this repo and asks which ones to install (space to toggle). To skip the picker:
```bash
# See what's available
npx skills add ScaleDockHQ/llm-skills --list
# Install one or more skills by name
npx skills add ScaleDockHQ/llm-skills --skill local-llms
# Same thing, shorter
npx skills add ScaleDockHQ/llm-skills@local-llms
```
Add `-g` to install globally (user level) instead of per project. Run `npx skills update` to pull the latest versions.
## Skills
Every skill has a `kind` (see `metadata.json`):
| Kind | What it is |
| -------------- | ---------------------------------------------------------------------------- |
| `workflow` | Decide, run, connect, improve or operate: the entry points, listed below. |
| `product` | One tool or service in depth (for example an inference engine or a gateway). |
| `platform` | A hardware platform or a cloud provider. |
| `technique` | A cross-product architecture technique (for example speculative decoding). |
| `model-family` | Chat templates, parsers and recommended quants for a model family. |
| `stack` | A named, opinionated combination of products. |
Workflow skills are the entry points. Product, platform, technique and model-family skills go deeper on one subject. Install any of them with `npx skills add ScaleDockHQ/llm-skills@<name>`.
### Workflow skills
#### Decide
| Skill | Description |
| ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| [`choosing-llm-hosting`](skills/choosing-llm-hosting) | Decide where models should run (local, owned server, rented GPUs, serverless, APIs or hybrid) and record the decision. |
#### Run
| Skill | Description |
| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`local-llms`](skills/local-llms) | Run models on a laptop or desktop with Ollama, LM Studio, llama.cpp or MLX. |
| [`self-hosted-llms`](skills/self-hosted-llms) | Serve models on your own NVIDIA Linux servers and DGX Spark as one authenticated endpoint (engines, llama-swap, LiteLLM, Cloudflare Tunnel or Tailscale). |
| [`cloud-llms`](skills/cloud-llms) | Run and consume models in the cloud: APIs, GPU VMs, serverless GPUs and managed endpoints, with budgets and teardown. |
| [`self-hosted-genmedia`](skills/self-hosted-genmedia) | Self-host image, video, speech-to-text and text-to-speech servers next to your LLMs. |
#### Connect
| Skill | Description |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| [`llm-gateway`](skills/llm-gateway) | One OpenAI- and Anthropic-compatible gateway in front of many providers: keys, budgets, routing, fallbacks, API bridging. |
| [`llm-clients`](skills/llm-clients) | Connect coding agents, IDEs, chat UIs and SDKs to any OpenAI- or Anthropic-compatible endpoint. |
#### Improve
| Skill | Description |
| ------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| [`llm-evals`](skills/llm-evals) | Evaluate models and option changes on your own tasks against any OpenAI-compatible endpoint. |
| [`model-quantization`](skills/model-quantization) | Produce and publish quantized checkpoints (GGUF, EXL3, FP8, NVFP4 and more). |
| [`fine-tuning-llms`](skills/fine-tuning-llms) | Fine-tune open-weight models locally or on rented GPUs, then serve the result. |
#### Operate
| Skill | Description |
| ----------------------------------------------- | ----------------------------------------------------------------------------------- |
| [`llm-observability`](skills/llm-observability) | Metrics, tracing, logging, cost tracking and alerting for LLM serving and LLM apps. |
### Product skills
#### Inference engines
| Skill | Description | Install |
| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| [`vllm`](skills/vllm) | Install, serve and tune vLLM, the OpenAI-compatible LLM inference server (`vllm serve`, `vllm bench`, vllm/vllm-openai, nvcr.io/nvidia/vllm). | `npx skills add ScaleDockHQ/llm-skills@vllm` |
| [`sglang`](skills/sglang) | Install, serve, tune and scale SGLang (sglang.launch_server, `sglang serve`) for fast LLM/VLM inference with RadixAttention. | `npx skills add ScaleDockHQ/llm-skills@sglang` |
| [`tensorrt-llm`](skills/tensorrt-llm) | Install, serve, tune and update NVIDIA TensorRT-LLM (trtllm-serve, trtllm-bench, trtllm-eval, LLM API) on NVIDIA GPUs incl. | `npx skills add ScaleDockHQ/llm-skills@tensorrt-llm` |
| [`llama-cpp`](skills/llama-cpp) | Install, serve, tune and update llama.cpp (llama-server, llama-cli, llama-bench, llama-quantize) for GGUF models on Metal, CUDA, Vulkan, ROCm/HIP, SYCL or CPU. | `npx skills add ScaleDockHQ/llm-skills@llama-cpp` |
| [`ollama`](skills/ollama) | Install, run and tune the Ollama local LLM server (ollama pull, run, serve, create, ps) on macOS, Linux, Windows and Docker, with Modelfiles, OLLAMA_* env vars,... | `npx skills add ScaleDockHQ/llm-skills@ollama` |
| [`tensorfold`](skills/tensorfold) | Installs, serves, tunes and troubleshoots TensorFold (`tensorfold-native`), a Zig LLM inference engine for Apple Silicon and NVIDIA GPUs with exact speculative decoding. | `npx skills add ScaleDockHQ/llm-skills@tensorfold` |
| [`exllamav3`](skills/exllamav3) | Installs, serves and tunes ExLlamaV3 and TabbyAPI (EXL3 quants, OpenAI API). | `npx skills add ScaleDockHQ/llm-skills@exllamav3` |
| [`ik-llama-cpp`](skills/ik-llama-cpp) | Build, serve and tune ik_llama.cpp, the llama.cpp fork with IQK/trellis quants, fast CPU and hybrid CPU/GPU MoE inference, split-mode graph and MTP. | `npx skills add ScaleDockHQ/llm-skills@ik-llama-cpp` |
| [`lmdeploy`](skills/lmdeploy) | Install, serve and tune LMDeploy, the InternLM LLM/VLM inference toolkit with TurboMind and PyTorch engines (`lmdeploy serve api_server`, `lmdeploy lite`, openmmlab/lmdeploy). | `npx skills add ScaleDockHQ/llm-skills@lmdeploy` |
| [`koboldcpp`](skills/koboldcpp) | Install, run and tune KoboldCpp, the single-file llama.cpp-based GGUF server with the KoboldAI Lite UI, OpenAI, Ollama and Anthropic-style APIs plus image, speech, music and MCP. | `npx skills add ScaleDockHQ/llm-skills@koboldcpp` |
| [`xinference`](skills/xinference) | Install, serve and operate Xorbits Inference (Xinference), the Python model server and cluster for LLM, embedding, rerank, image, audio and video models behind OpenAI, Responses and Anthropic APIs. Use for xinference-local, xinference launch, xinference-supervisor/worker, xprobe/xinference Docker or Helm, vLLM/SGLang/llama.cpp/MLX engines, auth and API keys, replicas, Xavier, P/D, upgrades. Fixes "model_engine" missing, 401, virtualenv install, GPU OOM. Not for choosing an engine (choosing-inference-engine). | `npx skills add ScaleDockHQ/llm-skills@xinference` |
| [`llamafile`](skills/llamafile) | Run and package LLMs as single-file executables with llamafile, with chat, CLI and OpenAI compatible server modes | `npx skills add ScaleDockHQ/llm-skills@llamafile` |
| [`mlc-llm`](skills/mlc-llm) | Install, serve and compile models with MLC LLM (MLCEngine on TVM): mlc_llm serve/chat/convert_weight/compile on CUDA, ROCm, Vulkan, Metal, WebGPU and mobile. | `npx skills add ScaleDockHQ/llm-skills@mlc-llm` |
| [`mistral-rs`](skills/mistral-rs) | Rust LLM inference engine with ISQ/UQFF quantization, paged attention, OpenAI, Responses and Anthropic APIs and a web UI | `npx skills add ScaleDockHQ/llm-skills@mistral-rs` |
| [`localai`](skills/localai) | Install, run and operate LocalAI, the self-hosted OpenAI, Anthropic and Ollama compatible server with gallery and backends | `npx skills add ScaleDockHQ/llm-skills@localai` |
| [`aphrodite-engine`](skills/aphrodite-engine) | Install, serve and tune Aphrodite Engine (now branded Sonar), the vLLM-derived LLM inference server with OpenAI, Anthropic and Kobold APIs. Use for the aphrodite CLI, aphrodite serve, aphrodite-engine pip package, sonar.dphn.ai, port 2242, quantization (exl3, gptq, awq, fp8), speculative decoding, LoRA, multi-node, upgrades. Fixes CUDA OOM, unsupported architecture, parser errors. Not for choosing an engine (choosing-inference-engine). | `npx skills add ScaleDockHQ/llm-skills@aphrodite-engine` |
| [`ktransformers`](skills/ktransformers) | CPU+GPU hybrid inference for huge MoE models with kt-kernel, the sglang-kt fork and the kt CLI | `npx skills add ScaleDockHQ/llm-skills@ktransformers` |
| [`nvidia-dynamo`](skills/nvidia-dynamo) | Deploy and operate NVIDIA Dynamo, the datacenter-scale LLM serving layer above vLLM, SGLang and TensorRT-LLM (python3 -m dynamo.frontend, dynamo.vllm, DynamoGraphDeployment). Use for disaggregated... | `npx skills add ScaleDockHQ/llm-skills@nvidia-dynamo` |
| [`nvidia-nim`](skills/nvidia-nim) | Run and operate NVIDIA NIM for LLMs and VLMs, the prebuilt vLLM or SGLang containers from nvcr.io/nim (model-specific or model-free). Use to pull NIM images, choose model profiles, set NIM_* env... | `npx skills add ScaleDockHQ/llm-skills@nvidia-nim` |
| [`triton-inference-server`](skills/triton-inference-server) | Serve models with NVIDIA Triton Inference Server (tritonserver, nvcr.io/nvidia/tritonserver, model repository, config.pbtxt, dynamic batching, perf_analyzer) across TensorRT, ONNX, PyTorch,... | `npx skills add ScaleDockHQ/llm-skills@triton-inference-server` |
#### Local runtimes
| Skill | Description | Install |
| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- |
| [`lm-studio`](skills/lm-studio) | Install, serve, tune and update LM Studio (desktop app, `lms` CLI, llmster headless daemon) for GGUF and MLX models on a local server at :1234. | `npx skills add ScaleDockHQ/llm-skills@lm-studio` |
| [`mlx`](skills/mlx) | Run, serve, convert, quantize and LoRA fine-tune models on Apple Silicon with MLX (mlx-lm, mlx-vlm, mlx_lm.server). | `npx skills add ScaleDockHQ/llm-skills@mlx` |
| [`exo`](skills/exo) | Pool Macs and other devices with exo: install, enable RDMA over Thunderbolt 5, place pipeline or tensor instances, call the :52415 API, update, troubleshoot. | `npx skills add ScaleDockHQ/llm-skills@exo` |
| [`jan`](skills/jan) | Run Jan desktop offline: install, Local API Server on :1337 with key and Trusted Hosts, GGUF and MLX models, custom providers, MCP, Claude Code, update, troubleshoot. | `npx skills add ScaleDockHQ/llm-skills@jan` |
#### Orchestration
| Skill | Description | Install |
| --------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [`llama-swap`](skills/llama-swap) | Install, configure and operate llama-swap, the Go proxy that hot-swaps local models behind one OpenAI/Anthropic-compatible URL. | `npx skills add ScaleDockHQ/llm-skills@llama-swap` |
| [`kserve`](skills/kserve) | Install and operate KServe on Kubernetes for LLMs: LLMInferenceService (llm-d routing, prefill/decode, multi-node, WVA autoscaling, canary) and InferenceService with the vLLM runtime. | `npx skills add ScaleDockHQ/llm-skills@kserve` |
| [`llm-d`](skills/llm-d) | Install and operate llm-d on Kubernetes: Router (EPP) with precise prefix-cache and predicted-latency routing, prefill/decode disaggregation, wide expert parallelism, KV offload and autoscaling. | `npx skills add ScaleDockHQ/llm-skills@llm-d` |
| [`vllm-production-stack`](skills/vllm-production-stack) | Deploy the vLLM production-stack Helm chart on Kubernetes: router plus vLLM engines with LMCache offload, session/prefix/KV-aware routing, KEDA and Grafana. | `npx skills add ScaleDockHQ/llm-skills@vllm-production-stack` |
| [`aibrix`](skills/aibrix) | Install and operate AIBrix on Kubernetes: Envoy gateway with LLM-aware routing, PodAutoscaler, LoRA ModelAdapter, StormService prefill/decode and KV cache offload. | `npx skills add ScaleDockHQ/llm-skills@aibrix` |
| [`ray-serve`](skills/ray-serve) | Serve LLMs with Ray Serve LLM (LLMConfig, build_openai_app, serve CLI, ray-llm images) and run them on Kubernetes with KubeRay RayService. | `npx skills add ScaleDockHQ/llm-skills@ray-serve` |
| [`gateway-api-inference-extension`](skills/gateway-api-inference-extension) | Route LLM traffic to vLLM pods with a Gateway API InferencePool and the llm-d endpoint picker: install, HTTPRoute, scheduling, metrics and troubleshooting. | `npx skills add ScaleDockHQ/llm-skills@gateway-api-inference-extension` |
| [`nvidia-gpu-operator`](skills/nvidia-gpu-operator) | Install and run the NVIDIA GPU Operator on Kubernetes for LLM pods: driver, container toolkit, device plugin, DCGM exporter, MIG and time-slicing. | `npx skills add ScaleDockHQ/llm-skills@nvidia-gpu-operator` |
| [`keda`](skills/keda) | Autoscale LLM serving pods on Kubernetes with KEDA: ScaledObject on vLLM queue depth, KV-cache and GPU metrics, queue scalers, scale to zero. | `npx skills add ScaleDockHQ/llm-skills@keda` |
#### Gateways
| Skill | Description | Install |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| [`litellm`](skills/litellm) | LiteLLM proxy, a self-hosted OpenAI- and Anthropic-compatible gateway to 100+ providers with virtual keys, budgets and fallbacks. | `npx skills add ScaleDockHQ/llm-skills@litellm` |
| [`cloudflare-ai-gateway`](skills/cloudflare-ai-gateway) | Cloudflare AI Gateway, a hosted proxy with logging, caching, rate and spend limits, retries and dynamic routing in front of OpenAI, Anthropic, Workers AI and more. | `npx skills add ScaleDockHQ/llm-skills@cloudflare-ai-gateway` |
| [`vercel-ai-gateway`](skills/vercel-ai-gateway) | Vercel AI Gateway, one hosted OpenAI-, Anthropic- and AI SDK-compatible endpoint with fallbacks, provider routing, BYOK, budgets and zero data retention. | `npx skills add ScaleDockHQ/llm-skills@vercel-ai-gateway` |
| [`bifrost`](skills/bifrost) | Bifrost, a Go open-source AI gateway with a web UI, virtual keys, budgets, fallbacks, semantic cache and MCP. | `npx skills add ScaleDockHQ/llm-skills@bifrost` |
| [`helicone`](skills/helicone) | Helicone, LLM observability and AI gateway, now in maintenance mode, with logging, caching, rate limits and migration paths. | `npx skills add ScaleDockHQ/llm-skills@helicone` |
| [`openrouter`](skills/openrouter) | OpenRouter hosted gateway: one OpenAI-compatible API for hundreds of models with provider routing, fallbacks, presets, BYOK, guardrails and spend limits. | `npx skills add ScaleDockHQ/llm-skills@openrouter` |
| [`portkey`](skills/portkey) | Portkey AI Gateway: open-source Node.js gateway with configs for fallbacks, load balancing, retries and conditional routing, plus the hosted Portkey platform. | `npx skills add ScaleDockHQ/llm-skills@portkey` |
| [`envoy-ai-gateway`](skills/envoy-ai-gateway) | Envoy AI Gateway (Agent Router): Kubernetes and standalone aigw gateway for LLM and MCP traffic, with model routing, upstream auth, fallback, token limits and InferencePool. | `npx skills add ScaleDockHQ/llm-skills@envoy-ai-gateway` |
| [`kong-ai-gateway`](skills/kong-ai-gateway) | Kong AI Gateway: ai-proxy on Kong OSS, Enterprise balancing and token limits, and the 2.x Konnect model with kongctl, for OpenAI-compatible LLM traffic. | `npx skills add ScaleDockHQ/llm-skills@kong-ai-gateway` |
#### Exposure and networking
| Skill | Description | Install |
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------ |
| [`cloudflare-tunnel`](skills/cloudflare-tunnel) | Cloudflare Tunnel (cloudflared) publishes a local LLM endpoint at a public HTTPS hostname with no open port, optionally behind Cloudflare Access. | `npx skills add ScaleDockHQ/llm-skills@cloudflare-tunnel` |
| [`tailscale`](skills/tailscale) | Tailscale gives your devices a private WireGuard network and publishes a local LLM endpoint as https://node.tailnet.ts.net via Serve, or to the internet via Funnel. | `npx skills add ScaleDockHQ/llm-skills@tailscale` |
| [`traefik`](skills/traefik) | Front LLM endpoints with Traefik Proxy: Compose install, routers and middlewares, Let's Encrypt, SSE-safe timeouts, forwardAuth, upgrade, troubleshoot. | `npx skills add ScaleDockHQ/llm-skills@traefik` |
| [`caddy`](skills/caddy) | Caddy is a web server and reverse proxy with automatic HTTPS, used as a loopback front for LLM endpoints. | `npx skills add ScaleDockHQ/llm-skills@caddy` |
| [`nccl-rdma-networking`](skills/nccl-rdma-networking) | NCCL over InfiniBand, RoCE and GPUDirect RDMA for multi-node GPU inference and training, verifying the fabric before tuning NCCL. Use to diagnose NET/Socket instead of NET/IB/GDRDMA, slow or hung... | `npx skills add ScaleDockHQ/llm-skills@nccl-rdma-networking` |
#### Quantization
| Skill | Description | Install |
| ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| [`llm-compressor`](skills/llm-compressor) | Quantize LLMs for vLLM with LLM Compressor (llmcompressor, vllm-project/llm-compressor). | `npx skills add ScaleDockHQ/llm-skills@llm-compressor` |
| [`nvidia-modelopt`](skills/nvidia-modelopt) | Quantize LLMs with NVIDIA Model Optimizer (ModelOpt, nvidia-modelopt, NVIDIA/Model-Optimizer, formerly TensorRT-Model-Optimizer). | `npx skills add ScaleDockHQ/llm-skills@nvidia-modelopt` |
| [`gguf-quantization`](skills/gguf-quantization) | Convert models to GGUF and quantize them with llama.cpp (convert_hf_to_gguf.py, llama-imatrix, llama-quantize, KLD checks, gguf-split, Hub upload). | `npx skills add ScaleDockHQ/llm-skills@gguf-quantization` |
| [`bitsandbytes`](skills/bitsandbytes) | Load and train LLMs in 8-bit and 4-bit with bitsandbytes (LLM.int8(), NF4/FP4 QLoRA, 8-bit and paged optimizers) through Transformers, PEFT, TRL and Accelerate. | `npx skills add ScaleDockHQ/llm-skills@bitsandbytes` |
| [`torchao`](skills/torchao) | Quantize, fine-tune and train PyTorch models with torchao (PyTorch AO, pytorch/ao) using quantize_() with Float8, Int8 and Int4 configs, QAT, float8 training and 8-bit or 4-bit optimizers. | `npx skills add ScaleDockHQ/llm-skills@torchao` |
| [`hqq`](skills/hqq) | Quantize LLMs to 8, 4, 3, 2 or 1 bit in minutes with HQQ (Half-Quadratic Quantization, dropbox/hqq, the hqq package), no calibration data needed. | `npx skills add ScaleDockHQ/llm-skills@hqq` |
| [`exl3-quantization`](skills/exl3-quantization) | Converts Hugging Face models to EXL3 with ExLlamaV3's convert.py (QTIP-style, fractional bits per weight) and measures the result. Use for EXL3 quants, exl3 bpw ladders, convert.py... | `npx skills add ScaleDockHQ/llm-skills@exl3-quantization` |
| [`gptqmodel`](skills/gptqmodel) | Quantizes and serves LLMs with GPT-QModel (ModelCloud) using GPTQ, AWQ, ParoQuant, QQQ, GGUF, FP8 and EXL3 from one Python API with Marlin, Machete and Exllama kernels. Use for GPTQModel,... | `npx skills add ScaleDockHQ/llm-skills@gptqmodel` |
#### Fine-tuning
| Skill | Description | Install |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| [`axolotl`](skills/axolotl) | Axolotl is a YAML-driven LLM fine-tuning framework (axolotl CLI) for LoRA, QLoRA, full fine-tuning, SFT, DPO, KTO and GRPO on Hugging Face models. | `npx skills add ScaleDockHQ/llm-skills@axolotl` |
| [`unsloth`](skills/unsloth) | Unsloth is a fast, low-VRAM fine-tuning and RL library (FastLanguageModel, unsloth CLI) with Unsloth Studio and Desktop apps for LoRA, QLoRA, full fine-tuning, SFT,... | `npx skills add ScaleDockHQ/llm-skills@unsloth` |
| [`trl`](skills/trl) | TRL (Hugging Face Transformer Reinforcement Learning) is the post-training library with SFTTrainer, DPOTrainer, GRPOTrainer, RLOOTrainer, KTOTrainer, RewardTrainer... | `npx skills add ScaleDockHQ/llm-skills@trl` |
| [`llama-factory`](skills/llama-factory) | LLaMA-Factory (llamafactory-cli) is the YAML-driven fine-tuning framework with the LLaMA Board web UI for LoRA, QLoRA, full tuning, SFT, DPO, KTO and reward training on 100+ LLMs and VLMs. | `npx skills add ScaleDockHQ/llm-skills@llama-factory` |
| [`peft`](skills/peft) | PEFT is Hugging Face's parameter-efficient fine-tuning library (LoraConfig, get_peft_model, merge_and_unload) for LoRA, QLoRA, DoRA, IA3 and many more adapters on Transformers models. | `npx skills add ScaleDockHQ/llm-skills@peft` |
| [`deepspeed`](skills/deepspeed) | DeepSpeed is Microsoft's training library for large models with ZeRO stages 1-3, CPU/NVMe offload, the deepspeed launcher, ds_config.json, ds_report and zero_to_fp32. | `npx skills add ScaleDockHQ/llm-skills@deepspeed` |
| [`verl`](skills/verl) | verl (Volcano Engine RL) is an RL post-training library for LLMs with PPO, GRPO, DAPO, RLOO and REINFORCE++ on Ray, FSDP/Megatron training and vLLM/SGLang rollout. | `npx skills add ScaleDockHQ/llm-skills@verl` |
| [`nvidia-nemo`](skills/nvidia-nemo) | NVIDIA NeMo Framework is a set of repos: NeMo AutoModel (PyTorch/FSDP2 fine-tuning via the automodel CLI), NeMo Megatron-Bridge (Megatron pretraining, HF conversion) and NeMo RL (GRPO, DPO, SFT on... | `npx skills add ScaleDockHQ/llm-skills@nvidia-nemo` |
| [`openrlhf`](skills/openrlhf) | OpenRLHF is a Ray + vLLM + DeepSpeed RLHF framework for SFT, reward models, DPO and PPO, REINFORCE++, GRPO and RLOO on Hugging Face models. Use to install it (pip, NGC Docker), run `deepspeed... | `npx skills add ScaleDockHQ/llm-skills@openrlhf` |
#### Evals and benchmarks
| Skill | Description | Install |
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| [`promptfoo`](skills/promptfoo) | Run promptfoo (npx promptfoo eval, promptfooconfig.yaml) to test prompts and compare models on a golden set against any OpenAI-compatible endpoint (vLLM, llama.cpp,... | `npx skills add ScaleDockHQ/llm-skills@promptfoo` |
| [`lm-evaluation-harness`](skills/lm-evaluation-harness) | Run EleutherAI's lm-evaluation-harness (lm_eval, lm-eval CLI) for public benchmarks (MMLU, GSM8K, HellaSwag, IFEval) on HF, vLLM or OpenAI-compatible endpoints via... | `npx skills add ScaleDockHQ/llm-skills@lm-evaluation-harness` |
| [`inspect-ai`](skills/inspect-ai) | Run Inspect AI (UK AISI) evals with the `inspect eval` CLI, Python Task/dataset/solver/scorer code, model_graded_qa judges, log files and Inspect View, again... | `npx skills add ScaleDockHQ/llm-skills@inspect-ai` |
| [`deepeval`](skills/deepeval) | Run DeepEval (`deepeval test run`, pytest-based LLM unit tests) with GEval, answer relevancy, faithfulness and other metrics, using a local or OpenAI-compatible judge (vLLM, Ollama, LiteLLM)... | `npx skills add ScaleDockHQ/llm-skills@deepeval` |
| [`lighteval`](skills/lighteval) | Run Hugging Face Lighteval benchmarks with the lighteval CLI (lighteval eval, vllm, endpoint litellm, accelerate) on MMLU, GSM8K, GPQA, IFEval and 1000+ tasks, against vLLM, LiteLLM or... | `npx skills add ScaleDockHQ/llm-skills@lighteval` |
| [`nvidia-aiperf`](skills/nvidia-aiperf) | Benchmark LLM serving with NVIDIA AIPerf (aiperf profile) for TTFT, ITL, throughput, goodput, concurrency and request-rate sweeps against any OpenAI-compatible endpoin... | `npx skills add ScaleDockHQ/llm-skills@nvidia-aiperf` |
| [`guidellm`](skills/guidellm) | Benchmark LLM servers with GuideLLM (guidellm run) for SLO-aware latency and throughput using sweep, concurrent, constant and Poisson profiles, TTFT/ITL, goodput and t... | `npx skills add ScaleDockHQ/llm-skills@guidellm` |
| [`evalplus`](skills/evalplus) | Score code-generation LLMs on HumanEval+, MBPP+ and EvalPerf with EvalPlus (evalplus.evaluate, evalplus.codegen, evalplus.sanitize) as pass@1 on base vs extended tests... | `npx skills add ScaleDockHQ/llm-skills@evalplus` |
| [`swe-bench`](skills/swe-bench) | Grade coding-agent patches on SWE-bench (Verified, Lite, Multilingual, Multimodal, full) with the swebench CLI v5 (swebench eval, report, images, dataset), Docker-isol... | `npx skills add ScaleDockHQ/llm-skills@swe-bench` |
#### Observability
| Skill | Description | Install |
| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`langfuse`](skills/langfuse) | Langfuse, open-source LLM observability with traces, prompt management, evals and scores. | `npx skills add ScaleDockHQ/llm-skills@langfuse` |
| [`sparkdash`](skills/sparkdash) | sparkDash, a web dashboard for DGX Spark and NVIDIA GPU hosts showing GPU, memory, network, RoCE, LLM engine probes, benchmarks and power controls. | `npx skills add ScaleDockHQ/llm-skills@sparkdash` |
| [`arize-phoenix`](skills/arize-phoenix) | Arize Phoenix, self-hosted LLM tracing, evals and prompts: install, auth, OpenInference and OTLP ingest, LiteLLM and vLLM, retention. | `npx skills add ScaleDockHQ/llm-skills@arize-phoenix` |
| [`openllmetry`](skills/openllmetry) | OpenLLMetry (Traceloop SDK), OpenTelemetry instrumentation for LLM apps sending spans to Phoenix, Langfuse, MLflow or a Collector. | `npx skills add ScaleDockHQ/llm-skills@openllmetry` |
| [`opentelemetry-genai`](skills/opentelemetry-genai) | OpenTelemetry GenAI semantic conventions, official instrumentation and a Collector that strips content, samples and fans out traces. | `npx skills add ScaleDockHQ/llm-skills@opentelemetry-genai` |
| [`mlflow-tracing`](skills/mlflow-tracing) | Self-host MLflow as an LLM tracing backend: install, auth, OTLP ingest and autolog, LiteLLM and vLLM, masking, archival, upgrades. | `npx skills add ScaleDockHQ/llm-skills@mlflow-tracing` |
| [`prometheus-grafana-llm`](skills/prometheus-grafana-llm) | Prometheus, Alertmanager and Grafana for LLM serving metrics. | `npx skills add ScaleDockHQ/llm-skills@prometheus-grafana-llm` |
| [`dcgm-exporter`](skills/dcgm-exporter) | NVIDIA DCGM Exporter, the Prometheus exporter for GPU telemetry (utilization, memory, power, temperature, XID errors, profiling counters). | `npx skills add ScaleDockHQ/llm-skills@dcgm-exporter` |
| [`weave`](skills/weave) | W&B Weave, Weights & Biases' toolkit for tracing and evaluating LLM apps and agents (weave.init, @weave.op, Evaluations). | `npx skills add ScaleDockHQ/llm-skills@weave` |
| [`posthog-llm-analytics`](skills/posthog-llm-analytics) | "PostHog AI Observability (formerly LLM analytics): traces, cost, latency and evaluations built on $ai_generation events. | `npx skills add ScaleDockHQ/llm-skills@posthog-llm-analytics` |
| [`datadog-llm-observability`](skills/datadog-llm-observability) | "Datadog Agent Observability (formerly LLM Observability): traces, cost, evaluations and ml_obs metrics for LLM apps. | `npx skills add ScaleDockHQ/llm-skills@datadog-llm-observability` |
| [`sentry-ai-monitoring`](skills/sentry-ai-monitoring) | "Sentry AI monitoring (Agent Tracing, formerly AI Insights): gen_ai spans, Conversations, model cost and agent dashboards next to errors and traces. | `npx skills add ScaleDockHQ/llm-skills@sentry-ai-monitoring` |
#### Clients and SDKs
| Skill | Description | Install |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| [`claude-code`](skills/claude-code) | Connect Claude Code to self-hosted and gateway models over the Anthropic Messages API: base URL, keys, model slots, limits, compatibility switches, troubleshooting. | `npx skills add ScaleDockHQ/llm-skills@claude-code` |
| [`codex-cli`](skills/codex-cli) | Connect the Codex CLI to self-hosted and gateway models with custom Responses providers: config.toml, profiles, keys, limits, known compatibility issues. | `npx skills add ScaleDockHQ/llm-skills@codex-cli` |
| [`opencode`](skills/opencode) | Connect OpenCode to self-hosted and gateway models with custom OpenAI- and Anthropic-compatible providers: opencode.json, keys, limits, v1 and v2 config, tool calling, troubleshooting. | `npx skills add ScaleDockHQ/llm-skills@opencode` |
| [`aider`](skills/aider) | Connect Aider to self-hosted and gateway models: OPENAI_API_BASE with the openai/ prefix, Ollama, keys in .env, model settings, edit formats, troubleshooting. | `npx skills add ScaleDockHQ/llm-skills@aider` |
| [`vercel-ai-sdk`](skills/vercel-ai-sdk) | Configure the Vercel AI SDK (ai, @ai-sdk/*) against self-hosted OpenAI- and Anthropic-compatible endpoints: keys, model aliases, tools, structured output, streaming. | `npx skills add ScaleDockHQ/llm-skills@vercel-ai-sdk` |
| [`zed`](skills/zed) | Connect Zed (Agent Panel, Inline Assistant, edit prediction) to self-hosted OpenAI- and Anthropic-compatible endpoints: providers, model mapping, keys, tools. | `npx skills add ScaleDockHQ/llm-skills@zed` |
| [`cursor`](skills/cursor) | Connect Cursor to self-hosted OpenAI-compatible endpoints (Override OpenAI Base URL, public HTTPS requirement, scoped keys, model names, tools). | `npx skills add ScaleDockHQ/llm-skills@cursor` |
| [`continue`](skills/continue) | Configure Continue (VS Code, JetBrains, cn CLI) for self-hosted OpenAI- and Anthropic-compatible endpoints: config.yaml, apiBase, secrets, capabilities. | `npx skills add ScaleDockHQ/llm-skills@continue` |
| [`cline`](skills/cline) | Configure Cline (VS Code, JetBrains, CLI) for self-hosted endpoints: OpenAI Compatible, Anthropic base URL, Ollama, LiteLLM, keys, limits, Compact Prompt. | `npx skills add ScaleDockHQ/llm-skills@cline` |
| [`kilo-code`](skills/kilo-code) | Configure Kilo Code (VS Code, JetBrains, kilo CLI) for self-hosted endpoints: custom providers, kilo.jsonc, wire APIs, keys, limits, tool_call. | `npx skills add ScaleDockHQ/llm-skills@kilo-code` |
| [`pydantic-ai`](skills/pydantic-ai) | Configure Pydantic AI (OpenAIChatModel, VLLMProvider, OllamaProvider, LiteLLMProvider, AnthropicProvider) against self-hosted endpoints: keys, profiles, output modes, tools, streaming. | `npx skills add ScaleDockHQ/llm-skills@pydantic-ai` |
| [`librechat`](skills/librechat) | Run LibreChat, the self-hosted multi-user chat UI, against self-hosted OpenAI- and Anthropic-compatible endpoints: librechat.yaml, keys, model names, tools, agents, MCP. | `npx skills add ScaleDockHQ/llm-skills@librechat` |
| [`anythingllm`](skills/anythingllm) | Run AnythingLLM, the all-in-one chat, RAG and agent app, against self-hosted endpoints: Docker, Generic OpenAI and Ollama providers, keys, context windows, @agent tools, MCP. | `npx skills add ScaleDockHQ/llm-skills@anythingllm` |
| [`sillytavern`](skills/sillytavern) | Connect SillyTavern, the local roleplay and character chat front end, to self-hosted endpoints: Chat and Text Completion, Claude proxy, secrets, post-processing, safe remote access. | `npx skills add ScaleDockHQ/llm-skills@sillytavern` |
| [`openai-sdk`](skills/openai-sdk) | Point the OpenAI Python and Node SDKs at self-hosted endpoints: base URL, keys, model names, Chat Completions vs Responses, streaming, tools, per-engine recipes. | `npx skills add ScaleDockHQ/llm-skills@openai-sdk` |
| [`anthropic-sdk`](skills/anthropic-sdk) | Point the Anthropic Python and TypeScript SDKs at self-hosted Messages endpoints: base URL, auth_token vs api_key, model names, streaming, tools, per-engine recipes. | `npx skills add ScaleDockHQ/llm-skills@anthropic-sdk` |
| [`llamaindex`](skills/llamaindex) | Configure LlamaIndex (OpenAILike, Anthropic, Ollama, OpenAILikeEmbedding) against self-hosted OpenAI- and Anthropic-compatible endpoints: keys, capability flags, structured output, agents, streaming. | `npx skills add ScaleDockHQ/llm-skills@llamaindex` |
| [`langchain`](skills/langchain) | Configure LangChain (langchain-openai, langchain-anthropic, langchain-ollama) against self-hosted OpenAI- and Anthropic-compatible endpoints: keys, model names, structured output, tools, streaming. | `npx skills add ScaleDockHQ/llm-skills@langchain` |
| [`lobehub`](skills/lobehub) | LobeHub (formerly LobeChat) v2 chat and agent workspace, self-hosted with Docker Compose. Use to deploy it, point its OpenAI-compatible or Anthropic proxy URLs at vLLM, llama.cpp, Ollama, LiteLLM... | `npx skills add ScaleDockHQ/llm-skills@lobehub` |
#### Generative media
| Skill | Description | Install |
| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| [`comfyui`](skills/comfyui) | Install, run headless and operate ComfyUI, the node-graph engine for image, video and audio diffusion models (FLUX, SDXL, Wan, LTX), with its HTTP API, custom nodes... | `npx skills add ScaleDockHQ/llm-skills@comfyui` |
| [`whisper-cpp`](skills/whisper-cpp) | Build, run and tune whisper.cpp, the C/C++ Whisper speech-to-text engine (whisper-cli, whisper-server) with ggml models, quantization, VAD and Core ML, CUDA, Vulkan... | `npx skills add ScaleDockHQ/llm-skills@whisper-cpp` |
| [`chatterbox`](skills/chatterbox) | Install and run Chatterbox, Resemble AI's MIT-licensed open text-to-speech family (Turbo 350M, Nano 110M, Multilingual V3 with 23 languages, original English)... | `npx skills add ScaleDockHQ/llm-skills@chatterbox` |
| [`f5-tts`](skills/f5-tts) | Install and run F5-TTS, the flow-matching zero-shot voice-cloning text-to-speech model (F5TTS_v1_Base, E2-TTS) with f5-tts_infer-cli, the Gradio app, the... | `npx skills add ScaleDockHQ/llm-skills@f5-tts` |
| [`kokoro-fastapi`](skills/kokoro-fastapi) | Run and tune Kokoro-FastAPI, the Dockerized FastAPI server for the Kokoro-82M text-to-speech model with an OpenAI-compatible /v1/audio/speech endpoint, voice... | `npx skills add ScaleDockHQ/llm-skills@kokoro-fastapi` |
| [`speaches`](skills/speaches) | Deploy and operate Speaches, the OpenAI-compatible speech server (faster-whisper STT, Kokoro and Piper TTS, Realtime API) with Docker or uv, dynamic model... | `npx skills add ScaleDockHQ/llm-skills@speaches` |
| [`faster-whisper`](skills/faster-whisper) | Install, run and tune faster-whisper, the CTranslate2 Python reimplementation of OpenAI Whisper for speech-to-text (WhisperModel, BatchedInferencePipeline)... | `npx skills add ScaleDockHQ/llm-skills@faster-whisper` |
| [`sglang-diffusion`](skills/sglang-diffusion) | Install and serve SGLang Diffusion, SGLang's image and video generation runtime (`sglang generate`, `sglang serve`, `sglang[diffusion]`, multimodal_gen). Use... | `npx skills add ScaleDockHQ/llm-skills@sglang-diffusion` |
| [`vllm-omni`](skills/vllm-omni) | Install and serve vLLM-Omni, vLLM's omni-modality extension for image, video, speech and audio generation plus Qwen-Omni style chat (`vllm serve MODEL --omni`,... | `npx skills add ScaleDockHQ/llm-skills@vllm-omni` |
| [`invokeai`](skills/invokeai) | Install, run and operate InvokeAI, the node-based creative engine and web UI for Stable Diffusion, SDXL, FLUX, Z-Image, Qwen-Image and Wan video, with its REST... | `npx skills add ScaleDockHQ/llm-skills@invokeai` |
| [`diffusers`](skills/diffusers) | Install, run and serve Hugging Face Diffusers, the PyTorch library for image, video and audio diffusion pipelines (FLUX, SDXL, Qwen-Image, Wan, LTX), including... | `npx skills add ScaleDockHQ/llm-skills@diffusers` |
| [`stable-diffusion-cpp`](skills/stable-diffusion-cpp) | Install, run and serve stable-diffusion.cpp, the ggml C/C++ engine for image and video diffusion (SD, SDXL, FLUX, Qwen-Image, Z-Image, Wan) with sd-cli and the... | `npx skills add ScaleDockHQ/llm-skills@stable-diffusion-cpp` |
#### Embeddings and RAG
| Skill | Description | Install |
| --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`text-embeddings-inference`](skills/text-embeddings-inference) | Run Hugging Face Text Embeddings Inference (TEI) for embedding and rerank models beside your LLM stack: image per GPU, sizing, API key, LiteLLM entries, update, teardown. | `npx skills add ScaleDockHQ/llm-skills@text-embeddings-inference` |
| [`milvus`](skills/milvus) | Run Milvus (Lite, single container or Compose standalone) beside your LLM stack: requirements, hardened compose, index memory math, auth, upgrades, LiteLLM embeddings and Open WebUI. | `npx skills add ScaleDockHQ/llm-skills@milvus` |
| [`weaviate`](skills/weaviate) | Run Weaviate beside your LLM stack: Docker install, memory sizing and quantization, API-key auth and RBAC, text2vec-openai via LiteLLM or TEI, backups, upgrades. | `npx skills add ScaleDockHQ/llm-skills@weaviate` |
| [`chroma`](skills/chroma) | Run Chroma beside your LLM stack: Docker install, RAM sizing, token proxy (v1 has no built-in auth), OpenAI-compatible embedding function via LiteLLM or TEI, backups, upgrades. | `npx skills add ScaleDockHQ/llm-skills@chroma` |
| [`infinity`](skills/infinity) | Run Infinity (infinity_emb), a multi-model REST server for embeddings, rerankers, CLIP and ColBERT, beside your LLM stack: install, config, security, LiteLLM, update. | `npx skills add ScaleDockHQ/llm-skills@infinity` |
| [`qdrant`](skills/qdrant) | Run Qdrant beside your LLM stack: Docker install, RAM sizing, memory tiers and quantization, API keys and TLS, snapshots, upgrades, LiteLLM embeddings and Open WebUI wiring. | `npx skills add ScaleDockHQ/llm-skills@qdrant` |
| [`pgvector`](skills/pgvector) | Add vector search to Postgres with pgvector beside your LLM stack: Docker images, sizing, HNSW and IVFFlat, iterative scans, security, upgrades, LiteLLM embeddings and Open WebUI. | `npx skills add ScaleDockHQ/llm-skills@pgvector` |
| [`mcp-servers`](skills/mcp-servers) | Connect a self-hosted LLM stack to MCP tools: stdio vs streamable HTTP (spec 2026-07-28), authentication, Qdrant, Chroma and Postgres servers, LiteLLM MCP gateway. | `npx skills add ScaleDockHQ/llm-skills@mcp-servers` |
### Platform skills
#### Hardware platforms
| Skill | Description | Install |
| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| [`nvidia-dgx-spark`](skills/nvidia-dgx-spark) | Set up, size, network and operate NVIDIA DGX Spark (GB10 Grace Blackwell, 128 GB unified memory, DGX OS) for LLM inference, single or dual node over ConnectX-7. | `npx skills add ScaleDockHQ/llm-skills@nvidia-dgx-spark` |
| [`nvidia-dgx-station`](skills/nvidia-dgx-station) | Set up, size and operate NVIDIA DGX Station (GB300 Grace Blackwell Ultra, 252 GB HBM3e plus 496 GB LPDDR5X, DGX OS) for LLM inference, single or two stations. | `npx skills add ScaleDockHQ/llm-skills@nvidia-dgx-station` |
| [`apple-silicon`](skills/apple-silicon) | Size, tune and operate Apple Silicon Macs (M-series MacBook, Mac mini, Mac Studio) for local LLM inference with Metal and MLX. Use for unified memory... | `npx skills add ScaleDockHQ/llm-skills@apple-silicon` |
| [`amd-instinct`](skills/amd-instinct) | Set up, verify and operate AMD Instinct GPUs (MI300X, MI325X, MI350X, MI355X, MI250, MI210) with ROCm 10.x for LLM inference. | `npx skills add ScaleDockHQ/llm-skills@amd-instinct` |
| [`amd-strix-halo`](skills/amd-strix-halo) | Set up, size and operate AMD Ryzen AI Max (Strix Halo, gfx1151, Radeon 8060S, up to 128 GB unified memory) for local LLM inference on Linux or Windows. | `npx skills add ScaleDockHQ/llm-skills@amd-strix-halo` |
| [`intel-gpu-gaudi`](skills/intel-gpu-gaudi) | Set up, verify and operate Intel accelerators for LLM inference, Gaudi 2 and 3 (Gaudi software 1.24.x, habana container runtime, vllm-gaudi plugin) and Intel GPUs (Arc, Arc Pro B-series, Data Center GPU with vLLM XPU, llama.cpp SYCL, OpenVINO). | `npx skills add ScaleDockHQ/llm-skills@intel-gpu-gaudi` |
| [`nvidia-rtx-workstation`](skills/nvidia-rtx-workstation) | Choose, size and run NVIDIA RTX PRO Blackwell workstation GPUs (6000 96 GB, 5000, 4500, 4000, Max-Q) for local LLM inference on Linux: open driver, sm_120, multi-GPU, power, MIG. | `npx skills add ScaleDockHQ/llm-skills@nvidia-rtx-workstation` |
| [`cpu-inference`](skills/cpu-inference) | Run, size and tune LLM inference on CPUs only (x86 AVX2/AVX-512/AMX, AArch64 servers with NEON and KleidiAI, Intel Macs) with llama.cpp. Use for no-GPU... | `npx skills add ScaleDockHQ/llm-skills@cpu-inference` |
| [`nvidia-datacenter-gpus`](skills/nvidia-datacenter-gpus) | Choose and operate NVIDIA datacenter GPUs (H100, H200, B200, B300, GB200) for LLM serving: HBM sizing, NVLink and Fabric Manager, R580 driver branch, MIG, DCGM. | `npx skills add ScaleDockHQ/llm-skills@nvidia-datacenter-gpus` |
| [`google-tpu`](skills/google-tpu) | Pick, provision and operate Google Cloud TPUs (v5e, v6e Trillium, TPU7x Ironwood) to serve LLMs with vLLM TPU (tpu-inference, vllm-tpu) on Compute Engine or GKE, then tear down. | `npx skills add ScaleDockHQ/llm-skills@google-tpu` |
| [`aws-neuron`](skills/aws-neuron) | Set up and operate AWS Inferentia2 and Trainium (Inf2, Trn1, Trn2, Trn3) with the Neuron SDK to serve LLMs through vLLM Neuron or NxD Inference, using Neuron DLAMIs and DLCs. | `npx skills add ScaleDockHQ/llm-skills@aws-neuron` |
#### Cloud providers
| Skill | Description | Install |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------- |
| [`runpod`](skills/runpod) | Run LLMs on Runpod GPU Pods and Serverless (worker-vllm, vLLM, OpenAI-compatible endpoints) with runpodctl, then connect them to a gateway and tear down. | `npx skills add ScaleDockHQ/llm-skills@runpod` |
| [`modal`](skills/modal) | Run LLMs on Modal serverless GPUs with the modal CLI and Python SDK (vLLM Servers, Endpoints, web functions, Volumes), connect them to a gateway and tear down. | `npx skills add ScaleDockHQ/llm-skills@modal` |
| [`lambda-cloud`](skills/lambda-cloud) | Run LLMs on Lambda Cloud (Lambda, lambda.ai) GPU instances, 1-Click Clusters and filesystems through the Cloud API. | `npx skills add ScaleDockHQ/llm-skills@lambda-cloud` |
| [`vast-ai`](skills/vast-ai) | Rent cheap GPUs on Vast.ai with the vastai CLI to serve LLMs (vLLM instances and Serverless endpoints), connect them to a gateway and destroy them. | `npx skills add ScaleDockHQ/llm-skills@vast-ai` |
| [`coreweave`](skills/coreweave) | Run LLMs on CoreWeave, CKS managed Kubernetes GPU clusters and Node Pools, SUNK (Slurm on Kubernetes), Serverless Inference (api.inference.wandb.ai, W&B/Forge) and... | `npx skills add ScaleDockHQ/llm-skills@coreweave` |
| [`nebius`](skills/nebius) | Run LLMs on Nebius, GPU VMs and Managed Kubernetes (mk8s) through the nebius CLI, plus Nebius Token Factory (formerly AI Studio) for OpenAI-compatible inference,... | `npx skills add ScaleDockHQ/llm-skills@nebius` |
| [`baseten`](skills/baseten) | Baseten model APIs and dedicated GPU deployments - call hosted models over OpenAI or Anthropic-style endpoints, or deploy your own with the baseten CLI and Truss... | `npx skills add ScaleDockHQ/llm-skills@baseten` |
| [`replicate`](skills/replicate) | Replicate predictions API, Cog and deployments - run hosted models over HTTP or the replicate Python/Node clients, push your own with Cog, run always-on deployments,... | `npx skills add ScaleDockHQ/llm-skills@replicate` |
| [`fal`](skills/fal) | fal (fal.ai) genmedia model APIs and fal serverless - call 1,000+ hosted image, video, audio and 3D models through the queue API or fal-client, run an... | `npx skills add ScaleDockHQ/llm-skills@fal` |
| [`crusoe`](skills/crusoe) | Run LLMs on Crusoe Cloud, GPU VMs and Managed Kubernetes (CMK) through the crusoe CLI, plus Crusoe Managed Inference (Intelligence Foundry) with an OpenAI-compatible... | `npx skills add ScaleDockHQ/llm-skills@crusoe` |
| [`hyperbolic`](skills/hyperbolic) | Hyperbolic GPU cloud - rent, run and tear down on-demand or reserved H100/H200/B200/B300 VMs and bare metal to self-host LLMs, via app.hyperbolic.ai or the... | `npx skills add ScaleDockHQ/llm-skills@hyperbolic` |
| [`together-ai`](skills/together-ai) | Use Together AI for open-model APIs - serverless chat, tools, JSON, batch, fine-tuning, dedicated endpoints. | `npx skills add ScaleDockHQ/llm-skills@together-ai` |
| [`fireworks-ai`](skills/fireworks-ai) | Use Fireworks AI for open-model APIs - serverless chat, tool calling, structured outputs, batch, fine-tuning, on-demand GPU deployments. | `npx skills add ScaleDockHQ/llm-skills@fireworks-ai` |
| [`groq`](skills/groq) | Use GroqCloud for fast open-model inference - OpenAI-compatible chat, Responses API, tools, structured outputs, batch, flex tier, LoRA. | `npx skills add ScaleDockHQ/llm-skills@groq` |
| [`cerebras`](skills/cerebras) | Use Cerebras Inference Cloud (api.cerebras.ai) for fast OpenAI-compatible open-model APIs, with key and project setup, model listing, rate limits, prompt caching,... | `npx skills add ScaleDockHQ/llm-skills@cerebras` |
| [`sambanova`](skills/sambanova) | Use SambaNova SambaCloud (api.sambanova.ai) for fast OpenAI- and Anthropic-compatible open-model APIs, with API keys, model listing, rate-limit tiers, prompt caching,... | `npx skills add ScaleDockHQ/llm-skills@sambanova` |
| [`deepinfra`](skills/deepinfra) | Use DeepInfra (api.deepinfra.com) for OpenAI- and Anthropic-compatible open-model APIs, private GPU deployments and GPU rentals, with API keys, per-key spend limits,... | `npx skills add ScaleDockHQ/llm-skills@deepinfra` |
| [`hugging-face`](skills/hugging-face) | Use Hugging Face (huggingface.co) to download and upload models with the `hf` CLI, manage tokens and cache, call Inference Providers through router.huggingface.co,... | `npx skills add ScaleDockHQ/llm-skills@hugging-face` |
| [`openai-api`](skills/openai-api) | Use the OpenAI API directly (api.openai.com) - keys, projects, Responses and Chat Completions, streaming, tools, structured output, Batch, prompt caching, files,... | `npx skills add ScaleDockHQ/llm-skills@openai-api` |
| [`anthropic-api`](skills/anthropic-api) | Use the Claude API directly (api.anthropic.com) - keys, workspaces, Messages API, OpenAI SDK compatibility, streaming, tools, structured outputs, Message Batches,... | `npx skills add ScaleDockHQ/llm-skills@anthropic-api` |
| [`gemini-api`](skills/gemini-api) | Use the Google Gemini API directly (generativelanguage.googleapis.com) - keys, projects, generateContent, Interactions API, OpenAI compatibility, streaming, tools,... | `npx skills add ScaleDockHQ/llm-skills@gemini-api` |
| [`aws-bedrock`](skills/aws-bedrock) | Use Amazon Bedrock to call hosted LLMs from your AWS account via Converse, InvokeModel, OpenAI-compatible Chat Completions or Responses, and Anthropic Messages. | `npx skills add ScaleDockHQ/llm-skills@aws-bedrock` |
| [`aws-sagemaker`](skills/aws-sagemaker) | Host your own LLM on Amazon SageMaker AI endpoints (real-time, async, serverless, inference components), with JumpStart and LMI/vLLM containers, scale to zero, SigV4... | `npx skills add ScaleDockHQ/llm-skills@aws-sagemaker` |
| [`google-vertex-ai`](skills/google-vertex-ai) | Use Google Cloud Vertex AI, renamed Gemini Enterprise Agent Platform in 2026, for Gemini, Claude and open Model Garden models via gcloud and ADC, the... | `npx skills add ScaleDockHQ/llm-skills@google-vertex-ai` |
| [`azure-foundry`](skills/azure-foundry) | Use Microsoft Foundry, formerly Azure AI Foundry and Azure OpenAI, to deploy and call GPT, Claude and open models with the az CLI, deployment types, the v1 API, Entra... | `npx skills add ScaleDockHQ/llm-skills@azure-foundry` |
### Technique skills
#### Techniques
| Skill | Description | Install |
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`speculative-decoding`](skills/speculative-decoding) | Speed up LLM token generation with speculative decoding (EAGLE-3, MTP, draft model, n-gram, suffix, DFlash) on vLLM, SGLang, TensorRT-LLM, llama.cpp, TensorFold,... | `npx skills add ScaleDockHQ/llm-skills@speculative-decoding` |
| [`disaggregated-serving`](skills/disaggregated-serving) | Split LLM inference into prefill and decode pools with KV transfer (vLLM NixlConnector/MooncakeConnector, SGLang --disaggregation-mode, TensorRT-LLM... | `npx skills add ScaleDockHQ/llm-skills@disaggregated-serving` |
| [`kv-cache-offloading`](skills/kv-cache-offloading) | Extend LLM KV cache capacity by offloading to CPU RAM, disk, object storage or peer instances (vLLM OffloadingConnector, LMCache, SGLang HiCache, TensorRT-LLM... | `npx skills add ScaleDockHQ/llm-skills@kv-cache-offloading` |
| [`prefix-aware-routing`](skills/prefix-aware-routing) | Route LLM requests to the replica that already holds the matching KV prefix (llm-d inference scheduler, Gateway API Inference Extension, vLLM production-stack router,... | `npx skills add ScaleDockHQ/llm-skills@prefix-aware-routing` |
| [`multi-node-inference`](skills/multi-node-inference) | Serve one LLM across several machines with tensor, pipeline, data and expert parallelism over NCCL, ConnectX-7, RoCE or InfiniBand (vLLM Ray and mp --nnodes, SGLang... | `npx skills add ScaleDockHQ/llm-skills@multi-node-inference` |
| [`moe-serving`](skills/moe-serving) | Serve Mixture-of-Experts LLMs efficiently with expert parallelism (EP), DeepEP all-to-all, EPLB load balancing, expert offload to CPU and MoE kernels, with per-engine... | `npx skills add ScaleDockHQ/llm-skills@moe-serving` |
| [`structured-outputs`](skills/structured-outputs) | Force LLM output to match a JSON schema, regex, choice list or grammar with guided decoding on vLLM, SGLang, TensorRT-LLM, llama.cpp, Ollama, TensorFold and... | `npx skills add ScaleDockHQ/llm-skills@structured-outputs` |
| [`tool-calling-parsers`](skills/tool-calling-parsers) | Pick and verify the tool-call parser for a model on each inference engine. | `npx skills add ScaleDockHQ/llm-skills@tool-calling-parsers` |
| [`reasoning-models`](skills/reasoning-models) | Configure and verify reasoning (thinking) output on each inference engine. | `npx skills add ScaleDockHQ/llm-skills@reasoning-models` |
| [`long-context`](skills/long-context) | Extend and serve long context windows safely with YaRN and RoPE scaling, context-length flags, chunked prefill, sliding window attention, KV-cache memory math and... | `npx skills add ScaleDockHQ/llm-skills@long-context` |
| [`multi-lora-serving`](skills/multi-lora-serving) | Serve many LoRA adapters on one base model with per-request adapter selection and runtime load/unload, with per-engine flags for vLLM, SGLang, TensorRT-LLM,... | `npx skills add ScaleDockHQ/llm-skills@multi-lora-serving` |
| [`kv-cache-quantization`](skills/kv-cache-quantization) | Fit more context or concurrent requests by quantizing the KV cache (FP8, NVFP4, int8, q8_0, q4_0) on vLLM, SGLang, TensorRT-LLM, llama.cpp, Ollama, ExLlamaV3 and... | `npx skills add ScaleDockHQ/llm-skills@kv-cache-quantization` |
| [`choosing-inference-engine`](skills/choosing-inference-engine) | Choose an LLM inference engine from hardware, model format, features and concurrency, with a decision flow and an engine matrix for vLLM, SGLang, TensorRT-LLM,... | `npx skills add ScaleDockHQ/llm-skills@choosing-inference-engine` |
| [`choosing-hardware`](skills/choosing-hardware) | Choose and size hardware for running an LLM, from model size, quantization, context and concurrency, using VRAM and memory-bandwidth math plus a spec table for NVIDIA... | `npx skills add ScaleDockHQ/llm-skills@choosing-hardware` |
### Model-family skills
#### Model families
| Skill | Description | Install |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| [`qwen`](skills/qwen) | Serve Alibaba Qwen (Qwen3.8, 3.6, 3.5, Coder-Next, Qwen3) with the right parsers, templates, thinking toggles and quants per engine and platform. | `npx skills add ScaleDockHQ/llm-skills@qwen` |
| [`llama`](skills/llama) | Serve Meta's open models, Muse Glimmer 30B and Llama 4 Scout/Maverick (plus Llama 3.x), with the right template, parsers, quant and sampling. | `npx skills add ScaleDockHQ/llm-skills@llama` |
| [`deepseek`](skills/deepseek) | Serve DeepSeek V4.1, V4 (Flash, Pro), V3.2 and R1 with the right parsers, templates, quants and sampling per engine and platform. | `npx skills add ScaleDockHQ/llm-skills@deepseek` |
| [`gpt-oss`](skills/gpt-oss) | Serve and configure OpenAI gpt-oss-120b and gpt-oss-20b (and gpt-oss-safeguard, NVIDIA gpt-oss-puzzle-88B) with the harmony format, reasoning_effort, MXFP4 weights,... | `npx skills add ScaleDockHQ/llm-skills@gpt-oss` |
| [`gemma`](skills/gemma) | Serve Google Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B, QAT, DiffusionGemma) with the right parsers, templates, quants and sampling per engine and platform. | `npx skills add ScaleDockHQ/llm-skills@gemma` |
| [`mistral`](skills/mistral) | Serve Mistral AI open-weight models (Small 4, Medium 3.5, Large 3, Ministral 3, Devstral 2, Magistral). | `npx skills add ScaleDockHQ/llm-skills@mistral` |
| [`glm`](skills/glm) | Serve Z.ai (Zhipu) GLM open-weight models (GLM-5.3, 5.3-Flash, 5.2, 4.7-Flash, 4.x). | `npx skills add ScaleDockHQ/llm-skills@glm` |
| [`kimi`](skills/kimi) | Serve Moonshot AI Kimi open-weight models (K3, K2.7-Code, K2.6, K2.5, K2-Thinking, Kimi-Linear, Kimi-VL) with correct parsers, templates, quants and sizing. | `npx skills add ScaleDockHQ/llm-skills@kimi` |
| [`nemotron`](skills/nemotron) | Serve NVIDIA Nemotron open models (3.5 Lightning, 3 Nano, Super, Ultra, Nano 4B, Nano Omni, Llama-Nemotron). | `npx skills add ScaleDockHQ/llm-skills@nemotron` |
| [`minimax`](skills/minimax) | Serve and configure MiniMax open-weight models (M3, M2.7, M2.5, M2.1, M2): parsers, chat templates, thinking modes, quants, licenses and bugs. | `npx skills add ScaleDockHQ/llm-skills@minimax` |
| [`phi`](skills/phi) | Serve Microsoft Phi small models (Phi-4, Phi-4-reasoning, mini, mini-flash-reasoning, multimodal, reasoning-vision-15B) with the right template, parser, quant and... | `npx skills add ScaleDockHQ/llm-skills@phi` |
### Stack skills
#### Stacks
| Skill | Description | Install |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------- |
| [`ollama-open-webui-tailscale`](skills/ollama-open-webui-tailscale) | Private web chat for local models: Ollama behind Open WebUI, reachable only over your Tailscale tailnet with HTTPS. | `npx skills add ScaleDockHQ/llm-skills@ollama-open-webui-tailscale` |
| [`litellm-langfuse-prometheus`](skills/litellm-langfuse-prometheus) | One LiteLLM gateway reporting traces to Langfuse and metrics to Prometheus and Grafana. | `npx skills add ScaleDockHQ/llm-skills@litellm-langfuse-prometheus` |
| [`voice-stack`](skills/voice-stack) | Self-hosted voice assistant: whisper.cpp speech-to-text, an LLM and Kokoro text-to-speech behind Open WebUI Voice Mode, with Pipecat and LiveKit paths. | `npx skills add ScaleDockHQ/llm-skills@voice-stack` |
| [`goose`](skills/goose) | Connect goose (Desktop and CLI agent) to self-hosted OpenAI- and Anthropic-compatible endpoints: OPENAI_HOST, custom providers, keys, tools, tool shim. | `npx skills add ScaleDockHQ/llm-skills@goose` |
| [`open-webui`](skills/open-webui) | Open WebUI chat UI: Docker install, OpenAI-compatible connections to self-hosted engines, tools, updates | `npx skills add ScaleDockHQ/llm-skills@open-webui` |
| [`runpod-serverless-vllm`](skills/runpod-serverless-vllm) | Stack recipe for vLLM on Runpod Serverless (worker-vllm) behind a LiteLLM gateway, with spend caps, cold-start handling and teardown. | `npx skills add ScaleDockHQ/llm-skills@runpod-serverless-vllm` |
| [`modal-vllm`](skills/modal-vllm) | Stack recipe for vLLM on Modal serverless GPUs (@app.server or a managed Endpoint) behind a LiteLLM gateway, with proxy-token auth, scale-to-zero, cold-start handling and teardown. | `npx skills add ScaleDockHQ/llm-skills@modal-vllm` |
| [`mac-cluster-exo-mlx`](skills/mac-cluster-exo-mlx) | Cluster several Apple Silicon Macs into one inference host with exo or MLX distributed (mlx.launch, JACCL RDMA, ring) over Thunderbolt 5. Use for Mac Studio... | `npx skills add ScaleDockHQ/llm-skills@mac-cluster-exo-mlx` |
| [`lm-studio-claude-code`](skills/lm-studio-claude-code) | Run Claude Code against a local model served by LM Studio over its Anthropic-compatible /v1/messages endpoint (ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, lms... | `npx skills add ScaleDockHQ/llm-skills@lm-studio-claude-code` |
| [`vllm-kserve-keda`](skills/vllm-kserve-keda) | Deploy vLLM on Kubernetes with KServe (InferenceService or LLMInferenceService) and autoscale it with KEDA from vLLM Prometheus metrics, with a pinned runtime and teardown. | `npx skills add ScaleDockHQ/llm-skills@vllm-kserve-keda` |
| [`llm-d-vllm-kubernetes`](skills/llm-d-vllm-kubernetes) | Deploy vLLM behind the llm-d router on Kubernetes (Gateway API Inference Extension): optimized baseline, prefill/decode pools over NIXL and KEDA autoscaling. | `npx skills add ScaleDockHQ/llm-skills@llm-d-vllm-kubernetes` |
| [`dynamo-trtllm-disaggregated`](skills/dynamo-trtllm-disaggregated) | Run TensorRT-LLM prefill and decode as separate NVIDIA Dynamo workers on Kubernetes (DynamoGraphDeployment, NIXL over RDMA, KV-aware router, Planner). | `npx skills add ScaleDockHQ/llm-skills@dynamo-trtllm-disaggregated` |
## Roadmap
[`catalog.json`](catalog.json) lists every published and planned skill with its kind, category, status and build wave. 208 of 217 catalogued skills are published; the rest are planned and not installable until they are published and listed above.
## Repository layout
```text
skills/<name>/
SKILL.md instructions the agent loads
README.md human overview: install, rules, references
metadata.json version, date, abstract, upstream references
references/ detailed, version-stamped guides loaded on demand
assets/ config and shell templates (some skills)
scripts/ bench.sh, a shared curl + jq benchmark (some skills)
template/skill/ starting point for a new skill
tools/ new-skill.sh and tools/skeleton/<kind>/ scaffolds
scripts/ validate-skills.mjs and the git hook installer
catalog.json every published and planned skill
docs/decisions/ architecture decision records
```
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md). Run `pnpm install`, then `pnpm verify` before opening a pull request. Agents follow [AGENTS.md](AGENTS.md). Report security issues as described in [SECURITY.md](SECURITY.md).
## License
[MIT](LICENSE)