| name | docker-debug |
| description | Container debugging runbook for this inference stack — per-service healthcheck endpoints, common CUDA/driver failure signatures, log locations, and the safe restart procedure. Use when a service is unhealthy or failing to start. |
Docker debugging runbook
For the llama.cpp / vLLM / open-webui stack defined in docker-compose.yaml.
Sandboxed execution: agents cannot reach the host Docker daemon, the services, or the
GPU. Every diagnostic and restart command in this runbook is executed by the human on
the workstation. The agent supplies the exact command, waits for the pasted output or
logs, and matches the failure signatures below against what the human reports.
Services and healthchecks
| Service | External port | Internal healthcheck |
|---|
| llama-cpu | 8001 | curl -f http://localhost:8001/v1/health |
| llama-gpu | 8004 | curl -f http://localhost:8002/v1/health (internal 8002) |
| vllm-gpu | 8005 | curl -f http://localhost:8003/health (internal 8003) |
| open-webui | 3000 | container :main, no explicit healthcheck |
| comfyui | 8188 | curl -f http://localhost:8188/system_stats |
Start with make status for the compose-level health summary and make health for the
detailed per-service check.
Common CUDA / driver failure signatures
CUDA error: no kernel image is available for execution — image built for the wrong
compute capability; the RTX 5090 is Blackwell (sm_120). Rebuild the GPU image.CUDA driver version is insufficient for CUDA runtime version — host NVIDIA driver older
than the container's CUDA toolkit; update the host driver.could not select device driver "nvidia" — NVIDIA Container Toolkit not wired into
Docker; check nvidia-smi on the host and the runtime config.- vLLM OOM at load — model + KV cache exceeds 32GB VRAM; reduce context length,
--gpu-memory-utilization, or quantization.
Log locations
- Follow logs:
make logs, or per-service make logs-gpu / make logs-cpu /
make logs-vllm / make logs-ui.
- Raw:
docker logs <container> (llama-cpu, llama-gpu, vllm-gpu, open-webui).
- Compose
*.log files are git-ignored.
Safe restart procedure
- Confirm which service is unhealthy (
make status); do not restart the whole stack for a
single failure.
- Restart just that service:
docker compose restart <service>.
- If a config or image changed,
make down then make up (or make rebuild for a
from-scratch image build). make reset is the full stop-clean-rebuild cycle.
- Never
docker system prune / docker volume rm to "fix" a hang — those are guarded
ask actions and destroy state; diagnose from logs first.
