ai-server-memory-diagnostics

name	ai-server-memory-diagnostics
description	Use when investigating swap, memory pressure, compressor usage, llama.cpp memory, Metal/unified memory, or model-server RAM issues on the Mac Studio ai-server.
license	MIT
metadata	{"author":"Airton Ponce @ton-anywhere"}

AI Server Memory Diagnostics

Purpose

Diagnose memory pressure on the Mac Studio ai-server without trusting misleading single-number summaries. For llama.cpp on macOS, ps RSS often understates the real cost; use vmmap physical footprint and swapped writable regions.

Quick Workflow

1. Confirm the remote host and global memory state:

ssh ai-server vm_stat
ssh ai-server sysctl vm.swapusage
ssh ai-server memory_pressure

Key fields:

• vm.swapusage: total, used, free encrypted swap.
• Pages stored in compressor: logical pages compressed.
• Pages occupied by compressor: RAM consumed by compressed pages.
• Swapins / Swapouts: whether swap is active over time.

Convert 16 KiB pages to GiB with:

GiB = pages * 16384 / 1024 / 1024 / 1024

2. Find the model processes:

ssh ai-server "ps aux | grep -E 'llama|mlx|omlx' | grep -v grep"
ssh ai-server "ps aux | sort -nrk 6 | head -30"

Treat ps RSS as a rough clue only. On macOS unified memory, llama.cpp / Metal allocations and swapped anonymous memory may not appear as expected in RSS.

3. Confirm listeners, health, and tmux panes before assuming a model is alive:

ssh ai-server "lsof -nP -iTCP:8080 -sTCP:LISTEN"
ssh ai-server "lsof -nP -iTCP:8081 -sTCP:LISTEN"
ssh ai-server "curl -sS --max-time 2 http://127.0.0.1:8080/health"
ssh ai-server "curl -sS --max-time 2 http://127.0.0.1:8081/health"
ssh ai-server "zsh -lc 'tmux ls'"
ssh ai-server "zsh -lc 'tmux list-panes -a -F \"#{session_name}:#{window_index}.#{pane_index} #{pane_current_command} #{pane_pid}\"'"
ssh ai-server "zsh -lc 'tmux capture-pane -pt llama-server-coder:0.0 -S -200'"

Use zsh -lc for tmux commands because non-login SSH shells may not have tmux on PATH. If a port is not listening but a tmux session remains, capture the pane tail; crash lines such as killed often survive in scrollback.

4. Inspect each suspicious PID with vmmap:

ssh ai-server "vmmap -summary <pid> | tail -80"

Read these lines first:

• Physical footprint
• Physical footprint (peak)
• Writable regions
• MALLOC_LARGE
• shared memory
• mapped file
• TOTAL

The most important columns are:

• RESIDENT: currently in RAM.
• DIRTY: private writable memory currently resident.
• SWAPPED: private writable memory moved to swap.
• VIRTUAL: address space, not direct RAM usage.

5. Attribute the pressure to launch profiles.

Compare the active command line to AGENTS.md model profiles. High-risk flags include:

• --ctx-size with very large contexts.
• --parallel, especially with --no-kv-unified.
• --cache-ram.
• large --batch-size / --ubatch-size.
• multiple co-resident models.

Interpretation Rules

• Large swap plus large MALLOC_LARGE swapped usually means model heap, KV cache, prompt cache, or batching buffers have been pushed out.
• Large shared memory swapped can point at shared model/runtime buffers rather than ordinary app memory.
• Large mapped file is often model weights or file-backed mappings; it is less alarming than dirty writable swap.
• Physical footprint is a better process-level signal than RSS for model servers on macOS.
• Near-full swap plus high memory_pressure free percentage usually means the system had earlier pressure and has not reclaimed swap; still inspect vmmap to find retained swapped writable regions.
• A crashed model server can leave a tmux shell alive. Verify the port and health endpoint, not just the tmux session name.
• Prompt-cache/checkpoint saves can be the immediate crash trigger. In tmux tails, look for prompt_save, created context checkpoint, state sizes in MiB/GiB, then killed.
• Compressor usage is not free. If many pages are stored in compressor and swap is nearly full, the machine is under real pressure.
• System daemons with tiny RSS are rarely the cause when model PIDs show tens of GiB in vmmap.

Output Shape

When reporting findings, include:

Global:
- swap used / total
- compressor pages stored and occupied, converted to GiB if useful
- current memory pressure/free percentage

Per model PID:
- command summary and port
- listener and /health status
- physical footprint and peak
- swapped writable total
- largest swapped regions, especially MALLOC_LARGE/shared memory

Tmux / crash evidence:
- relevant session and pane
- last crash line, if any
- prompt-cache or checkpoint sizes near the crash

Conclusion:
- likely memory source
- safest restart or relaunch profile change

Common Mistakes

Mistake	Better move
Trusting ps RSS alone	Use vmmap -summary <pid>
Treating all virtual memory as RAM	Focus on footprint, resident, dirty, and swapped columns
Blaming small macOS services first	Inspect model PIDs before system daemons
Ignoring compressor stats	Compressor plus near-full swap means real pressure
Treating full swap as current pressure by itself	Compare with memory_pressure; full swap can be residue from an earlier spike
Trusting a tmux session name	Check listener and /health; capture pane tail for killed processes
Looking only at total swap	Attribute swap to process regions with vmmap
Forgetting launch flags	Tie memory pressure back to context, parallelism, cache, and co-resident models