Restore machine responsiveness via safe, selective process cleanup. Use when system unresponsive, high CPU/load average, IO pressure, filesystem cache bloat, memory pressure from btrfs/ext4, stuck tests, competing cargo builds, confused agents in loops, swap thrashing, disk full, systemd-oomd kills, or tmux/zellij session sprawl.
Restore machine responsiveness via safe, selective process cleanup. Use when system unresponsive, high CPU/load average, IO pressure, filesystem cache bloat, memory pressure from btrfs/ext4, stuck tests, competing cargo builds, confused agents in loops, swap thrashing, disk full, systemd-oomd kills, or tmux/zellij session sprawl.
System Performance Remediation
Core Principle: First, do no harm. Kill OBVIOUSLY useless processes before touching anything potentially useful.
The Whack-a-Mole Anti-Pattern:
Killing child processes (cargo builds, tests) is POINTLESS if confused parent agents respawn them.
Kill the confused agents, not their children.
Known SIGTERM-ignorers:bun test — always needs SIGKILL after SIGTERM fails.
VM Tuning & Filesystem Cache Bloat (The Silent Killer)
Real-world incident (2026-02-23): On trj (499GB RAM, btrfs), vfs_cache_pressure=50 let btrfs
inode/dentry caches balloon to 388GB page cache + 40GB slab. Memory pressure hit 18%.
systemd-oomd killed user@1000.service, destroying the mux server and all 382 agent sessions
instantly. The fix: vfs_cache_pressure=200 + min_free_kbytes=2GB + drop caches.
Pressure dropped from 18% to 2.4% in minutes.
The Cache Bloat Pattern
High-RAM machines with many agents accumulate massive filesystem caches. The kernel hoards dentries, inodes, and page cache (especially on btrfs). This creates memory pressure even with "free" RAM because the kernel's reclaim paths stall under pressure.
Symptoms:
System feels sluggish despite free -h showing lots of "available" RAM
/proc/pressure/memory shows sustained avg10 > 5% (the key metric!)
kcompactd0 running at 2-5% CPU continuously
Slab cache (cat /proc/meminfo | grep Slab) is 20-40+ GB
vmstat 1 3 shows high si/so or bi/bo in first sample
Diagnose Cache Bloat
# 1. Check memory pressure (THE critical metric)cat /proc/pressure/memory
# some avg10=18.78 → 18.78% of time tasks stalled on memory = BAD# 2. Check VM tuning
sysctl vm.vfs_cache_pressure vm.min_free_kbytes
# 3. Check slab breakdownsudo slabtop -o -s c | head -15
# Look for: btrfs_inode (GB), radix_tree_node (GB), dentry (GB), ext4_inode_cache (GB)# 4. Check page cache vs actual usage
grep -E "Cached|Slab|SReclaimable|SUnreclaim|Dirty|MemAvail" /proc/meminfo
# 5. Check kcompactd (memory compaction daemon — should be ~0% CPU)
ps -o pid,pcpu,etime,cmd -p $(pgrep kcompactd) 2>/dev/null
Fix: Tune VM Parameters
Settings by filesystem and RAM size:
Machine Type
FS
vfs_cache_pressure
min_free_kbytes
Notes
499GB btrfs
btrfs
200
2GB (2097152)
btrfs caches are aggressive
251GB ext4
ext4
150
1-2GB (1048576-2097152)
ext4 is less cache-heavy
58GB ext4
ext4
150
512MB (524288)
VPS tier
29GB ext4
ext4
150
512MB (524288)
VPS tier
15GB ext4
ext4
150
256MB (262144)
Small VPS
# Apply immediatelysudo sysctl -w vm.vfs_cache_pressure=200 vm.min_free_kbytes=2097152
# Drop caches for immediate relief (only if pressure avg10 > 5%)sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches"# Persist to sysctl confsudotee /etc/sysctl.d/99-system-resource-protection.conf << 'EOF'# Tuned for heavy agent workloads
vm.swappiness = 10
vm.vfs_cache_pressure = 200
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.min_free_kbytes = 2097152
fs.inotify.max_user_watches = 524288
fs.inotify.max_user_instances = 1024
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
vm.max_map_count = 2147483642
EOF
WARNING: The default vfs_cache_pressure=100 is dangerous on high-RAM btrfs machines.
A value of 50 (often set by "desktop optimization" scripts) is even worse — it actively
tells the kernel to hoard caches. Always check this on any machine that feels sluggish.
The worst-case scenario:systemd-oomd kills user@1000.service, which cascades to
kill the wezterm-mux-server, destroying ALL agent sessions. This happened on trj when
a single session peaked at 404GB and the user slice hit 496GB/536GB EffectiveMemoryMax.
Set Per-Session Memory Limits
Prevent any single session from consuming enough memory to trigger oomd:
# Cap individual sessions (prevents one runaway session from killing the user slice)sudomkdir -p /etc/systemd/system/session-.scope.d
sudotee /etc/systemd/system/session-.scope.d/memory-limit.conf << 'EOF'
[Scope]
MemoryMax=64G
MemoryHigh=48G
EOF
# Cap the entire user slice (leave headroom for system)# NOTE: Check for existing override.conf that might set MemoryMax=infinitysudomkdir -p /etc/systemd/system/user-1000.slice.d
# Edit existing override.conf if present, or create new:sudotee /etc/systemd/system/user-1000.slice.d/memory-limit.conf << 'EOF'
[Slice]
MemoryMax=460G
MemoryHigh=400G
EOF
sudo systemctl daemon-reload
CRITICAL: Check for pre-existing override files that set MemoryMax=infinity — these
sort alphabetically after memory-limit.conf and will negate your limits. Consolidate
all settings into a single file or name yours zz-memory-limit.conf.
If resource-watchdog.service is crash-looping with IOPRIO errors:
# Check status
systemctl --user status resource-watchdog.service
# The fix: IOSchedulingClass=realtime requires root — change to best-effort# In ~/.config/systemd/user/resource-watchdog.service:# IOSchedulingClass=realtime → IOSchedulingClass=best-effort
systemctl --user daemon-reload
systemctl --user restart resource-watchdog.service
Diagnosis
CPU Pressure (Critical for Sluggishness)
Load average can look "OK" while system feels sluggish. CPU pressure reveals the truth.
cat /proc/pressure/cpu
# some avg10=57.18 → 57% of time tasks waiting for CPU = BAD
Metric
Healthy
Warning
Critical
CPU pressure avg10
<10%
10-30%
>30%
IO pressure avg10
<5%
5-15%
>15%
Memory pressure avg10
<5%
5-20%
>20%
Full Status Check (Linux)
uptime && nproc# Load vs cores (danger: ratio > 1.5)
free -h # Memory (danger: available < 10%)
swapon --show # Swap config (danger: 0B total or near-full)
ps -eo stat | grep -c '^Z'# Zombie countcat /proc/sys/fs/file-nr # File handles (allocated, free, max)
ps aux --sort=-%cpu | head -20 # Top CPU consumers
ps aux --sort=-%mem | head -10 # Top memory consumers
vmstat 1 3 # IO wait, context switches, swap in/outcat /proc/pressure/cpu # CPU pressurecat /proc/pressure/memory # Memory pressure (THE key sluggishness metric)cat /proc/pressure/io # IO pressuredf -h / /data /tmp /data/tmp # Disk space
sysctl vm.vfs_cache_pressure vm.min_free_kbytes # VM tuning (cache bloat check)
grep -E "Slab|SReclaimable" /proc/meminfo # Slab cache size
macOS Quick Status
top -l 1 -n 10 -stats pid,command,cpu,mem,state | head -20
memory_pressure # < 20% = warning, < 10% = critical
Swap & zram Management
The Swap Paradox
A machine can have 189GB free RAM yet feel sluggish because 30GB of process pages are stuck in swap from a past memory spike. The kernel doesn't proactively move pages back to RAM — they only fault back on access, causing latency spikes.
Symptom: Machine feels laggy, free -h shows plenty of available RAM but significant swap used.
Diagnose Swap Issues
swapon --show
free -h | grep Swap
Red flags:
Swap total = 0B — no safety net, OOM killer strikes without warning
Swap used >> 0 with lots of free RAM — past spike left pages in swap, causing latency
Swap near-full — next spike = OOM kill
Fix: Flush Swap Back to RAM
When swap is used but RAM is plentiful, flush to restore responsiveness:
# ONLY do this when free RAM >> swap usedsudo swapoff -a && sudo swapon -a
# Can take minutes for large swap usage (30GB ≈ 2-5 min)
Set Up zram Swap
zram creates compressed swap in RAM. Cold pages get compressed (2-3x ratio with lz4) instead of being evicted or hitting disk. Size at ~13% of total RAM.
Each claude/codex agent spawns ~4 MCP server processes (playwright npx, morphmcp npx, sh wrapper, playwright-mcp node). Stale agents leave these orphaned:
# Count MCP servers
ps aux | grep -E 'playwright|morphmcp' | grep -v grep | wc -l
# They die when their parent agent dies — kill the agent, not the MCP servers