| name | cleanup-disk |
| description | Find what's eating disk on full lab servers — old checkpoints, wandb and cache junk — and produce a deletion plan. Report-only; deletion needs per-path human approval. Use when a disk is full or the user asks to free space. |
Disk cleanup (report-only)
You find candidates and produce exact commands. Never delete during
discovery. Deletion happens only after the user approves specific paths, and
then only those exact paths — no wildcards the user hasn't seen.
Find
list_hosts → filesystems ≥85% full; work per host, worst first.- On each, size the usual suspects with
run_command (timeout/head caps keep
slow NFS from hanging):
-
Old checkpoints:
find /data* "$HOME" -maxdepth 6 \( -name '*.pth' -o -name '*.ckpt' -o -name '*.pt' -o -name '*.safetensors' \) -mtime +30 -size +100M 2>/dev/null | head -40 | xargs -r ls -lh
-
wandb local run data: du -sk */wandb 2>/dev/null | sort -rn | head
(from likely project parents)
-
Caches: du -sh ~/.cache/pip ~/.cache/torch ~/.cache/huggingface ~/miniconda3/pkgs ~/anaconda3/pkgs 2>/dev/null
-
Core dumps / tmp: find /tmp "$HOME" -maxdepth 2 -name 'core.*' -size +100M 2>/dev/null
- Checkpoint judgment: the newest checkpoint per run directory is what
resume needs — keep it; older epochs are candidates. Never list a run's
ONLY checkpoint as junk. Local
wandb/ run dirs are safe once synced
(a dashboard URL exists — check with list_wandb_runs).
Report
Table per host: path · size · age · why it's probably safe · the exact removal
command. End with totals ("~340G reclaimable on ml-train-01"). Files not owned
by the user go in a separate "ask the owner" list — never suggest deleting a
labmate's files. After explicit approval of specific paths, run exactly those
commands and show the freed space (df -h before/after).
