cleanup-disk

name	cleanup-disk
description	Find what's eating disk on full lab servers — old checkpoints, wandb and cache junk — and produce a deletion plan. Report-only; deletion needs per-path human approval. Use when a disk is full or the user asks to free space.

Disk cleanup (report-only)

You find candidates and produce exact commands. Never delete during discovery. Deletion happens only after the user approves specific paths, and then only those exact paths — no wildcards the user hasn't seen.

Find

1. list_hosts → filesystems ≥85% full; work per host, worst first.
2. On each, size the usual suspects with run_command (timeout/head caps keep slow NFS from hanging):

   • Old checkpoints:

     find /data* "$HOME" -maxdepth 6 \( -name '*.pth' -o -name '*.ckpt' -o -name '*.pt' -o -name '*.safetensors' \) -mtime +30 -size +100M 2>/dev/null | head -40 | xargs -r ls -lh
     

   • wandb local run data: du -sk */wandb 2>/dev/null | sort -rn | head (from likely project parents)

   • Caches: du -sh ~/.cache/pip ~/.cache/torch ~/.cache/huggingface ~/miniconda3/pkgs ~/anaconda3/pkgs 2>/dev/null

   • Core dumps / tmp: find /tmp "$HOME" -maxdepth 2 -name 'core.*' -size +100M 2>/dev/null

3. Checkpoint judgment: the newest checkpoint per run directory is what resume needs — keep it; older epochs are candidates. Never list a run's ONLY checkpoint as junk. Local wandb/ run dirs are safe once synced (a dashboard URL exists — check with list_wandb_runs).

Report

Table per host: path · size · age · why it's probably safe · the exact removal command. End with totals ("~340G reclaimable on ml-train-01"). Files not owned by the user go in a separate "ask the owner" list — never suggest deleting a labmate's files. After explicit approval of specific paths, run exactly those commands and show the freed space (df -h before/after).