| name | sweep |
| description | Launch a hyperparameter sweep distributed across free GPUs on multiple lab servers, one detached tmux session per config. Use when the user wants to run several variants of an experiment at once. |
Distributed sweep
Turns one experiment plus a parameter grid into k placed runs. This reuses the
launch-experiment checks, then multiplies them.
- Enumerate configs from the user's grid. Show the full list; if k > 8,
get an explicit go-ahead before launching.
- Build the placement pool.
list_hosts → truly free GPUs
(inUse: false) on hosts that pass the launch checks:
- dataset present (
list_datasets)
- code at the SAME commit on every target host
(
git -C <dir> rev-parse --short HEAD via run_command; refuse mixed
commits unless the user accepts the skew)
- env smoke test per host
- disk headroom for k sets of checkpoints
Etiquette: leave ≥1 GPU free on shared hosts; never take a GPU that is
holding someone's VRAM.
- Launch each config with
start_session (background): session name
sweep_<proj>_<tag>_<i>, command prefixed CUDA_VISIBLE_DEVICES=<id>, and
a distinct output/checkpoint dir per config (inject a run tag) so runs
can't clobber each other.
- Verify all alive ~30s later —
tmux capture-pane -p -t '<name>' -S -20
for each; relaunch or report failures.
Report
The placement map: config · host · GPU · session · output dir — plus "check on
them later with /runs-status". If the pool is smaller than k: place what fits
and list the remainder explicitly — do NOT silently queue.
