메뉴
Launch a training run on the best lab server — picks a host by data locality and free GPUs, verifies code and environment, starts it in a detached tmux session. Use when the user wants to run, launch, or train something on the servers.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Find what's eating disk on full lab servers — old checkpoints, wandb and cache junk — and produce a deletion plan. Report-only; deletion needs per-path human approval. Use when a disk is full or the user asks to free space.
Compare experiment results across servers — differing hyperparameters and final metrics from local wandb data, with dashboard links. Use for "which ablation won" or comparing runs of a project.
Morning briefing across all lab servers — what finished or crashed overnight, which GPUs are free, which disks are filling up. Use when the user asks for a briefing, "what happened overnight", or overall lab/server status.
Reproduce a working project environment from one lab server on another — code at the same commit, a CUDA-compatible python env, and a smoke test. Use for "make this run on host B too" or setting up a new server.
Check on the user's running experiments across all servers — progress, loss, ETA, stalled or crashed. Use for "how are my runs", "is training still going", or checking a specific experiment.
Launch a hyperparameter sweep distributed across free GPUs on multiple lab servers, one detached tmux session per config. Use when the user wants to run several variants of an experiment at once.
| name | launch-experiment |
| description | Launch a training run on the best lab server — picks a host by data locality and free GPUs, verifies code and environment, starts it in a detached tmux session. Use when the user wants to run, launch, or train something on the servers. |
Standard placement procedure. Ask only for what you cannot infer: the command (or repo + script) and the dataset it needs.
Data gravity first. list_datasets on candidate hosts — prefer a host
that already has the dataset. Copying terabytes is a last resort and needs
explicit user approval (then use copy_path / plan_migration).
Compute. list_hosts — among hosts with the data, find truly free GPUs
(inUse: false, i.e. <1.5GB VRAM held). Never place work on a GPU that is
holding someone's memory, even at 0% utilization. On shared hosts leave at
least one GPU free when you can.
Disk. The chosen host needs headroom for checkpoints — flag if the target filesystem is over ~85%.
Code. Verify the project on that host with run_command:
git -C <dir> rev-parse --short HEAD && git -C <dir> status --porcelain | head
Dirty tree or missing repo → tell the user; don't silently train on unknown code.
Env smoke test (adapt to the project's env activation):
cd <dir> && python -c 'import torch; print(torch.__version__, torch.cuda.is_available())'
Launch with start_session: the chosen host and dir, the command
prefixed with CUDA_VISIBLE_DEVICES=<ids>, and a short greppable session
name (e.g. train_<proj>_<mmdd>).
Verify it's alive ~20s later:
tmux capture-pane -p -t '<name>' -S -30
Confirm it got past imports and data loading. If it died, show the error and fix or report.
Host, GPU ids, tmux session name, code commit, and how to look at it (a hopmux
tab, or tmux attach -t <name>). If nothing was free: say which hosts free up
soonest (running sessions + their progress) — do NOT queue or wait silently.