메뉴
First-run setup. Detect SLURM account/partition/QoS from env and write configs/cluster/local.toml so every other skill can preflight.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Run `uv sync` and the four CI gate checks (ruff check, ruff format, pyright, pytest unit). First command after cloning. Auto-handles non-CUDA hosts via `--no-sources`.
Add a new optimizer to KempnerForge. Covers implementation, registry hook, config fields, tests, and a preset TOML.
Per-subsystem status report. What is implemented, what is tested, what is planned but unwired, and known limitations.
Walk through KempnerForge's subsystems in the order a forward pass encounters them. Starting point for anyone new to the codebase.
Submit a training job via sbatch. Wraps singlenode.sh for one node and multinode.sh for multiple. Injects account, partition, QoS, and time overrides from local.toml.
End-to-end one-GPU sanity check. Runs a short training loop to confirm torch, CUDA, NCCL, uv, and the dataloader all work before committing to longer runs.
| name | cluster-config |
| description | First-run setup. Detect SLURM account/partition/QoS from env and write configs/cluster/local.toml so every other skill can preflight. |
configs/cluster/local.toml (account, partition, shared_fs_root, etc.).This is the only skill that writes configs/cluster/local.toml. Every other skill is a read-only consumer.
Run:
uv run python scripts/check_env.py
If the exit code is non-zero, print stdout to the user verbatim and stop. The baseline checks (uv, repo layout) must pass before this skill can proceed.
Config target: configs/cluster/local.toml (gitignored) Schema reference: configs/cluster/local.toml.example Writer entry point: scripts/check_env.py do_init() Writer modes: interactive (default) or --non-interactive with per-field flags for agent-driven writes Precedence (do_init): CLI flag > env var > static default Precedence (check_env runtime): explicit --cluster-config override > env var > local.toml Discovered defaults: SLURM_CLUSTER_NAME, SLURM_ACCOUNT, SLURM_PARTITION, SLURM_QOS Fields written: [cluster].name, [cluster].scheduler, [slurm].{account, partition, qos, default_time, default_nodes, default_gpus_per_node, require_account, require_qos}, [network].{ib_interface, shared_fs_root}, [paths].checkpoints_root, [credentials].{wandb_entity, hf_user}
Dispatch on whether configs/cluster/local.toml already exists.
Collect values directly in chat, then call the non-interactive writer in one shot. Do not hand control back to a terminal prompt loop.
Read $SLURM_CLUSTER_NAME, $SLURM_ACCOUNT, $SLURM_PARTITION, $SLURM_QOS and echo them as suggested defaults.
Ask the user to confirm or override each field in one message. Use the field-by-field guidance below. Leave QoS blank if unused.
Run the writer with explicit flags:
uv run python scripts/check_env.py --init --non-interactive \
--cluster-name=<name> \
--slurm-account=<acct> --slurm-partition=<part> --slurm-qos=<qos> \
--shared-fs-root=<path> --checkpoints-root=<path> \
--wandb-entity=<entity> --hf-user=<user>
Omit any flag the user left at its default. The script falls back to env vars, then static defaults.
Verify:
uv run python scripts/check_env.py --requires slurm,multi-node
Expect OK or WARN for each tag. A MISS means a value needs correcting (usually shared_fs_root does not exist on disk or is not a distributed FS). Re-run step 3 with --force and the corrected flags.
Confirm the file is gitignored:
git check-ignore -v configs/cluster/local.toml
Should print configs/cluster/.gitignore:3:local.toml and exit 0.
Validate the existing file, surface any problems, and only rewrite if the user wants to change something.
Run validation:
uv run python scripts/check_env.py --requires slurm,multi-node
If every tag is OK or WARN: summarize what the file says (cluster, account, partition, qos, shared_fs_root, checkpoints_root) and ask the user whether any field needs updating. If they say no, stop. Do not rewrite.
If any tag is MISS, or the user wants to change a field: collect the new value(s) in chat, then rewrite with --force:
uv run python scripts/check_env.py --init --non-interactive --force \
--<field>=<new_value> [...]
Only pass the flags whose values are changing. The unchanged fields will fall back through env vars and static defaults, so pass flags for every field the user wants preserved (read the current file first and forward those values explicitly).
Re-verify with --requires slurm,multi-node.
kempner_h100.slurm if sbatch is on PATH, otherwise none. The script auto-detects if the flag is omitted.sacctmgr show assoc user=$USER. Leave QoS blank if the cluster does not require one.true): set false for clusters where sbatch runs without --account.false): set true only on clusters that reject jobs without --qos.auto unless the user knows a specific interface. check_env --requires multi-node probes /sys/class/infiniband to verify.<shared_fs_root>/checkpoints for multi-node, or <repo>/checkpoints for single-node.WANDB_API_KEY, HF_TOKEN).cat configs/cluster/local.toml and sanity-check each section against what the user specified.uv run python scripts/check_env.py --requires slurm returns OK.uv run python scripts/check_env.py --requires multi-node returns OK or WARN (WARN is fine for single-node-only users).git status shows no tracked changes under configs/cluster/local.toml.--non-interactive from an agent session. Without it, --init reads from stdin and blocks when stdin is not a TTY.--non-interactive without --force refuses to overwrite an existing file (exit 2). This is intentional: in Flow B, read the file first before rewriting.WANDB_API_KEY / HF_TOKEN env vars.--cluster-config <path> to select one./kempnerforge:smoke-test — next step after cluster-config on any GPU node./kempnerforge:slurm-launch — wraps sbatch and depends on [slurm] section being populated.