메뉴
Generates SLURM job scripts with omnihub-generate-job and runs them with sbatch. Use when the user wants to generate a job, run a job, use omnihub-generate-job, sbatch, set app-config, partition, or run a single OmniHub job.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Documents OmniHub repo layout, app configs, cluster and tools config, results layout, and when to use runner manual vs torchrun. Use when the user asks about conventions, app config, cluster config, tools config, results layout, or runner manual vs torchrun.
Creates and runs parameter sweeps with omnihub-sweep using templates, sweep-dir, and optional resume. Use when the user wants to run a sweep, use omnihub-sweep, create a parameter sweep, use a sweep template, set sweep-dir, or do a dry-run sweep.
Periodically checks the results of the current or specified OmniHub job: inspects results_dir and logs, alerts on errors, and reports success. Use when the user wants to monitor a job, check for errors in results or logs, see job status, or get a success/failure report for an OmniHub job.
Primus pretraining via OmniHub -- job generation, config overrides, MASTER_PORT, HF_HOME, GPU arch (FP8), and debugging multinode hangs. Use when the user works with applications/primus-pretrain/, pretrain_wrapper.py, Primus config, primus_turbo, or multinode Primus training.
Processes and indexes OmniHub job results with omnihub-process and omnihub-index to produce index.csv and per-job processed-data. Use when the user wants to post-process results, run omnihub-process, omnihub-index, build index.csv, or work with processed-data.
| name | generate-job-and-run |
| description | Generates SLURM job scripts with omnihub-generate-job and runs them with sbatch. Use when the user wants to generate a job, run a job, use omnihub-generate-job, sbatch, set app-config, partition, or run a single OmniHub job. |
-A <account> on the sbatch command (e.g. sbatch -A ven114 job.slurm). Do not add account lines to the generated OmniHub script; keep account selection in the dispatch command. If the first sbatch fails for account reasons, obtain the correct project name (ask the user, derive from stdout, or try site-appropriate discovery such as sshare / sacctmgr) and resubmit with -A.$PWD or $HOME/omnihub).applications/hf-infer/config-example.yaml).| Argument | Description | Default |
|---|---|---|
| --app-args | Application-specific CLI args (e.g. --VllmArguments.model=meta-llama/Meta-Llama-3.1-8B) | "" |
| --cluster | Cluster name: hpcfund, frontier | hpcfund |
| --partition | Partition / queue name (see config/.yaml) | cluster default |
| --num-nodes | Number of nodes | 1 |
| --runner | Distributed runner: manual, torchrun. Required for multi-node. | -- |
| --tools | One or more profiling tools (see config/tools/*.yaml) | [] |
| --time-limit | Job time limit (e.g. 30m, 2h) | 1h |
| --output | Output script path (default: stdout) | -- |
| --platform | apptainer, docker | apptainer |
| --rocm-version | ROCm version | 7.1.0 |
| --include-nodelist | Nodes to include (space-separated) | [] |
| --exclude-nodelist | Nodes to exclude (space-separated) | [] |
| --tasks-per-node | Override tasks per node (e.g. 1 for Primus) | -- |
Generated jobs automatically run scripts/sanity-check.sh before the main workload. It performs three checks: ROCm availability (rocminfo), GPU compute-mode (warns on exclusive partitions), and PyTorch NCCL broadcast + all_reduce via torchrun (uses scripts/sanity_torch_dist.py).
If the sanity step fails (e.g. NCCL connection refused, traffic to link-local 169.254.x.x addresses), fix cluster networking first (NCCL_SOCKET_IFNAME, GLOO_SOCKET_IFNAME, MASTER_ADDR reachability) before debugging application code.
--runner. Use manual when the application starts its own distributed launcher (e.g. Primus pretrain); use torchrun when OmniHub should launch torchrun.For Primus-specific runner guidance (tasks-per-node, MASTER_PORT, HF_HOME, GPU arch), see the primus-pretrain skill.
./omnihub-generate-job --omnihub-dir $PWD \
--app-config applications/vllm-latency/config-example.yaml \
--cluster hpcfund --partition mi3008x \
--tools omnistat --output job.slurm
sbatch job.slurm
Multi-node with manual runner (Primus -- always --tasks-per-node 1):
./omnihub-generate-job --omnihub-dir $PWD \
--app-config applications/primus-pretrain/config-example.yaml \
--num-nodes 2 --runner manual --tasks-per-node 1 --partition mi2508x \
--output job.slurm
sbatch job.slurm
sbatch failsMany systems reject jobs unless a valid charge account (project) is set. Typical errors mention Invalid account, association, no valid account, or bank.
If submission fails on the first try, the agent should:
Infer or obtain the account — Ask the user for their project / allocation ID when unsure; optionally try sshare or sacctmgr show assoc user=$USER format=Account%20 (site-dependent); $SLURM_JOB_ACCOUNT from a recent successful job may be the right value.
Resubmit with -A on sbatch — Do not modify the generated job script for account. Use:
sbatch -A YOUR_PROJECT_HERE job.slurm
applications/*/config-example.yamlconfig/<cluster>.yaml (e.g. config/hpcfund.yaml)config/tools/*.yaml (tools are auto-discovered; run omnihub-generate-job --help for the current tool list)