OmniHub

Generates SLURM job scripts with omnihub-generate-job and runs them with sbatch. Use when the user wants to generate a job, run a job, use omnihub-generate-job, sbatch, set app-config, partition, or run a single OmniHub job.

Documents OmniHub repo layout, app configs, cluster and tools config, results layout, and when to use runner manual vs torchrun. Use when the user asks about conventions, app config, cluster config, tools config, results layout, or runner manual vs torchrun.

Creates and runs parameter sweeps with omnihub-sweep using templates, sweep-dir, and optional resume. Use when the user wants to run a sweep, use omnihub-sweep, create a parameter sweep, use a sweep template, set sweep-dir, or do a dry-run sweep.

Periodically checks the results of the current or specified OmniHub job: inspects results_dir and logs, alerts on errors, and reports success. Use when the user wants to monitor a job, check for errors in results or logs, see job status, or get a success/failure report for an OmniHub job.

Primus pretraining via OmniHub -- job generation, config overrides, MASTER_PORT, HF_HOME, GPU arch (FP8), and debugging multinode hangs. Use when the user works with applications/primus-pretrain/, pretrain_wrapper.py, Primus config, primus_turbo, or multinode Primus training.

Processes and indexes OmniHub job results with omnihub-process and omnihub-index to produce index.csv and per-job processed-data. Use when the user wants to post-process results, run omnihub-process, omnihub-index, build index.csv, or work with processed-data.