| name | auxme-slurm-reliable-submit |
| description | Submit issue-791 style Auxme SLURM jobs with explicit config, live partition pressure checks, and max-time-safe wrapper routing. |
| category | slurm |
| kind | atomic |
| phase | implementation |
| requires_write | true |
| requires_slurm | true |
| requires_benchmark_artifacts | false |
| delegates_to | ["auxme-issue791-submit"] |
| output_schema | skill_run_summary.v1 |
| aliases | ["auxme-issue791-reliable-submit"] |
Auxme SLURM Reliable Submit
Compatibility entry point: for new routing, prefer slurm-campaign-submit for generic campaigns and auxme-issue791-submit for issue-791-specific Auxme jobs.
Purpose
Submit Auxme jobs for issue-791-style training reliably and reproducibly.
Use this when reliability, provenance, and correct config routing matter more than raw queue speed.
Workflow
- Read cluster-specific preflight and confirm the private operations overlay is configured:
SLURM/AGENTS.md
SLURM/Auxme/README.md
docs/dev/slurm_submission.md
- Confirm target intent:
- Validate the
--config path exists under configs/training/....
- Verify requested training horizon (
32k, 128k, 1m, 10m) matches the user request.
- Check live capacity before submit:
scripts/dev/auxme_partition_status.sh
scripts/dev/auxme_partition_status.sh --recommend
- Use free GPUs, pending depth, and per-user running slots only from the live output.
- Submit with explicit config:
scripts/dev/sbatch_auxme_issue791.sh --config <path> --job-name <name> SLURM/Auxme/<script>.sl
- Verify startup:
- Inspect stdout for exact config path and policy ID.
- Fail fast if wrapper-default config is used.
- Transient failure handling:
- If allocation handshake shows
Zero Bytes were transmitted or received, retry once with identical arguments.
Guardrails
- Never submit an issue-791 wrapper without explicit
ISSUE791_TRAIN_CONFIG/--config.
- Do not use stale partition status for a submission decision.
- Do not interpret infrastructure handshake failures as training quality regressions.
Output
- Chosen config path, partition/QoS decision, submit command.
- Startup provenance (stdout markers) and whether a retry was triggered.
- Final outcome (
submitted / blocked / retry suggested) with exact reason.
When to use
Use this skill for the scope named in its frontmatter description and registry metadata.