pre-commit-audit

Comprehensive pre-commit verification checklist with five independent responsibilities. (1) Launcher path coverage - verify a change to any launcher-chain file preserves correct behavior across all 16 combinations of entry point × launch mode × stack (Steps 1-4 + 5.1). (2) Ancillary scripts smoke - syntax / help / read-only / caller checks for any `.sh` or `.py` outside the launcher chain (Step 5.2; covers analysis utilities, sourced libraries, debug helpers, sweep tooling). (3) Code quality and design review (Step 6) - propose-first surface of code smells (duplication, long functions, magic numbers, deep nesting, unclear naming, primitive obsession, etc.) and design-decay signals (5th case in a switch, N-th env-var read, hand-rolled retry loops); auto-fix mechanical findings, hold design-shaped ones for explicit go-ahead. (4) Docs / comments / format-consistency (Step 7) - check any commit for stale prose, trailing-comment alignment drift, broken anchors / missing files in links, drifted cross-references, an

Aller à l'installation

Dépôt: AMD-AGI/maxtext-slurm
Dernière activité de la source: 9 mai 2026 à 18:07
Langue détectée de SKILL.md: anglais
Étoiles: 29
Forks: 2

Options d'installation

Le prompt qui vérifie d'abord la source est sélectionné par défaut. Vous pouvez passer à une commande directe ou télécharger une copie locale.

Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.

Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.

npx skills add https://github.com/AMD-AGI/maxtext-slurm --skill pre-commit-audit

La commande reste sur une seule ligne. Faites défiler horizontalement pour la vérifier avant de la copier.

Vous préférez une copie locale ? Téléchargez les fichiers actuellement disponibles dans SkillsMP.

Vérifiez les fichiers source

Lisez SKILL.md et les fichiers associés affichés par SkillsMP avant de décider de l'installer.

SKILL.md

Instructions source · Aperçu en lecture seule

Retour à la recherche Retour à l’installation

model-config-guide

Create GPU config files to support existing MaxText model definitions on AMD GPU clusters. Use when the user wants to add a model, create a config, support a new model, or asks about model configs, parallelism, batch size, OOM, quantization, or .gpu.yml files.

2026-05-09

profile-drill

Direct per-kernel time analysis from JAX / TensorFlow xplane traces via `utils/profile_drill.py`. Use when the user asks for a per-kernel breakdown, step-time composition, cross-variant kernel comparison, main-stream-blocking analysis, or any question that needs ground-truth kernel timings below what TraceLens reports. Triggers include "xplane", "trace.json.gz", "input_scatter_fusion", "RaggedAllToAllKernelImpl", "ncclDevKernel", "step − total kernel", "main-stream-busy", "profile drill-down", or suspicion that TraceLens numbers are off by ~1.5–2×.

2026-05-09

batch-sweep

Four sweep operations: (1) Model perf sweep — find optimal batch size / TGS for a model. Use for: sweep batch size, tune TGS, benchmark throughput, find optimal config. (2) Node perf sweep — compare per-node GPU performance to find outliers. Use for: check nodes, node performance, find slow node, compare nodes. (3) Node network health sweep — detect inter-node network issues via multi-node bisection. Use for: network health, IB issues, RCCL problems, node pair testing, isolate network problem. (4) Model sweep — run all model configs on one or two commits. Use for: regression test, validate commit, test all models, smoke test, CI, compare branches.

2026-05-09

Tout voir dans ce dépôt

Pre-commit audit

Comprehensive pre-commit verification of a code change in this repo. The skill bundles five independent audits that share a single output format and a single "ready / not ready" recommendation:

Launcher path coverage (Steps 1–4 + 5.1) — when the change touches the launcher chain, verify all 4 entry points × 2 launch modes × 2 stacks = 16 combinations still work. Most agents verify the path they personally cared about and miss the other 15.

Ancillary scripts smoke (Step 5.2) — when the change touches any .sh / .py outside the launcher chain (analysis utilities, sourced libraries, debug helpers, sweep tooling), syntax-check, help-smoke, read-only invoke, and cross-check callers. The 16-cell launcher matrix can be all green and a renamed function in a sourced library can still leave utils/analyze_job.py broken on the next invocation.

Code quality and design review (Step 6) — on any commit that changes code, surface code smells (duplication, long functions, magic numbers, deep nesting, primitive obsession, etc.) and design-decay signals (5th case in a switch, N-th env-var read in different files, hand-rolled retry loops). Propose-first: auto-fix the mechanical ones, hold the design-shaped ones for go-ahead.

Docs / comments / format-consistency (Step 7) — on any commit, check that prose stays in sync with code: stale in-file comments, trailing-comment alignment drift, doc cross-references that point at moved/renamed code, link-policy compliance (broken anchors, missing files, scheme conventions), and (especially) this skill itself drifting out of sync with the launcher topology. Runs after code quality (post-rename / post-extraction) and before the leak scan (so its prose edits feed into the security gate).

Sensitive-info leak scan (Step 8) — on any commit, scan the post-Steps-6+7 state for cluster hostnames, internal IPs, vendor mount paths, hard-coded credentials, internal job IDs, and other deployment-specific identifiers that don't belong in this open-source repo. Runs last — final security gate before commit.

Step 9 emits the launcher matrix (when path coverage ran) plus the per-step audit lists.

The methodology behind the launcher-path-coverage half (Steps 1–4 + 5.1) is the one used to verify the fix/training-teardown series of fixes (os._exit in monkey_patch_maxtext.py, bash instead of exec in _train_with_ray.sh, guarded ray.kill in _ray_actor.py) — three changes that each affected a different subset of the 16 combinations and that all needed to land before any single combination was provably clean. The ancillary-smoke, leak-scan, and doc-audit halves are general-purpose and apply to any commit that touches scripts / docs / comments.

Pre-commit audit

Comprehensive pre-commit verification of a code change in this repo. The skill bundles five independent audits that share a single output format and a single "ready / not ready" recommendation:

Step 9 emits the launcher matrix (when path coverage ran) plus the per-step audit lists.

Edge case

Diagnostic

Fix pattern

trap … EXIT followed by exec

The trap dies with the replaced bash. Cleanup never runs.

Replace exec script.sh with bash script.sh "$@"; exit "$?", or remove the trap before exec (trap - EXIT).

Unprotected call in a Python finally:

If the call raises, sys.exit(rc) is suppressed and Python exits 1 with a traceback even on success.

Wrap in try: …; except Exception: pass.

subprocess.Popen + .wait() without pipe management

Large stdout/stderr can deadlock the child.

Use stdout=subprocess.DEVNULL if not needed, or Popen(..., stdout=PIPE).communicate().

{ … } | tee under set -o pipefail

A transient tee failure aborts the whole pipeline (including the upstream commands).

Append || true, or capture ${PIPESTATUS[0]} and only fail on the upstream's status.

exec > >(tee -a …) process substitution

tee is not waited on; SIGPIPE / NFS-write errors in tee can kill the writer or truncate logs.

Use a foreground pipeline with ${PIPESTATUS[0]}, or accept the limitation and document.

pkill -f <pattern>

Matches across the whole PID namespace. Inside a Docker container without --pid=host this is bounded; outside a container or with --pid=host, it can kill another job's processes (very real once observed: pkill -f tensorboard killing the user's standalone TB viewer on the same host).

Track and kill by recorded PID, or anchor the pattern to $JOB_DIR / $OUTPUT_PATH.

os._exit(rc) skipping atexit handlers

Side effects normally run during sys.exit (Orbax flush, TB final flush, JAX device release) are skipped.

Verify the leaf calls (e.g., maxtext_train.main) commit those side effects before returning, and add os.sync() for kernel page-cache safety on async-mounted NFS. Provide an env-var opt-out (MAXTEXT_FAST_EXIT=0).

SIGTERM cascade from one rank's failure

When one rank's training subprocess dies, srun SIGTERMs the rest, producing all-exit 143 in the log. The first non-zero rc in wait order is the cause; the rest are noise.

Surface the first rank to exit non-zero (not the last), and treat exit-143 cascades after a non-walltime failure as secondary.

exit $? with an intervening command

$? is clobbered to the intervening command's status.

_rc=$?; intervening_command; exit "$_rc".

Race in pre-init helpers (_maybe_preinit_jax_distributed, profiler shim)

If a helper relies on env vars that aren't set in some flow (e.g., LOCAL_RANK is only set in 1-gpu-per-process), the helper must be a no-op there.

Guard with [[ -z "${LOCAL_RANK:-}" ]] && return or equivalent in Python.

stop_ray_cluster running twice

On K8s / in_container_run.sh it runs in two traps (the script's and _train_with_ray.sh's).

Make sure each step is idempotent (ray stop --force ‖ true, pkill ‖ true).

--time= walltime under-budgeted

Slurm SIGTERMs all ranks at exact --time=. Step T-1 sometimes makes it just before SIGTERM, sometimes after.

Use the adaptive formula walltime = ceil(1.5 × (compile_overhead + steps × step_time_seconds) / 3600) hours — measure step time from a 15-step calibration run, multiply by 1.5× to absorb step-time variance and Ray/JAX teardown overhead.

Smell

Diagnostic

Default action

Duplicated code

Same 5+ lines appear in 2+ places, or same logic with minor variations

Propose extraction for cross-file duplicates (suggest helper name + new home); auto-fix only when both sites are in the same file and the extraction is mechanical

Long function

> 50 lines for shell, > 80 for Python, or any function whose purpose can't be summarized in one sentence

Propose breakdown; suggest helper-function names

Long file

> 500 lines for shell, > 800 for Python; multiple unrelated responsibilities

Propose split with proposed new file boundaries

Deep nesting

More than 3 levels of if / for / case indentation

Auto-fix when the fix is mechanical (early-return guards); propose otherwise

Magic numbers

Numeric literals without context (sleep 5, timeout=30, port=8265)

Auto-fix: name as a top-of-file constant with a one-line rationale

Unclear / generic naming

tmp, x, do_thing, helper, process, single-letter outside loop indices

Propose rename (action-and-target form: read_config_file, not helper)

Dead code

Functions never called; branches under permanent if false / 0; commented-out blocks > 2 lines

Auto-fix: delete. If the comment is preserved intentionally as documentation, convert it to a normal explanatory comment

Inconsistent abstraction levels

One function mixes high-level orchestration with low-level operations (shell function does both git fetch and per-byte string parsing)

Propose: extract the lower-level work into a helper

Stringly-typed config / enum

Multiple if [[ "$mode" == "X" ]] branches scattered across the codebase

Propose: name the modes as constants in a single file; have callers reference them

Repeated regex / pattern

Same regex appears 3+ times

Propose: define as a top-level constant in the file that owns it

Comment-as-documentation overlap

Comment says exactly what the next line says (# increment counter next to i += 1)

Auto-fix: delete the redundant comment; keep only intent / rationale / non-obvious-edge-case comments

Primitive obsession

Function signature with 5+ raw-string args

Propose: introduce a config struct / sourced-shell config block; refactor callers

Inappropriate intimacy

One file directly mutates another module's globals or imports private helpers (_foo); test code reaches into production internals

Propose: add a public accessor / setter; tighten the API surface

Inconsistent style with neighbors

New function uses spaces where surrounding file uses tabs; new helper is snake_case where the file is camelCase

Auto-fix: match the file's existing convention

Hand-rolled retry / backoff loop

New for i in $(seq 1 N); do ...; sleep ...; done pattern that already exists elsewhere

Propose: extract into with_retries (or use the existing helper if one is in utils/)

Hardcoded path that should be a variable

A new /path/to/something that ought to be derived from $JOB_WORKSPACE, $OUTPUTS_DIR, etc.

Auto-fix: replace with the appropriate variable

Section to check

When it goes stale

What to update

Frontmatter description (file list, trigger keywords)

A new launcher-chain file was added, an old one was renamed / removed, or a new failure-class was identified that warrants its own trigger keyword

Add / remove the file name. Add a new trigger keyword if the change generalizes

The 16-cell matrix

A new entry point, launch mode, or stack option landed; any axis was removed

Update the cell count. Note: the math (4 × 2 × 2 = 16) is hard-coded in the prose — search and update both

"Two ways the container is reached" table

A new entry point bypasses both existing pairs (e.g., a serverless / lambda entry)

Add a third pair, with its container-entry mechanism and trap stack

"Four post-_train.sh flows" table

A new launch mode or stack option lands; the dispatch logic in _train.sh changes

Renumber and add the new flow; update the cross-references in §3 (Step 3)

Step 2 file → flow lookup table

Any change touching the launcher-chain files

Confirm the table row for each touched file still accurately lists the flow numbers and entry-pairs the file participates in. Add rows for newly-introduced files

Step 4 cross-cutting edge cases

The change discovers a new edge case worth memoizing for the next agent, or invalidates one that's listed

Add a new row when a new bug class surfaces; mark as "fixed by <commit>" rather than deleting if removal would lose context for old branches

Step 5.1 launcher smoke recipe

The smoke commands no longer exercise the changed code (e.g., a flag default changed making dataset_type=synthetic insufficient to hit the changed path)

Update the example invocation, or add a second smoke variant

Step 5.2 ancillary scripts table

A new analysis utility lands; an existing one was renamed; an existing one gained a --dry-run mode that should now be the recommended smoke

Add / rename the row; promote read-only invocations to --dry-run where supported

Step 6.1 code-smell scan table

A new code smell becomes recurrent in the codebase (e.g., a build-system migration introduces a new pattern of inconsistency that ought to be flagged), or an existing row is no longer applicable

Add / remove rows; update the default-action column when a smell that used to be auto-fixable becomes propose-only or vice versa

Step 6.2 holistic trajectory thresholds

The codebase has grown enough that the 1/2/3/4+ thresholds no longer match the right "how many is too many" intuition (e.g., the codebase deliberately tolerates more inline duplication than 3)

Adjust the threshold table; document the rationale

Step 6.3 design-choice revisit triggers

The codebase establishes a new pattern that ought to be revisited periodically (e.g., a third retry-loop variant lands), or removes an existing one

Add / remove triggers

Step 6.4 architectural-layer map

A new tier is introduced (e.g., a new external scheduler shim, a new observability sidecar), or an existing layer's responsibilities are reshuffled

Update the layer mapping; if docs/architecture.md and docs/extensibility.md both need updates, propose those as part of the same change

Step 7.1 stale-comment patterns

The change introduces a new comment-drift class worth memoizing (e.g., a build-system-specific comment style appears that needs alignment, or a new convention for // SAFETY: blocks)

Add a new row to the stale-comment-patterns table

Step 7.2 link policy table

The repo adopts a new doc renderer with different slug rules, a new external-link convention (e.g., always-archived URLs), or a new internal-doc layout (e.g., docs/v2/... parallel tree)

Update the policy table; update the slug algorithm if the renderer changed

Step 8 leak patterns and allowlist

The change introduces a new convenience default that shouldn't trigger the leak scan, or a new sensitive identifier that should

Add an allowlist row (with rationale) or a new pattern row

Output format items

Any of the above changes adds or removes evidence the agent must collect

Update the numbered list of items in §"Output format"

"Notes on this skill itself"

Anything material above changed

Re-read the whole bottom-of-file note block; rewrite the bullet describing whichever section just shifted

Pattern (ripgrep -P)

What it leaks

Fix

\b(chi|<your-prefix>)\[?\d+

The deploying cluster's hostname pattern

Replace with <node>, <nodelist>, <host> (or node[N-M] for syntax examples)

\b(?:\d{1,3}\.){3}\d{1,3}\b

Any concrete IP — public or private — that wasn't deliberately documented

Allowed: 127.0.0.1, 0.0.0.0, RFC5737 doc IPs (192.0.2.x, 198.51.100.x, 203.0.113.x), and explicitly-marked RFC1918 placeholders. Anything else → replace with <host>, <private-ip>, <public-ip>, or one of the RFC5737 ranges in docs

\b1[4-5]\d{3}\b (current cluster's job-ID range — adjust to your range) in code or comments

Refers to internal job tracking; doesn't generalize and dates the comment

Remove the job ID and describe the symptom instead (e.g., "asymmetric rank-N exit-1 on multi-node sweeps"). Ok in outputs/, log files, and external docs not shipped in the repo. False positives: plain numbers in this range often appear as timeout values (14400 = 4 h), byte counts, or port numbers — review every hit; if it isn't preceded by job, JOB_ID, sbatch, squeue, or appears as a timeline reference, it's almost certainly not a job ID

partition\s*=\s*["']?[a-z0-9_-]+["']? hard-coded outside ${VAR:-default}

Implies the deploying cluster's partition layout

Use ${PARTITION:-} or pass via the existing flag plumbing

(?:/mnt/[^/\s]+/[a-z0-9_-]+|/home/[a-z0-9_-]+|~[a-z0-9_-]+)

User home / scratch / vendor-specific mount paths (e.g., /mnt/<vendor>/<user>, /home/<user>/...)

Use $JOB_WORKSPACE, $HOME, $OUTPUTS_DIR, or repo-relative paths. Vendor-specific mount roots like /mnt/<storage-product-name>/... should be replaced with <your-shared-path> or set via env var

(?i)(token|secret|password|apikey|api_key)\s*[:=]\s*["']?[A-Za-z0-9_\-]{16,}

Hard-coded credential

Move to .local.sh (already gitignored), .env, or a secrets manager

https?://(?!github\.com|huggingface\.co|docs\.|en\.wikipedia\.org|tools\.ietf\.org|datatracker\.ietf\.org)[a-z0-9.-]+\.(?:com|org|net|io|ai|local|internal)

Org-private URLs / image registries / Slack / Linear

Genericize to ${DOCKER_REGISTRY}, <your-tracker-url>, <your-chat-channel>

(?<=[\s'"=])[a-z][a-z0-9_-]*\.(?:local|internal|corp|prod|stage)\b

Internal DNS suffixes

Replace with <host> or genericize via env var

\bC\d{4,}\b (Slack channel) \bT\d{8,}\b (Slack team) \b[A-Z0-9]{8,}\b patterns matching common ID schemas

Cross-tool linkage to org-internal resources

Remove or genericize

Axis	Values
Entry point	`submit.sh` (Slurm), `k8s_submit.sh` (Kubernetes), `run_local.sh` (single-host), `in_container_run.sh` (already-in-container)
Launch mode	1-node-per-process (default; `LOCAL_WORLD_SIZE=1`), 1-gpu-per-process (`ONE_GPU_PER_PROCESS=true`; `LOCAL_WORLD_SIZE=8`)
Stack	`RAY=0` (no Ray, no observability), `RAY=1` (Ray actor + Prometheus + TensorBoard + metrics_exporter)

Pair	Container-entry mechanism	EXIT-trap stack inside the container
`submit.sh`, `run_local.sh`	`_container.sh` runs `docker run`; the container's `bash -lcx FINAL_CMD` invokes `$MAXTEXT_RUNNER` (= `_train_with_ray.sh` if `RAY=1`, else `_train.sh`)	`_train_with_ray.sh`'s own EXIT trap (calls `stop_ray_cluster`) — only when `RAY=1`; otherwise no in-container trap
`k8s_submit.sh` (→ `_k8s_job.sh`), `in_container_run.sh`	the pod / shell is the container; no `_container.sh` `docker run` involved. `_k8s_job.sh` `exec`s into `in_container_run.sh` for rank 0 (and pipes through `tee` for non-rank-0)	`in_container_run.sh`'s `_cleanup_and_summary` trap (calls `stop_ray_cluster` when `RAY=1`) → `_train_with_ray.sh`'s own trap (also `stop_ray_cluster` when `RAY=1`) — redundant but idempotent

#	Launch	Stack	What runs in `_train.sh`
(1)	1-node-per-process	`RAY=0`	`python3 -u utils/monkey_patch_maxtext.py "${TRAIN_ARGS[@]}"` — single Python process, sees all local GPUs
(2)	1-gpu-per-process	`RAY=0`	`for i in 0..LOCAL_WORLD_SIZE-1: python3 -u utils/monkey_patch_maxtext.py … &` then `wait` loop captures first non-zero rc
(3)	1-node-per-process	`RAY=1`	`python3 -u _ray_actor.py …` → driver creates one `MaxTextTrainerActor` per node → `actor.run_training` does `subprocess.Popen(monkey_patch_maxtext.py)` + `p.wait()`
(4)	1-gpu-per-process	`RAY=1`	same as (3), but the actor calls `_fan_out_one_gpu_per_proc` which Popens `LOCAL_WORLD_SIZE` subprocesses and waits on all

File	Reached by entry-pair (1) `submit.sh`/`run_local.sh`	Reached by entry-pair (2) `k8s_submit.sh`/`in_container_run.sh`	Used by post-`_train.sh` flow
`submit.sh`	yes	—	—
`_job.sbatch`	yes	—	—
`_container.sh`	yes	—	—
`k8s_submit.sh`	—	yes	—
`_k8s_job.sh`	—	yes	—
`in_container_run.sh`	—	yes	—
`run_local.sh`	yes (single-host variant)	—	—
`utils/run_setup.sh`	yes	yes	—
`_train_with_ray.sh`	yes (RAY=1)	yes (RAY=1)	(3), (4)
`_train.sh`	yes	yes	(1), (2), (3), (4)
`_ray_actor.py`	yes (RAY=1)	yes (RAY=1)	(3), (4)
`utils/monkey_patch_maxtext.py`	yes	yes	(1), (2), (3), (4)
`utils/ray_cluster.sh`	yes (RAY=1)	yes (RAY=1)	(3), (4) (head- and worker-side)
`utils/prometheus.sh`	yes (RAY=1, head only)	yes (RAY=1, head only)	(3), (4)
`utils/metrics_exporter.sh`	yes (RAY=1)	yes (RAY=1)	(3), (4)
`utils/coredump.sh`	yes	yes	(1), (2), (3), (4)
`utils/detect_ip.sh`	yes (read by `_job.sbatch`, `_container.sh`, `utils/ray_cluster.sh`)	yes (read by `_k8s_job.sh`, `utils/ray_cluster.sh`)	(3), (4) (binds Ray GCS to private IP)
`utils/stage_timeout.sh`	yes (orchestration tier, in `_job.sbatch` only)	—	—

Script	Smoke invocation
`utils/analyze_job.py`	`utils/analyze_job.py outputs/<recent-job>/<log>.log \| head -30`
`utils/perf_server.py`	`utils/perf_server.py --port 8080 &` then `curl -s localhost:8080/api/jobs \| head`; kill
`utils/profile_drill.py`	`utils/profile_drill.py outputs/<job-with-xplane>/.../*.xplane.pb 2>&1 \| head`
`utils/slurm_job_monitor.sh`	`utils/slurm_job_monitor.sh -j <recent-jobid> --dry-run` (or no-args usage)
`utils/tail_job_log.sh`	`utils/tail_job_log.sh <recent-jobid> 2>&1 \| head -5`
`utils/tag_tgs.sh`	`utils/tag_tgs.sh outputs/<recent-job>/ 2>&1 \| head`
`utils/tgs_tagger.py`	`utils/tgs_tagger.py outputs/<recent-job>/<log>.log`
`utils/cleanup_artifacts.sh`	`utils/cleanup_artifacts.sh --dry-run` (must support `--dry-run`; if not, document the proposed invocation)

pre-commit-audit

Options d'installation

Vérifiez les fichiers source

Pre-commit audit

Pre-commit audit

Audit posture: holistic, not per-file

When to use this skill — decision tree

The 16-cell matrix

Decomposition: what the 16 cells actually exercise

Two ways the container is reached

Four post-`_train.sh` flows

Audit procedure

Step 1 — identify the change

Step 2 — map the change to flows

Step 3 — for each affected flow, walk the chain

Step 4 — check the cross-cutting edge cases

Step 5 — smoke-test the changed code paths

5.1 — Launcher-chain smoke (when Steps 1–4 produced affected flows)

5.2 — Ancillary scripts and utilities smoke (when the change touches anything outside Step 2's lookup table)

Step 6 — code quality and design review (propose-first; modify only with explicit go-ahead)

6.1 — Per-change code-smell scan

6.2 — Holistic code-smell trajectory check (walk outward from the diff)

6.3 — Per-change design-choice revisit

6.4 — Architectural fit and boundary check (holistic)

6.5 — Output

Step 7 — docs, comments, and format-consistency check (mandatory before commit)

7.1 — In-file comments in the touched files

7.2 — Cross-references in docs and other skills

Markdown link policy (applies to every md file the change touches)

Slug algorithm (github-slugger)

Audit recipe (Python — handles fenced code blocks, inline code spans, and the slug algorithm correctly)

Symbol and file-path cross-references

7.3 — Self-consistency: does this skill itself need updating?

7.4 — Output

Step 8 — sensitive-info leak check (mandatory before commit)

Step 9 — output the matrix and audit lists

Output format

Notes on this skill itself

Repo-wide count	Recommendation
1 (just this change)	acceptable as-is; record nothing
2	acceptable; mention the existing siblings in the audit so the next change has context
3	propose a follow-up extraction commit (separate from the change at hand)
4+	propose extraction now, blocking on user confirmation; the cost of N+1 is low only if N is < 3

Trigger	When to escalate	Question to raise
Switch with 5+ cases	The change adds the 5th case to a `case` / `if-elif` / `match` block, or extends a 4-case block to 5	Should this be a dispatch table / lookup map / polymorphism instead of inline branching?
`try / except: pass` proliferating	The change adds a 4th unprotected `except Exception: pass` near siblings that already use the same pattern	Are some of these errors actually actionable? Should we catch specific exception types? Logging the error before swallowing?
New global variable	The change adds a new `EXPORT FOO=...` at module scope	Can this be passed through env vars, function args, or a config struct? Globals couple unrelated callers.
Same env var read in 3+ places	The change introduces or grows a third callsite reading the same env var	Centralize the read in one place; have callers receive the parsed value via function arg
Helper function with 6+ args	The change grows a helper to take a 6th positional / keyword arg	Time for a config struct or builder pattern?
Parallel data structures kept in sync manually	Two arrays / dicts / lists that must stay in lockstep, with no enforcement	Collapse into a single struct-of-fields, or document the invariant and add an assertion
N-th place doing the same thing differently	The change introduces a 3rd implementation of an operation that's done 2 other ways elsewhere (e.g., 3 different ways to check Slurm job status)	Pick one; standardize the rest as follow-up
Conditional that always-true / always-false in current setup	New `if has_X: …` where `has_X` is always true on this stack	Either make `has_X` a real switch the user can flip, or remove the conditional entirely

Stale-comment pattern	What to look for	Fix
Behavior drift	Comment describes what the code used to do (e.g., "exec into _train.sh" beside a `bash _train.sh; exit "$?"` line)	Rewrite the comment, or delete it if the new code is self-evident
Symbol drift	Comment names a function / variable / file path that the change renamed or moved	Update the reference, or replace with a description that doesn't pin specifics
Job-ID / dated references	Comment cites a specific incident ("see job 14837") that the code no longer relates to	Replace with the symptom ("asymmetric rank-N exit-1 on multi-node sweeps") and let the symptom outlive the incident
TODO / FIXME / HACK / XXX referencing the now-fixed problem	Comment explicitly says "TODO: fix the X bug" — and the change is the X fix	Delete the TODO; if it referenced a follow-up, file an issue and replace with a one-line link
Counter-example comments	Comment has a "DO NOT do X because Y" directive — but the change just made X safe	Either delete the warning, or update it to describe a still-relevant guardrail
Line-number / size pinning	Comment cites byte / line / step counts that the change shifted (e.g., "the next 12 lines are…")	Re-count and update, or rewrite to stop pinning
Trailing-comment alignment drift	Two or more consecutive lines in a usage / option / table block had their trailing `#` (or `//`) comments aligned at the same column, and the change shifted the pre-comment text on one of them — leaving the comments staircased (e.g., one column off)	Re-pad the spaces so all `#` line up at the same column. The longest pre-comment text in the block sets the comment column; pad shorter lines with spaces to match. Verify with `head -<N> <file> \| cat -A` (the `$` end-markers expose the trailing-space layout). Examples: option lists in shell scripts, env-var tables in `train_env.sh` / `container_env.sh`, ASCII tables in markdown

Rule	Required form	Forbidden forms
Internal cross-doc	Relative path, no leading `./`	Absolute paths (`/maxtext-slurm/docs/...`), `file://`, repo-root `/...`
Cross-tier internal (skill → doc, doc → repo root)	`..`-anchored relative path (e.g., `../docs/k8s-job-submission.md` from a skill, `../../README.md` from a nested skill)	Absolute paths
External	`https://` URL, no trailing `/` redundancy	`http://` (insecure), bare URLs, `www.` without scheme
Anchor refs	GitHub-slugger kebab-case slug of the heading text — see "Slug algorithm" below	Custom slugs, hash-encoded entities, raw heading text
Link style	Inline `[text](target)` form	Reference-style `[text]: target` (the repo doesn't use this; mixing styles hurts grep-ability)
First-mention linkability	When linkable terms (Slurm, JAX, Kubernetes, Docker, Ray, Prometheus, TensorBoard, Cursor) appear for the first time in a doc, link them to their official site; later mentions in the same doc remain plain	Repeated linking of the same term within one file (visual clutter)
No links to non-shipping files	Reference only files that exist in the open-source repo	Linking to project-private docs that don't ship (e.g., a sweep-prompt that lives only on the deployer's branch) — these read as broken to anyone who checks out the public branch

File	Lines / fields	Why it stays
`container_env.sh`	`DATASET_DIR="${DATASET_DIR:-/mnt/vast/datasets}"`, `COREDUMP_EXTRA_DIRS=("/perf_apps/maxtext_coredump" …)`	These are deployment-specific fallback defaults that the deployer intentionally keeps for convenience (saves typing on every job). They're documented as overridable via env var (`DATASET_DIR=...`, `COREDUMP_EXTRA_DIRS=...`), and the leakage cost is low — vendor mount-points and a per-cluster path tag, not credentials or topology. The skill must not rewrite these to empty/generic defaults; doing so reduces ergonomics for the deployer with no security gain
`docs/job-submission.md`	the table row showing the same `DATASET_DIR` / `COREDUMP_EXTRA_DIRS` values	Mirrors `container_env.sh`; if the source is allowlisted, the doc must match. Keep
`skills/job-log-triage/SKILL.md`	the example `/perf_apps/maxtext_coredump` reference	Mirrors `container_env.sh`; documenting the allowlisted default
`outputs/**`	everything	Job logs, profiler dumps, and HLO traces will contain hostnames, IPs, and job IDs by construction. The repo's `.gitignore` already excludes most of this, and the sweep recipe below excludes the directory unconditionally. Never run the leak scan against `outputs/` content
`.lock`, `.git/*`	everything	Lockfiles and git metadata legitimately reference branch / tag / SHA strings that match some patterns; do not scan

pre-commit-audit

Informations de source

Options d'installation

Vérifiez les fichiers source

Plus depuis ce dépôt

Plus depuis ce dépôt

Pre-commit audit

Pre-commit audit

Audit posture: holistic, not per-file

When to use this skill — decision tree

The 16-cell matrix

Decomposition: what the 16 cells actually exercise

Two ways the container is reached

Four post-_train.sh flows

Audit procedure

Step 1 — identify the change

Step 2 — map the change to flows

Step 3 — for each affected flow, walk the chain

Step 4 — check the cross-cutting edge cases

Step 5 — smoke-test the changed code paths

5.1 — Launcher-chain smoke (when Steps 1–4 produced affected flows)

5.2 — Ancillary scripts and utilities smoke (when the change touches anything outside Step 2's lookup table)

Step 6 — code quality and design review (propose-first; modify only with explicit go-ahead)

6.1 — Per-change code-smell scan

6.2 — Holistic code-smell trajectory check (walk outward from the diff)

6.3 — Per-change design-choice revisit

6.4 — Architectural fit and boundary check (holistic)

6.5 — Output

Step 7 — docs, comments, and format-consistency check (mandatory before commit)

7.1 — In-file comments in the touched files

7.2 — Cross-references in docs and other skills

Markdown link policy (applies to every md file the change touches)

Slug algorithm (github-slugger)

Audit recipe (Python — handles fenced code blocks, inline code spans, and the slug algorithm correctly)

Symbol and file-path cross-references

7.3 — Self-consistency: does this skill itself need updating?

7.4 — Output

Step 8 — sensitive-info leak check (mandatory before commit)

Step 9 — output the matrix and audit lists

Output format

Notes on this skill itself

Four post-`_train.sh` flows