AMD-AGI

Launches and monitors Hyperloom's multi-agent inference optimizer for LLM serving on AMD GPUs. Use when the user asks to optimize an inference model, run Magpie benchmarks/profiles, resume an inference_optimizer session, tune SGLang/vLLM serving parameters,…

Build a single self-contained `session_breakdown.json` capturing every fact a dashboard needs about one hyperloom optimization session. Use when the user mentions session-breakdown, kernel attribution, a stats/reporting service, capability summary, or wants…

Independent guardian daemon for Hyperloom inference optimization. Implements the inference_optimizer "robustness" reactor so the Coordinator can call it as a Backend, plus a standalone loop for dev. Owns continuous health monitoring, RCA, and…

Run an advanced configurable Hyperloom optimization session with explicit model, framework, workload, objective, and phase toggles. Use when the user wants more control than the fixed 3h or 12h demo presets.

Run a 3-hour Hyperloom Qwen3-8B optimization session without the Kernel Agent. Use when the user wants a short, no-kernel Hyperloom demo on the local AMD ROCm environment.

Critic layer for the inference optimizer. Use when Conductor asks for a Critic Review verdict on Orchestration or Kernel proposals, conversation-driven decision review, KB recall/ingest guidance, cross-run synthesis, or Devil's advocate review signals.

Run a 12-hour Hyperloom Qwen3-14B-FP8 optimization session. Use when the user wants a medium-length Hyperloom demo on the local AMD ROCm environment.

Run a 4-hour multi-node Hyperloom Qwen3-30B-A3B optimization (Infera PD-disaggregated or RayJob aggregated) with --nodes 2 and sglang MoE tuning on MI325X. Hand this skill to the agent to launch and monitor the run.

This skill should be used when optimizing AMD GPU kernels on MI300 using the aiter project, including running op tests, benchmarking, iterating on kernel changes, and recording results in the kernel experiment database.

This skill should be used when reasoning about GPU architecture fundamentals to guide kernel optimization choices such as memory hierarchy usage, execution model mapping, block sizing, and latency-aware tuning across HIP, Triton, and PyTorch.

This skill should be used when writing or tuning HIP kernels on AMD/NVIDIA GPUs, covering memory coalescing, shared-memory tiling, bank conflict avoidance, warp primitives, occupancy, vectorization, async ops, loop unrolling, and profiling.

This skill should be used when optimizing kernels in this repo and needing to consult past optimization experiments, or when recording the current optimization iteration back into the kernel experiment database.

MI300/CDNA3 architecture guide for HIP/Triton optimization—MFMA variants, dual register files, data formats, sparsity, LDS/GWS, and best practices.

CDNA3/MI300 HIP programming insights—chiplet/cache model, Infinity Cache, memory coherency, matrix cores, sparsity, and best practices.

MI300 HIP programming differences vs NVIDIA—wavefront vs warp, memory hierarchy, MFMA usage, occupancy, and profiling pitfalls.

This skill should be used when optimizing PyTorch models and kernels, including efficient tensor operations, torch.compile, custom autograd/CUDA/Triton extensions, mixed precision, memory and data pipeline tuning, model optimization techniques, CUDA graphs,…

Get a container ready to tune GPU ops on AMD Instinct — audit which tuning tools are present, install the missing bench clients (hipblaslt-bench, ckProfiler, aiter tuners) at versions that match the runtime, and avoid the version-skew failures that silently…

Tune GEMM and fused ops through aiter on AMD Instinct — run the gradlib tuner across hipblaslt/triton/flydsl/asm/torch backends, write tuned config CSVs to the directory the runtime actually reads, and prove engagement with AITER_LOG_TUNED_CONFIG. Use when…

Tune GEMMs on the Composable Kernel path — race CK's compiled instances with ckProfiler, read the instance name to understand why one won, and route the result into a framework through aiter's per-op CK tuners. Use when the op dispatches through CK or when…

The universal loop for tuning GEMM and other GPU ops on AMD Instinct — scope the search space, measure honestly, gate on correctness, and prove the tuned artifact actually engaged. Start here before any language- or framework-specific tuning skill.

Tune FlyDSL kernels on AMD Instinct — build and prune a Config space, set the autotune key, and race FlyDSL against the other backends through aiter's libtype mechanism. Use when the kernel is written in FlyDSL or when deciding whether the FlyDSL path is…

Tune hand-written HIP kernels on AMD Instinct — choose launch geometry against the wave64/CU/LDS limits of the target, use rocprofv3 to get ground-truth kernel timings and occupancy, and confirm which kernel actually ran. Also the profiler reference for…

Tune GEMMs on the hipBLASLt path — capture the exact problem from a live workload, race all solutions with hipblaslt-bench, replay the winner by solution index, and avoid the bracket-number and hot-cache traps. Use when the op dispatches through hipBLASLt…

Tune ops inside an SGLang serving deployment on AMD Instinct — capture real shapes, tune the Triton fused-MoE path into the correct triton_<version> config directory, deploy via SGLANG_MOE_CONFIG_DIR, and read the server log to distinguish an exact config hit…

Create GPU config files to support existing MaxText model definitions on AMD GPU clusters. Use when the user wants to add a model, create a config, support a new model, or asks about model configs, parallelism, batch size, OOM, quantization, or .gpu.yml files.

Comprehensive pre-commit verification checklist with five independent responsibilities. (1) Launcher path coverage - verify a change to any launcher-chain file preserves correct behavior across all 16 combinations of entry point × launch mode × stack (Steps…

Direct per-kernel time analysis from JAX / TensorFlow xplane traces via `utils/profile_drill.py`. Use when the user asks for a per-kernel breakdown, step-time composition, cross-variant kernel comparison, main-stream-blocking analysis, or any question that…

Four sweep operations: (1) Model perf sweep — find optimal batch size / TGS for a model. Use for: sweep batch size, tune TGS, benchmark throughput, find optimal config. (2) Node perf sweep — compare per-node GPU performance to find outliers. Use for: check…

Find the XLA flag / NCCL env-var combination that maximizes steady-state TGS for one (model × parallelism) cell. Produces an evidence-backed leaderboard, mechanistic explanation of the winning flag, and a deployment recipe. Use when the user asks to tune XLA…

Triage MaxText training jobs from log files — failed, hanging, running, or completed. Use when the user asks why a job failed, wants to diagnose an error, sees a crash, hang, timeout, OOM, NCCL error, heartbeat timeout, wants to understand a job's status, or…

Diagnose training job incidents and check cluster health using the per-job Prometheus TSDB. Use when the user asks to diagnose a failure root cause, check GPU/network health, query Prometheus metrics, investigate a hang, or when the triage skill recommends…

Analyze MaxText training job performance using tgs_tagger, TraceLens, and IRLens. Use when the user asks to analyze a training run, profile traces, HLO IR, TGS metrics, GPU utilization, or mentions tag_tgs, TraceLens, IRLens, xplane, or performance analysis.

Create a pull request from current branch. Use when: create pr, make pr, open pull request, submit pr, pull request from branch.

Generate release notes from merged PRs between releases. Use when: release notes, create release notes, changelog, what changed, PR summary, release summary, diff between versions, categorize changes.

Create a GitHub release with tag and notes. Use when: create release, tag release, publish release, draft release, new version, cut release.

Diagnose a GitHub Actions workflow run. Use when: check run, diagnose run, analyze run, what failed, run status, debug workflow, why did it fail, check build.

Get benchmark results from a workflow run. Use when: benchmark results, get benchmarks, latency numbers, run results, performance results, compare benchmarks.

Maintainer-only release creation and build dispatch. Use when: orchestrate release, release version, full release, end to end release, release pipeline, release v, cut and build release.

Shared prerequisite checks for repository skills. Used by other skills before running GitHub CLI workflows.

Maintainer-only Build and Benchmark xDiT workflow dispatch. Use when: release build, trigger build, run benchmarks, dispatch workflow, start CI, benchmark image, MIOpen tuning, trigger pipeline, build-and-benchmark.

Inventory and explain the patch (monkey-patch) optimizations Primus layers over upstream training backends such as Megatron-LM, TorchTitan, and MaxText, including their version compatibility, dependencies, and Primus-Turbo integration details, by reading the…

Validate SLURM cluster nodes by running actual training jobs in groups. Use when the user wants to test which idle nodes can successfully run training, verify node health through real workloads, or identify broken nodes in a SLURM cluster.

Compare a Primus backend against an upstream repository or reference, verify git state, dependencies, directory changes, and integration coupling, then generate comparison reports, dashboard metadata, and a deployable dashboard index. Also owns the shared…

Generate a validation plan and test scaffolding to check a Primus optimization that has been ported into a user's own training framework, across correctness (numerical accuracy vs a reference), performance (throughput parity), and integration (no…

Inspect the current Spur (AMD SLURM-compatible) cluster node-allocation state and produce a Markdown report covering the caller's account/QoS permissions, partitions and per-state node counts, per-QoS and per-account node usage, reservations, queue pressure,…

Opinionated guide for using Primus Projection to choose parallelism (TP/PP/EP/CP/DP) and pipeline schedules, validate memory fit on target nodes, reason about communication collectives, and explore optimization trade-offs with minimal compute. Use when the…

Check available idle nodes in a SLURM cluster. Use when the user wants to find usable idle nodes, verify node health, check docker status on SLURM nodes, check NIC QoS/DCQCN configuration, check RDMA link status, validate GID table, or troubleshoot cluster…

Primus-Turbo handoff to the autonomous kernel-optimize loop — collect the prerequisites (kernel path, focused test/bench commands, scoring metric, execution mode, quick-validation harness) a kernel campaign needs and pass them on. Use when pushing a…

Develop, debug, and validate Primus-Turbo operators and modules on AMD GPUs. Covers the layered architecture (ops / kernels-dispatcher / Triton / HIP-CK csrc / modules), how to add or change a feature end-to-end, accuracy verification (SNR, tolerances,…

AI-driven operator performance optimization framework. Defines the optimization loop, execution environment selection, knowledge routing, and logging conventions to drive agent-autonomous iteration toward hardware limits.

Primus-Turbo feature development workflow — the layered architecture (ops / kernels-dispatcher / Triton / HIP-CK csrc / modules), how to wire a new operator end-to-end, which layer to touch, and which existing file to copy. Use when adding or changing a…

Primus-Turbo performance verification — run single-operator and suite benchmarks, read the latency/TFLOPS metrics, source real-model shapes, and derive a combined training-step metric. Use when measuring latency or throughput of a Primus-Turbo operator.

Primus-Turbo accuracy verification — compare an operator against a higher-precision reference for forward and backward, with the right gate (allclose for bf16/fp16/fp32, SNR for fp8/fp4) and FP8 encoding awareness. Use when validating numerical correctness of…

ROCm profiling workflow for AMD GPU kernels using rocprofv3 and rocprof-compute. Use when profiling hot kernels, collecting counters, diagnosing memory-vs-compute-vs-stall bottlenecks, reading Perfetto traces, or validating low-precision AMD kernels.

Benchmarks LLM inference and drives GPU kernel optimization with Magpie. Use when the user wants to benchmark vLLM, SGLang, or Atom; capture torch traces; post-process inference traces with TraceLens into prefill/decode and roofline reports; identify top…

Find kernel source code, test files, and test commands for AMD GPU kernels identified in profiler traces. Use when the user wants to locate kernel implementations, find tests for specific kernels, or enrich gap_analysis results with source information.…

Orchestrates modular PyTorch profiler trace analysis with TraceLens: generates perf reports, prepares category data, runs system-level and compute-kernel subagents in parallel, validates outputs, and writes a prioritized stakeholder report (analysis.md). Use…