メニュー
Primus-Turbo performance verification — run single-operator and suite benchmarks, read the latency/TFLOPS metrics, source real-model shapes, and derive a combined training-step metric. Use when measuring latency or throughput of a Primus-Turbo operator.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | verify-performance |
| description | Primus-Turbo performance verification — run single-operator and suite benchmarks, read the latency/TFLOPS metrics, source real-model shapes, and derive a combined training-step metric. Use when measuring latency or throughput of a Primus-Turbo operator. |
Measure latency and throughput. Benchmarks live in benchmark/ops/training/; shared shapes and helpers in benchmark/ops/training/config.py. Read the hub ../SKILL.md first.
cd benchmark/ops/training
python bench_gemm_turbo.py --dtype bf16
python bench_gemm_turbo.py --dtype fp8 --granularity blockwise # tensorwise|rowwise|blockwise|mxfp8
python bench_gemm_turbo.py --dtype bf16 -o result.csv
# Force a backend to compare (env beats autotune beats default)
PRIMUS_TURBO_GEMM_BACKEND=CK python bench_gemm_turbo.py --dtype fp8 --granularity blockwise
PRIMUS_TURBO_AUTO_TUNE=1 python bench_gemm_turbo.py --dtype fp8 --granularity blockwise
Other ops follow the same bench_<op>_turbo.py naming (bench_attention_turbo.py, bench_grouped_gemm_turbo.py), with *_torch.py / *_te.py baselines and bench_deepep_intranode.py, bench_symmetric_memory.py.
Each bench_*_turbo.py runs the correctness check first, then profiles with torch.utils.benchmark.Timer (20 warmup + 100 timed iters) and writes a CSV row:
| Column | Meaning |
|---|---|
Check | correctness gate — PASS / FAIL / ERROR (FAIL invalidates the timing) |
Forward Time (ms) / Forward TFLOPS | 2*M*N*K / time / 1e12 |
Backward Time (ms) / Backward TFLOPS | 2 * forward FLOPs / time / 1e12 |
The in-bench check uses SNR for fp8/fp4 (check_gemm_correctness_by_snr; thresholds 25/20/10 — canonical gate doc is ../verify-accuracy/SKILL.md) and allclose for bf16. A FAIL row means the number is not trustworthy — fix correctness first.
benchmark/ops/training/config.py derives shapes from real models, so numbers reflect training workloads:
MBS ∈ {1,2,4} (Llama-2, Llama-3.1, Qwen2.5, Mistral).MoEModelConfigs (DeepSeek-V3/V2, Qwen3-MoE, Mixtral, Kimi-K2, ...) with gen_grouped_gemm_group_lens(b, m, balance=...) (balance=True → uniform, balance=False → skewed) for both token distributions.gen_attention_test_cases() (dedup over dense + MoE/MLA heads).Add shapes for a new op by extending config.py, not by hardcoding in the bench script.
python benchmark/ops/training/run_suite.py -d output/ # all tasks (benchmark_suite.yaml)
python benchmark/ops/training/run_suite.py -d output/ -g gemm_fp8 # one group
python benchmark/ops/training/run_suite.py -d output/ -n 4 # 4 GPUs
Aggregate with benchmark/ops/training/summarize_results.py.
For ops optimized as a forward+backward pair, compare the combined step, not each direction alone:
Combined Step Time (ms) = Forward Time (ms) + Backward Time (ms)
Combined Step TFLOPS = 6 * M * N * K / (Combined Step Time * 1e-3) / 1e12
Use the geometric mean of Combined Step TFLOPS across shapes as the single comparison score; keep forward/backward TFLOPS as diagnostics.
For module/model-level impact (a real training step), use benchmark/pretrain/pytorch/ (entry pretrain_main.py, model defs in models/turbo_llama.py) — confirm a microbench gain transfers to training.
Spot-checking perf is this file. Driving a kernel toward the hardware limit (profiling-guided, accept/rollback rounds) is the optimization loop — hand off via ../optimize-handoff/SKILL.md. Runtime profiling evidence: ../run_profile/tool-rocprof/SKILL.md.
- [ ] Check=PASS before trusting any timing
- [ ] Shapes from config.py (real model shapes), incl. skew for MoE/grouped
- [ ] Backends compared via PRIMUS_TURBO_*_BACKEND when relevant
- [ ] Combined-step geomean used for fwd+bwd comparison
- [ ] CSV saved (-o / suite output dir) for before/after comparison
Primus-Turbo handoff to the autonomous kernel-optimize loop — collect the prerequisites (kernel path, focused test/bench commands, scoring metric, execution mode, quick-validation harness) a kernel campaign needs and pass them on. Use when pushing a Primus-Turbo kernel toward the hardware limit, not just spot-checking perf.
Develop, debug, and validate Primus-Turbo operators and modules on AMD GPUs. Covers the layered architecture (ops / kernels-dispatcher / Triton / HIP-CK csrc / modules), how to add or change a feature end-to-end, accuracy verification (SNR, tolerances, reference implementations), performance benchmarking, the backend dispatch system, and build/test/bench commands. Use for any Primus-Turbo development task (GEMM, Attention, GroupedGEMM, MoE, quantization, normalization, activation) and for accuracy or performance validation.
AI-driven operator performance optimization framework. Defines the optimization loop, execution environment selection, knowledge routing, and logging conventions to drive agent-autonomous iteration toward hardware limits.
Primus-Turbo feature development workflow — the layered architecture (ops / kernels-dispatcher / Triton / HIP-CK csrc / modules), how to wire a new operator end-to-end, which layer to touch, and which existing file to copy. Use when adding or changing a Primus-Turbo operator or module on AMD GPUs.
Primus-Turbo accuracy verification — compare an operator against a higher-precision reference for forward and backward, with the right gate (allclose for bf16/fp16/fp32, SNR for fp8/fp4) and FP8 encoding awareness. Use when validating numerical correctness of a Primus-Turbo operator.
ROCm profiling workflow for AMD GPU kernels using rocprofv3 and rocprof-compute. Use when profiling hot kernels, collecting counters, diagnosing memory-vs-compute-vs-stall bottlenecks, reading Perfetto traces, or validating low-precision AMD kernels.