Loading the repository README…
Showing 33 of 33 collected skills.
Reproduce the Artificial Analysis (AA) language-model performance workload shapes against an OpenAI-compatible chat endpoint using NVIDIA AIPerf. Drives the three AA text shapes (1k input / >=1k answer, 10k / >=1.5k, 100k / >=2k) with temperature 0, top_p 1,…
Correlate DCGM Prometheus byte-traffic counters with an inference-perf-bench sweep window to compute byte-grounded workload-level Speed-of-Light. The third tier of the SoL rigor hierarchy (after zymtrace sample-share and ncu per-kernel arithmetic intensity).…
Measure the low-concurrency (c=1..c=8) decode hot-path of a live vLLM pod FAST and CORRECTLY: where each token's time actually goes (GPU-busy vs host-idle gap vs comm), whether the workload is kernel-bound / host-bound / comm-bound, and what the addressable…
Diff the compiled FX / Inductor graphs across two vLLM versions or two helm configs to see exactly which fused kernels / passes / partitions changed. Uses `torch._dynamo.explain` + `torch.compile`'s graph-dump hooks. Useful when you anticipate a…
Profile live inference traffic into a token/shape distribution artifact that drives profile-matched speculative-decoding draft training -- an analog of Fireworks FireOptimizer's "profile-driven customization" (the documented source of its higher draft…
Read-only coverage auditor for the mirage / MPK persistent-megakernel's GENERATED task graph: cross-references DECLARED tensors (all_tensors[...] in kernel_N.cu) against tensors CONSUMED by tasks (inputs/outputs base_ptr in task_graph_N.json) and flags any…
Investigate a GPU or CPU workload through the zymtrace MCP. The MCP does most of the analysis. This skill enforces the cross-view discipline -- always pull the matching opposite-side flamegraph (CPU for GPU workloads, GPU for CPU workloads) with the same…
Scaffold a new evidence bundle directory ready for reproducibility-grade evidence capture: SOURCE.md (operator + cluster + git SHA + UTC timestamp), summary.md (verdict skeleton), commands/ (for the four-file .cmd/.stdout/.stderr/.exit tuple capture per shell…
SLA-first GPU capacity sizing for a serving deployment: given a tokens-per-minute (TPM) target AND the interactivity SLA (output tokens/s/user), compute the pods and GPUs needed from a model's measured tok/s/user-vs-concurrency curve. Sizing MUST start from…
Render cross-model fleet leaderboards from the local perf-report campaigns in ONE command (`perftunereport fleet_leaderboard`): a latency tier (aa-1k/10k/100k tok/s/user + TTFT + cost at c=1/c=10), a throughput tier (peak tok/s/GPU per (model,quant,TP) with…
Capture per-kernel CUDA hardware-counter data (occupancy, achieved warps active, regs/thread, smem/block, DRAM throughput, arithmetic intensity, branch divergence %, warp-stall reason %) from a live vLLM inference pod via an `ncu` debug sidecar (no production…
Capture per-kernel CUDA profile data from a live vLLM inference pod via an nsys debug sidecar (no production image rebuild). Outputs `.nsys-rep` + summary CSV + top-kernels table that joins with the zymtrace per-kernel breakdown and the inference-perf-bench…
White-box debug a custom CUDA/CUTLASS kernel producing a WRONG numeric result (over-amplification, NaN, coherence) after black-box bisection is EXHAUSTED: every external lever varied (grid, pipeline depth, layout, dtype), the result deterministic +…
Capture + enforce per-model KNOWN-GOOD serving configs: the REQUIRED serve flags (boot-blockers / crash-at-high-c / deploy-correctness workarounds) plus the champion + the bug each flag avoids, in one queryable registry…
Drive lm-eval-harness quality evals (GPQA, MMLU-Pro) inside model pods plus optional ExternalEval (externally operated). Use to validate model quality before promoting to staging or prod, after vLLM / quantization / KV-cache changes, or to compare against…
Coordinate an evidence-backed inference model optimization run from a user-supplied deployment or campaign configuration. Establishes readiness and a production-shaped baseline, profiles measured bottlenecks, runs a bounded tune sweep, checks quality, selects…
Bridge between the inference-perf-bench / ai-bench output bundle and the profile-and-optimize perf-baseline registry. Knows the canonical inference perf metric shape (TTFT p50/p95/p99, ITL p50, throughput, tok/s/user, request latency p50, prefix cache hit…
Canonical inference perf-bench skill (formal name. The colloquial alias is `ai-bench` - identical behaviour). Drives NVIDIA AIPerf + the replay-playback dataset against an in-cluster vLLM endpoint to measure TTFT, ITL, throughput, tok/s/user, request latency,…
Fuse the four profilers -- nsys (timeline + kernel durations), ncu (roofline / occupancy / arithmetic intensity), zymtrace (CPU+GPU flamegraph), and DCGM (byte-grounded %SoL) -- into ONE ranked, data-backed recommendation ledger for a vLLM deployment. Every…
Build a multi-page benchmark report PDF (scatter panels + per-concurrency heatmap tables) from `vllm bench sweep serve_workload` and/or AIPerf outputs, in the GLM-5.1 reference layout (5x2 scatter facet by max_num_batched_tokens + 3x2 heatmap tables for…
Applies Jeff Dean and Sanjay Ghemawat's Performance Hints to GPU inference. Builds a back-of-the-envelope cost ledger, identifies the hot-path class, bounds each optimization with profile share, and routes the highest-value hypothesis to the repo's benchmark,…
Produce quantized inference weights from a BF16/FP8 base checkpoint via a post-training-quantization (PTQ) pipeline -- instead of only ever pulling NVFP4 weights pre-quantized. A calibration prep Job then a quantize Job: NVIDIA ModelOpt -> NVFP4 (default), or…
Coordinate a measured speculative decoding workflow for a served model. Profile representative traffic, build a matched corpus, run an operator-provided draft trainer, compare the draft with the standing config, and promote only a measured win. Use for…
Train + validate a speculative-decoding draft head (EAGLE3 or DFlash) for an ARBITRARY target LLM, generalizing the GLM-5.1-only SpecForge recipe so any new model gets one. Parameterizes what that offline recipe hard-codes: target path, aux-hidden-states…
Run a bounded comparison of speculative decoding draft training settings. Use measured serving acceptance and latency as the decision metrics, with training accuracy only as a triage signal. Use when one draft recipe already trains correctly and the operator…
Search a bounded inference serving configuration matrix, compare every cell with a named baseline, and select a throughput or latency champion. Uses the checked-in campaign, evidence, baseline, aggregation, and report tools. Use when asked to tune batching,…
Render the leadership value-prop ledger for the inference effort: the deployable wins vs the FlashInfer-TRTLLM + best-tuned-vLLM 0.21/0.22 baseline, grouped DONE / IN-PROGRESS / NOT-DONE / CLOSED-NEGATIVE, each row data-backed by a perf-lake campaign (live…
Expert Kubernetes troubleshooting assistant for diagnosing and resolving issues across the full stack - pods, control plane, nodes, networking, storage, and underlay infrastructure - in GPU cloud environments. Triggers on any report of a broken, degraded, or…
Diff a current performance measurement against a registered baseline from the perf-baselines registry. Works for any measurement type (NCCL BW, MFU, step-time, latency, throughput, per-kernel times, structured heatmaps, nsys-rep profiles). Honors the…
Capture any performance measurement (NCCL bandwidth, MFU, step-time, latency, throughput, kernel-time, anything operator-defined) into a versioned baseline registry under experiments/artifacts/perf-baselines/. Workload-agnostic. Records full provenance…
Reusable wrapper for the knowledge-base-first PromQL pattern. Operator names the metric / question / time range. Skill calls query_observability_knowledge_base first to confirm labels + cardinality, derives the safe narrow PromQL, runs it via the Prometheus…
Reusable wrapper for the knowledge-base-first SQL pattern, adapted to the zymtrace ClickHouse profiling backend. Operator names the metric / question / time range. Skill anchors the `zymtrace_profiling.events` schema first (DESCRIBE + label-value +…
Copy this directory to plugins/profile-and-optimize/skills/<your-skill-name>/ and rewrite this description to be third-person + include WHAT the skill does and WHEN (specific trigger phrases). Max 1024 chars. Do NOT leave the word "template" in the real…
Copy this prompt into the AI assistant you're using.
Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/cfregly/gpu-perf-tune.The repository README could not be loaded. You can still explore its Skills below.