القائمة
جمع SkillsMP عدد ٣٣ من skills من cfregly/gpu-perf-tune. افتح أي skill لمراجعة مصدره وتفاصيله.
عرض ٣٣ من أصل ٣٣ skills مجمعة.
SLA-first GPU capacity sizing for a serving deployment: given a tokens-per-minute (TPM) target AND the interactivity SLA (output tokens/s/user), compute the pods and GPUs needed from a model's measured tok/s/user-vs-concurrency curve. Sizing MUST start from…
لغة النص الأصلي: الإنجليزية
Measure the low-concurrency (c=1..c=8) decode hot-path of a live vLLM pod FAST and CORRECTLY: where each token's time actually goes (GPU-busy vs host-idle gap vs comm), whether the workload is kernel-bound / host-bound / comm-bound, and what the addressable…
لغة النص الأصلي: الإنجليزية
Capture per-kernel CUDA hardware-counter data (occupancy, achieved warps active, regs/thread, smem/block, DRAM throughput, arithmetic intensity, branch divergence %, warp-stall reason %) from a live vLLM inference pod via an `ncu` debug sidecar (no production…
لغة النص الأصلي: الإنجليزية
Capture per-kernel CUDA profile data from a live vLLM inference pod via an nsys debug sidecar (no production image rebuild). Outputs `.nsys-rep` + summary CSV + top-kernels table that joins with the zymtrace per-kernel breakdown and the inference-perf-bench…
لغة النص الأصلي: الإنجليزية
End-to-end orchestrator that takes a NEW model from a bare HuggingFace id to a perf-lake-published, validated CROSS-ENGINE (vLLM + SGLang) champion on B200/GB300. Scaffolds a per-model harness + evidence bundle (run-id == experiment-id), then drives…
لغة النص الأصلي: الإنجليزية
Canonical inference perf-bench skill (formal name. The colloquial alias is `ai-bench` - identical behaviour). Drives NVIDIA AIPerf + the replay-playback dataset against an in-cluster vLLM endpoint to measure TTFT, ITL, throughput, tok/s/user, request latency,…
لغة النص الأصلي: الإنجليزية
Applies Jeff Dean and Sanjay Ghemawat's Performance Hints to GPU inference. Builds a back-of-the-envelope cost ledger, identifies the hot-path class, bounds each optimization with profile share, and routes the highest-value hypothesis to the repo's benchmark,…
لغة النص الأصلي: الإنجليزية
Search a vLLM serving model's config space for its best inference performance: a `perftunereport campaign_run` matrix sweep over concurrency x max_num_batched_tokens x kv-cache-dtype x cudagraph_mode, plus a same-node controlled A/B (`run-controlled-ab.sh`)…
لغة النص الأصلي: الإنجليزية
Copy this directory to plugins/profile-and-optimize/skills/<your-skill-name>/ and rewrite this description to be third-person + include WHAT the skill does and WHEN (specific trigger phrases). Max 1024 chars. Do NOT leave the word "template" in the real…
لغة النص الأصلي: الإنجليزية
Scaffold a new evidence bundle directory ready for reproducibility-grade evidence capture: SOURCE.md (operator + cluster + git SHA + UTC timestamp), summary.md (verdict skeleton), commands/ (for the four-file .cmd/.stdout/.stderr/.exit tuple capture per shell…
لغة النص الأصلي: الإنجليزية
Bridge between the inference-perf-bench / ai-bench output bundle and the profile-and-optimize perf-baseline registry. Knows the canonical inference perf metric shape (TTFT p50/p95/p99, ITL p50, throughput, tok/s/user, request latency p50, prefix cache hit…
لغة النص الأصلي: الإنجليزية
Fuse the four profilers -- nsys (timeline + kernel durations), ncu (roofline / occupancy / arithmetic intensity), zymtrace (CPU+GPU flamegraph), and DCGM (byte-grounded %SoL) -- into ONE ranked, data-backed recommendation ledger for a vLLM deployment. Every…
لغة النص الأصلي: الإنجليزية
Build a multi-page benchmark report PDF (scatter panels + per-concurrency heatmap tables) from `vllm bench sweep serve_workload` and/or AIPerf outputs, in the GLM-5.1 reference layout (5x2 scatter facet by max_num_batched_tokens + 3x2 heatmap tables for…
لغة النص الأصلي: الإنجليزية
Produce quantized inference weights from a BF16/FP8 base checkpoint via a post-training-quantization (PTQ) pipeline -- instead of only ever pulling NVFP4 weights pre-quantized. A calibration prep Job then a quantize Job: NVIDIA ModelOpt -> NVFP4 (default), or…
لغة النص الأصلي: الإنجليزية
Diff a current performance measurement against a registered baseline from the perf-baselines registry. Works for any measurement type (NCCL BW, MFU, step-time, latency, throughput, per-kernel times, structured heatmaps, nsys-rep profiles). Honors the…
لغة النص الأصلي: الإنجليزية
Capture any performance measurement (NCCL bandwidth, MFU, step-time, latency, throughput, kernel-time, anything operator-defined) into a versioned baseline registry under experiments/artifacts/perf-baselines/. Workload-agnostic. Records full provenance…
لغة النص الأصلي: الإنجليزية
Reusable wrapper for the knowledge-base-first SQL pattern, adapted to the zymtrace ClickHouse profiling backend. Operator names the metric / question / time range. Skill anchors the `zymtrace_profiling.events` schema first (DESCRIBE + label-value +…
لغة النص الأصلي: الإنجليزية
Closed-loop speculative-decoding-as-a-service orchestrator -- a self-hosted analog of Fireworks FireOptimizer's adaptive speculative execution. Composes the per-phase skills into one profile-matched loop: profile live traffic (inference-workload-profile) ->…
لغة النص الأصلي: الإنجليزية
Investigate a GPU or CPU workload through the zymtrace MCP. The MCP does most of the analysis. This skill enforces the cross-view discipline -- always pull the matching opposite-side flamegraph (CPU for GPU workloads, GPU for CPU workloads) with the same…
لغة النص الأصلي: الإنجليزية
Reproduce the Artificial Analysis (AA) language-model performance workload shapes against an OpenAI-compatible chat endpoint using NVIDIA AIPerf. Drives the three AA text shapes (1k input / >=1k answer, 10k / >=1.5k, 100k / >=2k) with temperature 0, top_p 1,…
لغة النص الأصلي: الإنجليزية
Correlate DCGM Prometheus byte-traffic counters with an inference-perf-bench sweep window to compute byte-grounded workload-level Speed-of-Light. The third tier of the SoL rigor hierarchy (after zymtrace sample-share and ncu per-kernel arithmetic intensity).…
لغة النص الأصلي: الإنجليزية
Render cross-model fleet leaderboards from the local perf-report campaigns in ONE command (`perftunereport fleet_leaderboard`): a latency tier (aa-1k/10k/100k tok/s/user + TTFT + cost at c=1/c=10), a throughput tier (peak tok/s/GPU per (model,quant,TP) with…
لغة النص الأصلي: الإنجليزية
Diff the compiled FX / Inductor graphs across two vLLM versions or two helm configs to see exactly which fused kernels / passes / partitions changed. Uses `torch._dynamo.explain` + `torch.compile`'s graph-dump hooks. Useful when you anticipate a…
لغة النص الأصلي: الإنجليزية
White-box debug a custom CUDA/CUTLASS kernel producing a WRONG numeric result (over-amplification, NaN, coherence) after black-box bisection is EXHAUSTED: every external lever varied (grid, pipeline depth, layout, dtype), the result deterministic +…
لغة النص الأصلي: الإنجليزية
Capture + enforce per-model KNOWN-GOOD serving configs: the REQUIRED serve flags (boot-blockers / crash-at-high-c / deploy-correctness workarounds) plus the champion + the bug each flag avoids, in one queryable registry…
لغة النص الأصلي: الإنجليزية
Drive lm-eval-harness quality evals (GPQA, MMLU-Pro) inside model pods plus optional ExternalEval (externally operated). Use to validate model quality before promoting to staging or prod, after vLLM / quantization / KV-cache changes, or to compare against…
لغة النص الأصلي: الإنجليزية
Train + validate a speculative-decoding draft head (EAGLE3 or DFlash) for an ARBITRARY target LLM, generalizing the GLM-5.1-only SpecForge recipe so any new model gets one. Parameterizes what that offline recipe hard-codes: target path, aux-hidden-states…
لغة النص الأصلي: الإنجليزية
Tune a speculative-decoding DRAFT head's TRAINING hyperparameters (global batch, learning rate, accumulation, warmup) for a target LLM, optimizing the TRUE serving in-engine acceptance length (vLLM spec_decode counters) with a cheap training-acc proxy for…
لغة النص الأصلي: الإنجليزية
Render the leadership value-prop ledger for the inference effort: the deployable wins vs the FlashInfer-TRTLLM + best-tuned-vLLM 0.21/0.22 baseline, grouped DONE / IN-PROGRESS / NOT-DONE / CLOSED-NEGATIVE, each row data-backed by a perf-lake campaign (live…
لغة النص الأصلي: الإنجليزية
Profile live inference traffic into a token/shape distribution artifact that drives profile-matched speculative-decoding draft training -- an analog of Fireworks FireOptimizer's "profile-driven customization" (the documented source of its higher draft…
لغة النص الأصلي: الإنجليزية
Expert Kubernetes troubleshooting assistant for diagnosing and resolving issues across the full stack - pods, control plane, nodes, networking, storage, and underlay infrastructure - in GPU cloud environments. Triggers on any report of a broken, degraded, or…
لغة النص الأصلي: الإنجليزية
Read-only coverage auditor for the mirage / MPK persistent-megakernel's GENERATED task graph: cross-references DECLARED tensors (all_tensors[...] in kernel_N.cu) against tensors CONSUMED by tasks (inputs/outputs base_ptr in task_graph_N.json) and flags any…
لغة النص الأصلي: الإنجليزية
Reusable wrapper for the knowledge-base-first PromQL pattern. Operator names the metric / question / time range. Skill calls query_observability_knowledge_base first to confirm labels + cardinality, derives the safe narrow PromQL, runs it via the Prometheus…
لغة النص الأصلي: الإنجليزية