inference-perf-tune-report

name	inference-perf-tune-report
last_validated	"2026-05-24T00:00:00.000Z"
description	Build a multi-page benchmark report PDF (scatter panels + per-concurrency heatmap tables) from `vllm bench sweep serve_workload` and/or AIPerf outputs, in the GLM-5.1 reference layout (5x2 scatter facet by max_num_batched_tokens + 3x2 heatmap tables for C={8,16,32} x {tok/s/GPU, TTFT}. First-class failed/partial cells). Also renders the cross-engine (vLLM + SGLang) CHAMPION page 8: baseline vs top-X variants + the overlaid roofline + a RECOMMENDED-FOR-PRODUCTION pick via `champion_select`. Backed by the `perf_tune_report` CLI library. Triggers on "build a perf report", "render the atlas PDF", "perf-report campaign", "glm5p1-style report", "pareto report", "perftunereport smoke", "champion selection", "which variant to ship", "production champion", "baseline vs top variants", or any combination of "perf / atlas / report / champion" with "inference / vllm / sglang / aiperf / b200 / gb300 / nvfp4 / fp8".
allowed-tools	["mcp__profile_and_optimize__perf_tune_report_campaign_init","mcp__profile_and_optimize__perf_tune_report_cell_run","mcp__profile_and_optimize__perf_tune_report_import_variant_ab","mcp__profile_and_optimize__perf_tune_report_atlas_aggregate","mcp__profile_and_optimize__perf_tune_report_dcgm_correlate","mcp__profile_and_optimize__perf_tune_report_import_roofline_sweep","mcp__profile_and_optimize__perf_tune_report_champion_select","mcp__profile_and_optimize__perf_tune_report_report_render","mcp__profile_and_optimize__perf_tune_report_report_smoke","mcp__profile_and_optimize__perf_tune_report_publish_to_lake","mcp__profile_and_optimize__search_runbooks","mcp__profile_and_optimize__search_evidence","Bash(perftunereport:*)","Bash(kubectl:*)","Bash(vllm:*)","Bash(open:*)","Read","Write"]

inference-perf-tune-report

Purpose

Build a multi-page benchmark report PDF -- scatter panels + per-concurrency heatmap tables -- from inference benchmark sweep output. Mirrors the GLM-5.1 reference layout:

Steady-state window: the report only renders what the sweep measured. A throughput sweep cell needs num_prompts >= 2*c or its high-c throughput is undercounted ~1.6-1.8x (ramp/drain-dominated window, docs/METHODOLOGY.md trap 4). The import_roofline_sweep importer flags steady_window_undercount + WARNs on any cell < 2c - heed it before publishing the report.

• Page 1: 5 rows (one per max_num_batched_tokens in {1024, 2048, 4096, 8192, 16384}) x 2 columns of scatter panes -- left=TTFT vs request throughput, right=tok/s/user vs tok/s/GPU. Inline-labels selected profiling concurrencies (default {1, 8, 32}). Color encodes (hardware, TP). Circle marker = EP, X marker = TP. Hollow markers indicate MTP.
• Page 2: 3 rows (C in {8, 16, 32}) x 2 columns of heatmap tables showing output tok/s/GPU and TTFT avg (ms) for every (variant, mbt) combo, with gray cells for missing / failed / partial measurements.

The renderer is workload-agnostic. It ingests any AtlasCell-shaped JSONL. Backends (vllm-sweep + AIPerf) normalize into the same schema.

When to use

• Comparing inference performance across multiple hardware generations (e.g. H100 FP8 vs B200 NVFP4 vs GB300 NVFP4) for the same model.
• Comparing parallelism strategies (TP vs EP) and speculative-decoding variants (MTP on / off) on a single deployment.
• Re-rendering a stakeholder-grade report from previously captured benchmark data without re-running the cluster sweep.
• Producing a release-day PDF for a new model variant before promoting it to staging / prod.

Do not use this skill for:

• Single-deployment, single-concurrency snapshot benchmarks -- the upstream perf-bench skill is the right tool. Use this skill once you have multiple atlas cells to compare side-by-side.
• Training-benchmark reports (step-time / MFU) -- out of scope for this plugin.
• Quality / accuracy evaluation -- use inference-model-eval.

Example prompts