inference-kernel-whitebox-debug

White-box debug a custom CUDA/CUTLASS kernel producing a WRONG numeric result (over-amplification, NaN, coherence) after black-box bisection is EXHAUSTED: every external lever varied (grid, pipeline depth, layout, dtype), the result deterministic + lever-independent, yet the structure reads correct. Two tracks: (A) an in-kernel operand + accumulator trace (`if constexpr`-gated to survive megakernel pipelining), (B) a standalone reproducer .cu instantiating the EXACT kernel template + TMA descriptors with controlled inputs (all-ones then real-dump) vs a host GEMM. Localizes the defect to operand-load vs MMA vs epilogue vs descriptor. Escalation tier above `inference-kernel-ncu-profile` when H/P are fine but the output is numerically wrong. Triggers on "white-box kernel", "kernel over-amplifies / NaN", "standalone reproducer", "in-kernel trace", "operand dump", "cutlass UMMA bug", or any combination of "white-box / reproducer / in-kernel / operand-trace" with "kernel / cutlass / UMMA / megakernel / vllm".

설치로 이동 파일 검토

저장소: cfregly/gpu-perf-tune
최근 소스 활동: 2026년 8월 16일 20:38
감지된 SKILL.md 언어: 영어
스타: 4
포크: 0

설치 방법

기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.

AI 어시스턴트로 설치

이 프롬프트를 복사해 사용 중인 AI 어시스턴트에 입력하세요.

Follow https://skillsmp.com/skill-install/prompt.md to install only the Agent Skill at https://github.com/cfregly/gpu-perf-tune/tree/main/plugins/profile-and-optimize/skills/inference-kernel-whitebox-debug.

직접 명령은 검토 Prompt를 거치지 않습니다. 실행하기 전에 소스를 확인하세요.

npx skills add https://github.com/cfregly/gpu-perf-tune --skill inference-kernel-whitebox-debug

명령은 한 줄로 유지됩니다. 복사하기 전에 가로로 스크롤해 전체 내용을 확인하세요.

로컬 사본을 원하시나요? SkillsMP에서 현재 제공할 수 있는 파일을 다운로드하세요.

소스 파일 검토

설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.

SKILL.md

소스 지침 · 읽기 전용 미리보기

name

inference-kernel-whitebox-debug

description

검색으로 돌아가기 설치로 돌아가기

inference-aa-workload

Reproduce the Artificial Analysis (AA) language-model performance workload shapes against an OpenAI-compatible chat endpoint using NVIDIA AIPerf. Drives the three AA text shapes (1k input / >=1k answer, 10k / >=1.5k, 100k / >=2k) with temperature 0, top_p 1, and the vLLM-style min_tokens + ignore_eos "at least N answer tokens" guarantee. Two modes: synthetic (AIPerf generates the prompt at the token mean) and dataset-replay (a generated o200k_base-counted JSONL replayed identically). Ships a self-contained script and an `aa` perf_tune_report cell_run backend. Use when comparing a hosted inference endpoint to AA leaderboard numbers or reproducing AA's methodology. Triggers on "artificial analysis workload", "AA benchmark", "AA 1k/10k/100k shapes", "reproduce artificialanalysis.ai", "AA methodology", "aa-10k", "compare to AA leaderboard", or any combination of "artificial analysis / AA" with "workload / shape / benchmark / dataset".

2026-08-17

inference-dcgm-correlate

Correlate DCGM Prometheus byte-traffic counters with an inference-perf-bench sweep window to compute byte-grounded workload-level Speed-of-Light. The third tier of the SoL rigor hierarchy (after zymtrace sample-share and ncu per-kernel arithmetic intensity). Reads the sweep window from `inference_perfbench_v1.json.bench`, queries Prometheus via the Prometheus MCP for the DCGM PROF group (`DRAM_ACTIVE`, `NVLINK_TX/RX_BYTES`, `PIPE_TENSOR_ACTIVE`, `PIPE_FP16_ACTIVE`), falls back to `DCGM_FI_DEV_*` counter-tier metrics when PROF is not exported, and writes `<cell>/dcgm_correlation.json` with per-resource %SoL = real byte traffic / (peak * duration * n_gpus). Triggers on "dcgm correlate", "dcgm sol", "byte-grounded sol", "workload %sol", "dram active over sweep", "nvlink bytes", "tensor pipe active", "real-vs-peak workload bandwidth", or combinations of "dcgm / prometheus" with "sweep / window / sol / workload / byte-traffic".

2026-08-17

inference-decode-step-budget

Measure the low-concurrency (c=1..c=8) decode hot-path of a live vLLM pod FAST and CORRECTLY: where each token's time actually goes (GPU-busy vs host-idle gap vs comm), whether the workload is kernel-bound / host-bound / comm-bound, and what the addressable lever is. Uses vLLM's native /start_profile + /stop_profile HTTP endpoints (enable once, then restart-free shell-in captures in seconds). Encodes the correctness gates that prevent contaminated or under-counted budgets (clean single-stream driver, GPU-busy must include CUDA-graph execution, reconcile against driver-measured TPOT, reject load-time/lull/truncated captures). Triggers on "decode step budget", "c=1 hot path", "where does decode time go", "TPOT breakdown", "is decode kernel-bound or host-bound", "latency-tier profile", "GPU idle during decode", "start_profile", "torch profiler vllm", or any combination of "decode / c=1 / latency / TPOT / ITL" with "profile / budget / breakdown / hot-path / where" on a vllm deploy.

2026-08-17

이 저장소의 모든 Skills 보기

inference-kernel-whitebox-debug

소스 정보

설치 방법

AI 어시스턴트로 설치

소스 파일 검토

이 저장소의 다른 Skills

AI 어시스턴트로 설치

이 저장소의 다른 Skills