一键在 Manus 中运行任何 Skill

benchmark

星标60

分支10

更新时间2026年5月31日 08:57

Reference for the active benchmark harness — what it IS and how it behaves (the active benchmark is flashinfer-bench). Covers config.toml structure (`[solution]`/`[build]`/`[benchmark]` tables), the status enum, the workload / fresh-inputs model, reference-baseline + scoring mechanics, tolerance keys, TVM-FFI builder link limits, and the silent-skip cascade. Invoke whenever you decode a bench status string, hit an unfamiliar config.toml field, suspect "correctness passed but the headline is implausible", or need to know what's frozen before proposing a bench edit — do NOT guess field or status semantics. Bench commands and noise methodology (A/B compare, variance check, drift cancellation) live in the `bench` skill.

安装

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

在 Manus 中运行

来源

TongmingLAIC

TongmingLAIC/AKO4X

打开 GitHub 仓库查看创作者相关仓库

下载

在 Manus 中运行

Benchmark

The active benchmark harness — what this project's bench skill is a frontend for. Reference for what the benchmark is and how it behaves — distinct from the bench skill which covers how to drive it from the harness shim. The active benchmark is flashinfer-bench; this skill is the single home for its schema, status, scoring, and frozen-for-comparability behavior. Detailed body: benchmark.md.

When this skill applies:

Looking up a config.toml field you don't recognize.
Decoding a status string from bench output (COMPILE_ERROR, INCORRECT_NUMERICAL, etc.).
Reasoning about why correctness "passed" but the headline looks too good (silent-skip cascade ends here).
Checking what's frozen for bench comparability (scoring/baseline/tolerance) before proposing a harness edit.
Understanding the TVM-FFI link limit for C++/CUDA (no compile-time linking) and which kernel libraries are disallowed (cuBLAS / cuDNN) vs. allowed building blocks (CUTLASS, the DSLs) — see "Valid solution: write your own kernel".

同仓库更多 Skills

同仓库

bench

TongmingLAIC/AKO4X

Run performance benchmarks to get a verdict on whether a kernel change actually helped. Use --ab-compare for sub-1x deltas (drift cancels in same container), --variance-check for noise floor, and the subset modes (--first, --smoke; --extremes is modal-backend only) for compile-correctness probes only — those are NOT performance verdicts. The only authoritative source for "is this change real". Benchmark specifics (config.toml schema, status enum, scoring formula, baseline rule, fresh-inputs contract) live in the `benchmark` skill.

2026-05-3160

cpp

TongmingLAIC/AKO4X

Host-side C++ kernels (`[build].language = cpp`) — the `solution/{kernel.cpp, binding.py}` two-file TVM-FFI binding pattern, and when to reach for it (host-side orchestration of a kernel you write, or composing CUTLASS headers into it). Distinct from the `cuda` skill which covers `.cu` device kernels compiled by nvcc. Use when writing a host-side C++ kernel or deciding cpp-vs-cuda.

2026-05-3160

cuda

TongmingLAIC/AKO4X

CUDA C++ (.cu) kernel reference — TVM-FFI direct-export vs Python-binding entry points, the chevron-launch null-stream pitfall under CUDA-graph capture, the sm_100 fp8→bf16 cvt PTX gap, the load_inline name-cache trap, and the __launch_bounds__ register-spill lever. Also the single-source for generic per-call-overhead / CUDA-graph-capture / PDL theory (Waves-Per-SM decision table) regardless of DSL. Use when writing or debugging hand-written CUDA C++ kernels, or whenever you need generic CUDA-graph / sync-audit / PDL reasoning.

2026-05-3160

cute-dsl

TongmingLAIC/AKO4X

CuTe DSL reference — NVIDIA's CUTLASS Python DSL. Use whenever writing or debugging a CuTe DSL kernel — covers the `@cute.kernel` + `@cute.jit` + `.launch()` host pattern, `from_dlpack` tensor conversion, API-probing against the installed wheel, the launch-kwarg table, and the well-known graph-capture pitfall (CuTe's TVM-FFI stream binding does NOT pick up CUDA-graph capture mode — flashinfer-bench issue 414).

2026-05-3160

profiler-ncu

TongmingLAIC/AKO4X

Run NVIDIA Nsight Compute (NCU) per-kernel profiling for register pressure, occupancy, stall-reason, memory-throughput, and IPC analysis. Use BEFORE architecting an optimization fix — not only after — when a hypothesis about microarchitectural behavior needs verification. NCU `Duration` is NOT comparable to bench timing; use NCU for ratios only.

2026-05-3160

sanitizer

TongmingLAIC/AKO4X

Run NVIDIA compute-sanitizer (memcheck / racecheck / initcheck / synccheck). FIRST action on `INCORRECT_NUMERICAL` or flaky output — rolling back destroys the evidence; sanitizer often points at the exact line. Also useful for diagnosing race conditions in multi-block kernels and uninitialized-memory reads.

2026-05-3160

name

benchmark

description

Benchmark

When this skill applies:

Looking up a config.toml field you don't recognize.
Decoding a status string from bench output (COMPILE_ERROR, INCORRECT_NUMERICAL, etc.).
Reasoning about why correctness "passed" but the headline looks too good (silent-skip cascade ends here).
Checking what's frozen for bench comparability (scoring/baseline/tolerance) before proposing a harness edit.
Understanding the TVM-FFI link limit for C++/CUDA (no compile-time linking) and which kernel libraries are disallowed (cuBLAS / cuDNN) vs. allowed building blocks (CUTLASS, the DSLs) — see "Valid solution: write your own kernel".