Plan, execute, and analyze LLM serving benchmarks across vLLM and SGLang configurations. Use when user says "run benchmarks", "benchmark config", "QPS sweep", "compare serving configs", "custbench", "benchmark analysis", or "generate benchmark report". Do NOT use for writing Terraform (use terraform-automation), deployment validation (use deployment-orchestrator), or GPU hardware diagnostics (use gpu-infra-troubleshooting).
Launch and manage detached agent runs on EKS via the agent-runner CLI. Use when the user says "launch an agent run", "agent runtime", "agent-runner", "run this spec on the cluster", "check run status", "agent run logs", or "stop a run". Do NOT use for writing the agent-runner code itself (edit the sibling ../agent-runner repo), Terraform of the foundation (use terraform-automation), or GPU serving deployments (use infra-deployer).
Evaluate upstream PRs or commits individually against the K2.6 baseline. Tests each change in isolation using the same L0-L4 pipeline, identifying which are helpful, harmful, or neutral for our specific workload. Use when new upstream PRs land (Alpha-MoE, Mega MoE, FP8 fusion) or when evaluating a bundle of commits. Never assume a bundle is net-positive — the EFA harness found bundled expert commits regressed LL performance by 10%.
Analyze a failed or underperforming kernel candidate using ncu profile data and telemetry history. Produces a structured diagnosis (what's wrong, why, what to try next) and updates the constraint database. Use after /verify-kernel returns a failure or an L3/L4 regression, or when the agent needs guidance on which optimization direction to pursue.
Generate an optimized kernel candidate using the current state vector approach. Injects all active constraints from the database, uses the specified DSL (Triton/TileLang/CUDA), and targets the highest-headroom region identified by profiling. Use after profiling identifies a target, or when the previous candidate fails and the state vector has advanced.
View, query, add, or demote constraints in the kernel optimization constraint database. The database is append-only — constraints are never deleted, only demoted in severity if contradicted by evidence. Use to inspect what the agent has learned, manually inject domain knowledge, or query relevant constraints before generating a candidate.
Profile a GPU kernel or full serving pipeline using NSight Compute. Produces roofline classification (compute/memory/latency-bound), identifies top-N kernels by wall-clock time, and maps each to pipeline stage (MoE dispatch, MLA decode, attention, comm). Use when starting optimization, after a new baseline is established, or when the freeze manager redirects to a new target region.
Run the cascaded L0-L4 verification pipeline on a kernel candidate. Each level gates the next — a candidate that fails L1 never wastes time on L4. Records telemetry at every stage regardless of pass/fail. Use after /generate-candidate produces a new kernel, or when evaluating an upstream PR cherry-pick.