| name | opt-layout-alignment |
| description | Optimization skill — choose activation layouts so consecutive kernels hand off on-device without a host-side transpose. Canonical case: seq-first (seq, n_heads·head_dim) so RMSNorm → RoPE → FlashAttention → O-proj stay seq-first, eliminating 1–4 host transposes per layer. Invoked by phase-4-prefill-optimization (and phase-5 when decode introduces a transpose phase-4 didn't fix). |
Purpose
When two consecutive kernels disagree on activation layout, the host inserts a
transpose between them — a data round-trip that adds up across many per-layer
calls. This skill removes those transposes by choosing layouts that let
consecutive kernels hand off directly on-device. The canonical alignment is
seq-first activations (seq, n_heads·head_dim), which keeps RoPE,
FlashAttention, and the O projection on the same layout the GEMMs/RMSNorm
already produce — no host transpose between them.
Most inheritance deployments already run seq-first end-to-end (nothing to do —
skip). This skill applies when the deployment still has a host transpose
between two kernels.
Success criteria
Applying this skill is "successful" when ALL hold:
- Output cosine ≥ 0.99 vs the pre-alignment baseline (changing layout must
not change the math). Log
max_abs / max_rel informational.
make verify still PASSES (end-to-end gate).- The targeted host transpose(s) are gone — fewer host ops, lower wall time.
If (1)/(2) regress → a kernel did not actually accept the new layout; revert.
Knowledge base references
programming_examples/flash_attention/kernel_fusion_based/attn_npu2_seqfirst.py
— the seq-first FlashAttention variant (head_dim ≤ 64).programming_examples/flash_attention/kernel_fusion_based/ — the
head-first kernel + wrapper used for head_dim ≥ 128 (see caveat below)..claude/skills/debug-fa-runtime-failure — owns the of the
head_dim ≥ 128 routing.
