| name | phase-3-full-model-validation |
| description | Phase 3 of LLM deployment — wire all N layers and verify NPU matches the HF bf16 reference end-to-end (per-layer cosine via the shared `programming_examples/llms/verify/` diagnosis lens + token-level top-5 set-inclusion via its token-set gate) at canonical prompts. Catches accumulated drift, KV cache bugs, layer-indexed weight loading errors. Invoked after Phase 2 gate. |
Purpose
Phase 2 verified one transformer block. Phase 3 scales to all N layers
and confirms the full prefill stays numerically aligned with the HF bf16
reference end-to-end. Catches: accumulated BF16 drift across deep stacks,
KV cache layout bugs, layer-indexed weight loading errors, LM head
precision drops.
Phase 3 reuses the verify/ subsystem's two lenses:
make diagnosis — per-layer ffn_out cosine (NPU vs HF bf16) for
ALL layers. Informational by default; Phase 3 promotes it to a gate.make verify — token-level top-5 set-inclusion (NPU vs HF bf16
greedy sequences). Already a hard gate (exit 0/1).
There is no hand-written CPU full-model forward — the reference is HF
transformers bf16 throughout.
Phase 3 PASS criteria (HARD GATES)
Run make diagnosis (per-layer cosine, all layers) and make verify
(token-set gate) on the canonical prompts. All must hold:
Semantic correctness (token-level, vs HF bf16 — THE GATE)
make verify PASSES: at the first divergence between NPU and HF
greedy sequences, NPU's chosen token is in HF's top-5 AND HF's chosen
token is in NPU's top-5 (the compute_topk_set_check gate, GATE_K=5,
GATE_N_TOKENS=32), measured against HF bf16 on generated tokens. bf16
noise can flip top-1 even between mathematically equivalent
implementations, but almost never displaces a token out of the top-5.
This top-k token-set gate —
the GPU/industry-standard end-to-end signal, and the same gate Phase 6/7
re-run.
