| name | mirage-graph-coverage |
| last_validated | "2026-06-05T00:00:00.000Z" |
| description | Read-only coverage auditor for the mirage / MPK persistent-megakernel's GENERATED task graph: cross-references DECLARED tensors (all_tensors[...] in kernel_N.cu) against tensors CONSUMED by tasks (inputs/outputs base_ptr in task_graph_N.json) and flags any tensor declared but never consumed as an input - the "feature defined in code but never wired into the task graph" bug class (e.g. RoPE cos/sin tables allocated but consumed by 0 tasks => position-blind attention => incoherent / degenerate-repetition output). A numerics / relL2 harness silently PASSES on this class, so run THIS first when a megakernel compiles + exits 0 but emits incoherent output. Distinct from inference-graph-diff (vLLM FX / Inductor graphs). Triggers on "graph coverage", "task graph audit", "megakernel wiring", "declared but unconsumed tensor", "rope not wired", "missing task", or "megakernel / mirage / MPK / task-graph" with "coverage / wiring / unconsumed / unwired / audit". |
mirage megakernel graph-coverage auditor
Catches the missing-wiring bug class in a mirage/MPK generated megakernel:
a tensor (or whole feature) is defined and allocated but never wired into a
task, so the kernel runs without it. The canonical case is GLM-5.1 on GB300
where the RoPE cos/sin tables were attached but consumed by 0 tasks, so
attention ran position-blind and the model emitted fluent-but-incoherent,
repetition-collapsing tokens - while every per-kernel numerics test passed.
Quick start
python3 assets/mk-graph-coverage-audit.py <gendir>
Exit 0 = PASS (every critical tensor consumed as input), 1 = FAIL (a critical
tensor is declared but never consumed = missing wiring), 2 = no artifacts.
What it does
Per TP rank, it parses two generated artifacts and diffs them:
- declared =
all_tensors["NAME"] = NAME; lines in kernel_<rank>.cu
- consumed =
base_ptr of every all_tasks[].inputs[] / .outputs[] in
task_graph_<rank>.json, mapped to the set of consuming task_types + counts
It then classifies each declared tensor and FAILs iff a critical tensor
(rope_cos,rope_sin,cos_pos_embed,sin_pos_embed by default) has 0 input
consumers.
Reading the output
[OK] <t>: input_refs=N consumer_task_types=[...] - wired (consumed by a task).[MISSING-WIRING] <t>: input_refs=0 - declared but never read by any task => the bug.INFO write-only - produced by a task but never read (often a final output like
output_token. Suspicious only for an intermediate/feature tensor).INFO dead - declared, 0 refs at all (allocated scratch / codegen leftover. The
nullptr sentinel shows here as a benign false positive).
Worked example (GLM-5.1 on GB300)
Before the wiring fix the auditor FAILs: rope_cos input_refs=0 (rope absent from the
graph). After the fix it PASSes: rope_cos/rope_sin input_refs=156,
consumer_task_types=[278, 296] (296 = q-rope-apply, 278 = k_pe / KV-cache
gather). Same artifacts a relL2 numerics harness would have called clean in both
states - graph-coverage is what distinguishes them.
Triage rule
For a megakernel coherence bug (compiles, exits 0, incoherent output), run the
graph-coverage audit before building a per-kernel numerics harness. A
missing/mis-wired task makes a numerics harness pass while the model stays wrong,
the audit names the unwired tensor directly. Only once coverage is clean does a
relL2 harness make sense.
Caveat
Verifies structural wiring (is the tensor consumed by a task), not runtime
numerical coherence. A PASS removes "feature absent" as the suspect. It does not
by itself prove the output is correct.
