Visualize a specific transformer decoder layer from an AutoDeploy FX graph text dump as a hierarchical DOT/PNG diagram. Optionally annotate nodes with actual GPU kernel names and durations from an nsys trace. Use when the user wants to visualize, inspect, or debug a layer in an AutoDeploy model graph dump. Triggers on: "visualize layer", "show layer", "graph of layer", "layer visualization", "dump graph layer". Assumes graph dumps already exist in a directory (produced by AD_DUMP_GRAPHS_DIR).
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
Visualize a specific transformer decoder layer from an AutoDeploy FX graph text dump as a hierarchical DOT/PNG diagram. Optionally annotate nodes with actual GPU kernel names and durations from an nsys trace. Use when the user wants to visualize, inspect, or debug a layer in an AutoDeploy model graph dump. Triggers on: "visualize layer", "show layer", "graph of layer", "layer visualization", "dump graph layer". Assumes graph dumps already exist in a directory (produced by AD_DUMP_GRAPHS_DIR).
license
Apache-2.0
metadata
{"author":"NVIDIA Corporation"}
AutoDeploy Layer Visualizer
Visualize a single transformer decoder layer from an AutoDeploy SSA graph dump.
Optionally overlay actual GPU kernel names and durations from an nsys trace.
Prerequisite knowledge: This skill assumes familiarity with the ad-graph-dump skill, which covers how to enable dumps via AD_DUMP_GRAPHS_DIR, file naming conventions, SSA format basics, and GraphModule section structure. Refer to ad-graph-dump for that context.
Inputs
Dump directory — path to a directory containing graph dump .txt files
Layer number — which decoder layer to visualize (e.g. 5)
(Optional) Dump file — specific .txt file. If not given, pick the file with the highest numeric prefix (final transform stage).
(Optional) Trace file — path to .nsys-rep or .sqlite trace file. When provided, GPU kernel names and durations are extracted and annotated onto each node in the visualization.
Workflow
Phase 0: Ask the user
Before starting, ask the user two questions (skip any already answered in their request):
Do you have an nsys trace file? (.nsys-rep or .sqlite) — if yes, kernel names and durations from the trace will be annotated directly on each node in the diagram.
If yes — prefill or decode? The CUDA graph structure differs between prefill and decode phases. Knowing which phase the trace captures helps correctly segment layers and map kernels.
Proceed once both are answered (or the user says no trace).
Phase 1: Select the dump file
If the user didn't specify a file, pick the file with the highest numeric prefix (the final transform stage). See ad-graph-dump for the file naming convention and how lexicographic sort matches pipeline order.
Phase 2: Read the graph dump
Read the selected .txt file. If it contains multiple GraphModule sections (delimited by ======== headers), pick the one labeled monolithic.model or the first/largest one with real operation nodes.
Phase 3: Extract the layer subgraph
This is the core step — you do this yourself by understanding the graph structure. Do NOT delegate to a script.
The dump is an SSA-form graph where each line is one of:
Seed nodes: Any operation whose arguments include a weight reference matching layers_N_ or layers.N. or a fused weight pattern like fused_*_N_* (where these are non-% references, i.e. weight parameters, not activation outputs from other ops).
Forward follow: From seeds, follow the dataflow forward — if a node consumes the output of a seed/layer node, it belongs to this layer too. Stop when you hit a node that consumes weights from a different layer (e.g. layers_M_ where M ≠ N).
Backward follow: From seeds, follow the dataflow backward to pick up intermediate ops (getitem, sub, floordiv, eq, mul, view, reshape, etc.) that feed into seed nodes. Stop when you hit:
A placeholder node (model input)
A node that belongs to a different layer (i.e., it was already identified as part of layer M ≠ N by the seed rule, or its own backward chain reaches a different layer's weights)
The output of a residual-add from the previous layer (this is the boundary between layers)
Critical: Generic arithmetic ops like sub_1, floordiv_1, mul_1, eq_1 that sit between two layers' MoE routing logic can be tricky. Trace their inputs backward — if they ultimately derive from layer M's weights/operations (not layer N's), they belong to layer M, not layer N. The suffix number on these generic ops does NOT indicate which layer they belong to; you must trace the dataflow.
External inputs: Nodes from outside the layer that feed into layer nodes are "external inputs" (show them as input nodes in the diagram). These include:
Previous layer's residual output
Global model buffers (RoPE caches like _ad_rotary_cos_sin_N, batch metadata)
KV cache placeholders
Phase 4: Produce the JSON
After extraction, output a JSON file at <dump_dir>/<dump_stem>_layer<N>.json with this structure:
inputs: list of node IDs that this node consumes (only nodes within the layer or external inputs)
weight_inputs: list of weight parameters consumed (name, shape, dtype)
Edge fields:
from, to: node IDs (both must be in nodes or external_inputs)
Group assignment heuristic:
norm: ops consuming input_layernorm or post_attention_layernorm weights
mla/attention: ops consuming self_attn_* weights or named *mla*, *rope*, *sdpa*
moe: ops consuming MoE weights (*moe*, *experts*), router ops (noaux_tc_op, topk), and the arithmetic ops that process router outputs (sub, floordiv, eq, mul between router and MoE fused op)
This script uses graphNodeId to extract a single CUDA graph replay, groups kernels by stream, and identifies which streams belong to which layer. It outputs a JSON with per-layer kernel sequences including short names, full names, durations, and stream IDs.
Phase 6: (Optional) Annotate each node with its trace kernels
When trace kernel data is available, you must map GPU kernels to individual FX graph nodes. This is the key step — the render script will display kernel names and durations directly on each node in the diagram.
How to map kernels to nodes:
Read the trace kernel JSON and the layer JSON side by side. For each FX graph node, identify which GPU kernel(s) it corresponds to based on the op type and the kernel execution order. Common mappings:
This reads the JSON and produces .dot and .png files. If nodes have trace_kernels fields, the script renders kernel names and durations directly on each node's label (prefixed with ⚡).
Present the output paths to the user.
Notes
The script requires graphviz (dot command) to be installed for PNG rendering
If the dump has multiple GraphModules, prefer the monolithic.model module or auto-select the largest
When in doubt about a node's layer membership, trace its weight dependencies all the way back — weight parameter names are the ground truth for layer assignment
The trace extraction script auto-converts .nsys-rep to .sqlite if needed (requires nsys on PATH)
CUDA graph kernels run across multiple streams per layer; the script handles multi-stream grouping automatically
