| name | sglang-torch-profiler-analysis |
| description | Unified SGLang torch-profiler skill for trace generation, kernel/category breakdown, two-stage overlap analysis, and small Perfetto trace repair. Use when Codex should inspect an existing `trace.json(.gz)` or profile directory, trigger `sglang.profiler` against a live server, break down prefill/decode GPU time by kernel family, correlate a graph-off mapping trace with a graph-on formal trace to find overlap headroom tied back to Python code, or rewrite a trace so Perfetto renders overlapped events more reliably. |
SGLang Torch Profiler Analysis
Overview
Use this skill for all SGLang torch-profiler work. It replaces the old split between:
- kernel/category breakdown
- overlap-specific diagnosis
- small trace post-processing
Prefer the unified entrypoint:
This entrypoint exposes four subcommands:
-
triage: the default compact workflow that prints three main tables
-
breakdown: one-trace kernel/category share analysis
-
overlap: required two-trace overlap analysis with source mapping
-
perfetto-fix: rewrite a trace when Perfetto drops some overlapped lanes
For normal use, prefer triage. It already collapses the result into three main tables:
- kernel table
- overlap-opportunity table
- fuse-opportunity table
Internal analyzers live here:
When To Use It
- inspect an SGLang torch profiler trace or profile directory
- profile a live SGLang server and immediately analyze the output
- quantify which kernel families dominate prefill or decode
- compare communication, attention, MoE, quantization, norm, or memory share
- map kernels back to Python code paths
- judge whether a kernel still has overlap headroom in production shape
- get a text table and a small ASCII timeline without opening Perfetto first
- repair a trace so Perfetto can render overlapping events more faithfully
Do not use Nsight Systems as the default path for this workflow. This merged skill is torch-profiler-first.
Main Commands
1. Compact triage from existing trace directories
python3 scripts/analyze_sglang_torch_profile.py triage \
--mapping-input /path/to/graph_off_profile_dir \
--formal-input /path/to/graph_on_profile_dir
2. Compact triage from running servers
python3 scripts/analyze_sglang_torch_profile.py triage \
--mapping-url http://127.0.0.1:31025 \
--formal-url http://127.0.0.1:31026 \
--num-steps 8 \
--profile-by-stage
3. Breakdown from an existing trace or profile dir
python3 scripts/analyze_sglang_torch_profile.py breakdown \
--input /path/to/profile_dir
4. Breakdown from a running server
python3 scripts/analyze_sglang_torch_profile.py breakdown \
--url http://127.0.0.1:30000 \
--num-steps 8 \
--profile-by-stage \
--table-only
5. Two-stage overlap analysis
python3 scripts/analyze_sglang_torch_profile.py overlap \
--mapping-input /path/to/graph_off_profile_dir \
--formal-input /path/to/graph_on_profile_dir \
--table-only
Or profile both servers directly:
python3 scripts/analyze_sglang_torch_profile.py overlap \
--mapping-url http://127.0.0.1:31025 \
--formal-url http://127.0.0.1:31026 \
--num-steps 8
6. Perfetto-friendly trace rewrite
python3 scripts/analyze_sglang_torch_profile.py perfetto-fix \
--input /path/to/trace.json.gz
This small repair step is inspired by torch_utils/src/convert_to_perfetto_compatible/convert_to_perfetto_compatible.py.
profile_by_stage
profile_by_stage is not only for PD disaggregation.
- On ordinary non-PD serving, it is still useful because prefill and decode usually have very different bottlenecks.
- On the current profile-v2 path inside SGLang, stage-based profiling is effectively the normal path.
- PD-disaggregated serving adds one extra rule: prefill workers and decode workers must be profiled separately. That is stricter than ordinary
profile_by_stage.
Which Mode To Choose
triage
Use when you want the lowest-friction output:
- one kernel table
- one overlap-opportunity table
- one fuse-opportunity table
- optional stage-aware rows when the trace directory includes both
EXTEND and DECODE
This is the recommended default for final user-facing reports.
breakdown
Use when you need:
- category share such as attention, communication, MoE, norm, quantize, memory
- top kernels by cumulative GPU time
- stage-aware prefill vs decode summaries
- kernel tables already joined with Python locations and CPU ops
- conservative source-backed fusion opportunities
This mode works with one trace. A graph-off pre-pass plus --kernel-map is optional but recommended for the final polished report.
overlap
Use when you need:
- a strong answer about which code paths still have overlap headroom
- a table that says which kernels are already hidden and low ROI
- dependency-risk hints near adjacent kernels
- an ASCII timeline around the most actionable windows
This mode requires two traces for a final answer:
- mapping trace with
--disable-cuda-graph --disable-piecewise-cuda-graph
- formal trace with the real serving optimizations enabled
Do not call the mapping pass a "fast profile". It exists to recover kernel -> cpu_op -> python scope.
perfetto-fix
Use only when Perfetto fails to render obviously overlapping events cleanly. It is a post-processing utility, not the main analysis flow.
Workflow
One-trace breakdown workflow
- If the user only wants kernel/category share, one trace is enough.
- Prefer rank-local
TP-0 traces over merged traces.
- For a live server, this skill can call
sglang.profiler and automatically send a small probe request.
- Prefer
--profile-by-stage even on standard serving unless the user explicitly wants an all-stage mixed trace.
Two-trace overlap workflow
- Produce a mapping trace first with graph disabled.
- Produce a formal trace second with graph enabled and the real serving flags kept on.
- Run
triage for the compact three-table report, or overlap if you also want source context and ASCII timelines.
- Read the results in this order:
- kernel table
- overlap-opportunity table
- fuse-opportunity table
- Before calling something a "new" optimization idea, compare the top rows against references/fuse-overlap-catalog.md. Prefer reporting:
- an existing fused or overlap path that should already apply here
- an existing path that appears disabled, unsupported, or regressed in this trace
- a truly new opportunity only when no catalog entry fits
- Use the deeper
overlap report only when you need source context or ASCII timelines beyond the compact three-table artifact.
References
Load these only when needed:
Output Contract
For breakdown
Return:
- trace path
- model/server args when available
- top categories
- top kernels
- one short conclusion about what dominates the run
- any source-backed fusion opportunities worth checking
If the user wants a compact artifact from breakdown only, use --table-only so the output centers on the kernel table and the fuse-opportunity table.
For overlap
Return:
- mapping trace path
- formal trace path
- key rows from the action table
- Python code paths with the strongest overlap headroom
- kernels that are already hidden and should be deprioritized
- at least one ASCII timeline
If one of the required traces is missing, say that clearly and identify which trace still needs to be collected.
If the user wants a compact artifact from overlap only, use --table-only so the output centers on the overlap-opportunity table.
For triage
Return:
- mapping trace path or directory
- formal trace path or directory
- one kernel table
- one overlap-opportunity table
- one fuse-opportunity table
- one short note saying whether the top findings match an existing catalog entry or look genuinely new
This should be the default final artifact unless the user explicitly asks for the longer overlap walkthrough.
