togethercomputer/xorl-sglang

add-jit-kernel

Step-by-step tutorial for adding a new lightweight JIT CUDA kernel to sglang's jit_kernel module

원문 언어: 영어

cookbook-add-model

Add a new model to the SGLang Cookbook (docs/, Mintlify), config-driven format — instantiate the model-agnostic template into a per-model config (+ benchmarks) JSX under src/snippets/configs/, an MDX page, the docs.json nav entry, NEW-tag hygiene, and the…

원문 언어: 영어

llm-torch-profiler-analysis

Unified LLM torch-profiler triage skill for `sglang`, `vllm`, `TensorRT-LLM`, and `TokenSpeed`. Use it to inspect an existing `trace.json(.gz)` or profile directory, or to drive live profiling against a running server when supported and return one three-table…

원문 언어: 영어

sglang-runtime-context

How SGLang's runtime configuration and process-global state are organized (RuntimeContext tiers, publish + namespace config bags, the pristine ServerArgs seed, override entry points, resource/stream/buffer leases, per-forward flags), the CI guardrails that…

원문 언어: 영어

sglang-diffusion-add-model

Use when adding a new diffusion model or Diffusers pipeline to SGLang.

원문 언어: 영어

sglang-diffusion-benchmark-profile

Use when benchmarking denoise latency or profiling a diffusion bottleneck in SGLang.

원문 언어: 영어

sglang-diffusion-modelopt-quant

Use when quantizing a diffusion DiT with NVIDIA ModelOpt and making the resulting FP8 or NVFP4 checkpoint loadable, verifiable, and benchmarkable in SGLang Diffusion.

원문 언어: 영어

sglang-diffusion-performance

Use when choosing the fastest SGLang Diffusion flags for a model, GPU, and VRAM budget.

원문 언어: 영어