nvidia-tensorrt-llm-ad-conf-check

name	nvidia-tensorrt-llm-ad-conf-check
description	Check whether AutoDeploy YAML configs were actually applied by analyzing server logs and optionally graph dumps (AD_DUMP_GRAPHS_DIR). Use when the user wants to verify config application, debug config issues, or check if AutoDeploy transforms (piecewise CUDA graph, multi-stream, sharding, fusion, etc.) were applied or fell back. Triggers on: "check config", "verify config", "ad-conf-check", "were my configs applied", "config not working", "check if piecewise is enabled", "check log for config", or any request to compare AD YAML settings against runtime behavior.
license	Apache-2.0
metadata	{"author":"NVIDIA Corporation"}

AutoDeploy Config Checker

Verify that AutoDeploy YAML configs were applied at runtime by cross-referencing with server logs and optionally graph dumps.

Input

• TensorRT-LLM source directory (required) — path to the TensorRT-LLM repo root. Used to read the latest default.yaml and source code for up-to-date log patterns (the bundled reference doc may be stale).
• YAML config file path(s) (required) — one or more AutoDeploy YAML configs. When multiple files are provided, they are deep-merged left-to-right (later files override earlier ones for overlapping keys).
• Server log file path (required) — log output from the AutoDeploy server run.
• Graph dump directory (optional) — AD_DUMP_GRAPHS_DIR output directory containing per-transform graph snapshots (NNN_stage_transform.txt). Provides additional evidence for resolving UNKNOWN results.
• Nsys trace file (optional) — Nsight Systems profile (.nsys-rep or .sqlite) from the server run. Useful for verifying executor-level configs that produce no log output (e.g., enable_chunked_prefill, multi-stream concurrency, CUDA graph capture/replay).
• Table output file path (optional) — path to write human-friendly table results.
• JSON output file path (optional) — path to write machine-friendly JSON results.

Output

Human-friendly table (always presented to user)

• Verification table — one row per config key with columns: Config (key=value), Result (APPLIED / FAILED / SKIPPED / DISABLED / UNKNOWN), Evidence (log line or graph analysis proving the result).
• Summary line — total counts per status (e.g., Total configs checked: 29 | APPLIED: 23 | UNKNOWN: 4 | ...).
• FAILED/WARNING details — expanded information for any configs that failed or had warnings.

Machine-friendly JSON (when JSON output path is given)

JSON file with two top-level keys:

• results — array of objects, each with config, value, status, evidence.
• summary — object with total (int) and counts (object mapping status to count, only non-zero statuses included).

Workflow

1. [Collect Inputs] Ask the user for the following inputs:

   • TensorRT-LLM source directory (required) — path to the TensorRT-LLM repo root. Used to cross-check default.yaml and source code for the latest log patterns.
   • YAML config file path(s) (required) — one or more AutoDeploy configs used for the run. When multiple YAMLs are provided, they are deep-merged left-to-right: later files override earlier ones for overlapping keys. Tell the user: "If you have multiple configs (e.g., a default config and a user override), list them in priority order — lowest priority first, highest priority last."
   • Server log file path (required) — the log output from the server
   • Graph dump directory (optional but recommended) — the AD_DUMP_GRAPHS_DIR output directory containing per-transform graph snapshots. Files are named NNN_stage_transform.txt and show the graph AFTER each transform. When provided, graph analysis provides additional evidence (e.g., verifying sharded weights, collective ops, fused ops). This is especially useful for resolving UNKNOWN results.
   • Nsys trace file (optional) — Nsight Systems profile (.nsys-rep or .sqlite) from the server run. Useful for verifying executor-level configs that produce no log output (e.g., enable_chunked_prefill, multi-stream concurrency, CUDA graph capture/replay).
   • TensorRT-LLM source reference paths:
     • Example configs: <trtllm_src>/examples/auto_deploy/model_registry/configs/*.yaml
     • Default transform config (all available transforms and their defaults): <trtllm_src>/tensorrt_llm/_torch/auto_deploy/config/default.yaml

2. [Update Reference Doc] Before checking configs, ensure the bundled reference doc is up-to-date with the TensorRT-LLM source.

   Launch the ad-conf-check-update agent with:

   • <trtllm_src> — the TensorRT-LLM source directory from step 1
   • <skill_dir> — the directory containing this SKILL.md file

   The agent compares <trtllm_src>/tensorrt_llm/_torch/auto_deploy/config/default.yaml and the AutoDeploy source code against <skill_dir>/references/config_log_patterns.md. If any configs were added, removed, renamed, or if log patterns have changed, the agent updates the reference doc in-place and reports what changed.

Key Patterns to Know

• Every transform logs: [stage=<stage>, transform=<name>] [SUMMARY] matches=N | time: ...
• Piecewise success chain: dual-mode enabled -> prepared with N submodules -> captured graphs
• Piecewise failure: "model is not a GraphModule...Falling back to eager execution"
• Sharding: "Using allreduce strategy: SYMM_MEM", "Applied N TP shards from config"

Gotchas

• Every YAML key must appear in the output. Check all configs from the YAML, not just ones with known patterns. If a config key has no entry in the reference doc, grep the log for the key name and value. New/unknown configs should still be reported — never silently skip them.
• UNKNOWN does not mean the config was ignored. Some configs (e.g., enable_chunked_prefill, enable_block_reuse) are consumed at executor/runtime level and produce no log output. UNKNOWN means "no log evidence found", not "config was not applied".
• Deprecated config names may cause FAILED. For example, torch_dtype is deprecated in favor of dtype, and cuda_graph_batch_sizes (top-level) is replaced by cuda_graph_config.batch_sizes. Look for deprecation warning messages in the log. Old keys may be silently ignored.
• Runtime may adjust configured values. For example, max_seq_len may be configured as 262144 but adjusted down to 16384 at runtime due to memory constraints. Report this as APPLIED with a WARNING annotation.
• ANSI color codes in logs. AutoDeploy uses colored log output. Strip or ignore ANSI escape sequences when matching patterns.
• Reference doc is auto-updated. Step 2 runs the ad-conf-check-update agent to sync references/config_log_patterns.md with the latest TensorRT-LLM source before any config checking begins. If the agent reports changes, review its summary to understand what shifted.