Build a Quark ONNX PTQ quantization plan from `model_analysis.json` and user intent. Use when the user needs preset selection (XINT8 / A8W8 / A16W8 / BF16 / BFP16 / MX* / MXFP* …), calibration method choice (MinMax / Entropy / Percentile / Distribution / NonOverflow / MinMSE / LayerWisePercentile), algorithm selection (CLE / AdaRound / AdaQuant / BiasCorrection / AutoMixprecision), deployment-target gating (CPU / CUDA / ROCm / AMD NPU CNN / AMD NPU Transformer), op-type include/exclude lists, weights-only INT4 (MatMulNBits) decisions, dynamic vs static quantization, or a draft `quant_plan.json`. Trigger for "what preset should I use for my ONNX model", "choose XINT8 vs A8W8", "plan ONNX PTQ", "INT4 "BFP16 / MXFP4 for my model", "calibration method for Ryzen AI", "should I use AdaRound or AdaQuant", "SmoothQuant alpha", or when the user has a model analysis and needs to decide how to quantize an ONNX model.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
Instruções da origem · Visualização somente leitura
name
quark-onnx-quant-plan
description
Build a Quark ONNX PTQ quantization plan from `model_analysis.json` and user intent. Use when the user needs preset selection (XINT8 / A8W8 / A16W8 / BF16 / BFP16 / MX* / MXFP* …), calibration method choice (MinMax / Entropy / Percentile / Distribution / NonOverflow / MinMSE / LayerWisePercentile), algorithm selection (CLE / AdaRound / AdaQuant / BiasCorrection / AutoMixprecision), deployment-target gating (CPU / CUDA / ROCm / AMD NPU CNN / AMD NPU Transformer), op-type include/exclude lists, weights-only INT4 (MatMulNBits) decisions, dynamic vs static quantization, or a draft `quant_plan.json`. Trigger for "what preset should I use for my ONNX model", "choose XINT8 vs A8W8", "plan ONNX PTQ", "INT4 "BFP16 / MXFP4 for my model", "calibration method for Ryzen AI", "should I use AdaRound or AdaQuant", "SmoothQuant alpha", or when the user has a model analysis and needs to decide how to quantize an ONNX model.
Convert an ONNX model_analysis.json plus the user's intent into a confirmed quant_plan.json.
This skill makes the quantization decisions for the ONNX-to-ONNX flow — which preset, which
calibration method, which algorithm, which op types to include/exclude, whether to enable an NPU
target, whether to use external-data — without generating scripts or executing the
quantization. The plan is the contract between the user's intent and the execution step.
Inputs
model_analysis.json from quark-onnx-model-intake (architecture, op histogram, opsets,
external-data state, preset_candidates, risks)
env_context.json for accelerator-aware preset gating (CUDA major / ROCm major / NPU presence)
onnx_install_result.json (optional) — gates BFP/MX/Extended presets (require the custom-ops
library to be compiled for the target EP)
User preferences: deployment target, accuracy goal, model-size goal, calibration data
availability
Outputs: quant_plan.json
Records the chosen preset (or custom config), calibration method, algorithm list, layer/op
overrides, NPU flag, external-data setting, and evaluation intent. Shares the schema with the
Torch plan; ONNX-specific fields live under onnx_specific.
Quark ONNX ships 50+ named presets in quark/onnx/quantization/config/custom_config.py. Pick the
smallest viable set for the user's architecture + deployment target; never list them all.
AMD NPU CNN — Ryzen AI / VAI (enable_npu_cnn=True, PoF2 scales, NHWC)
Preset
Description
Picks
XINT8
INT8 input + INT8 weight, optimized for NPU
Default for any CNN targeting NPU
XINT8_ADAROUND
XINT8 + AdaRound fast-finetune
When base XINT8 loses accuracy
XINT8_ADAQUANT
XINT8 + AdaQuant fast-finetune
When AdaRound is not enough
VINT8
INT8 optimized for VAIML
VAIML deployment
AMD NPU Transformer (enable_npu_transformer=True)
Preset
Description
Picks
INT16_TRANSFORMER_DEFAULT
INT16 activations + INT8 weights, fast
Outlier-heavy activations
INT16_TRANSFORMER_ACCURATE
INT16 + accuracy algorithms
Largest accuracy headroom
General CPU / CUDA / ROCm INT8 (deployment-agnostic)
Required for AMD NPU XINT8 — picks the smallest PoF2 scale that doesn't overflow
PowerOfTwo_MinMSE (a.k.a. MinMSE)
Required for AMD NPU XINT8 — picks the PoF2 scale that minimizes MSE; usually better than NonOverflow
Int16Method.MinMax
For INT16 configs
NPU CNN / XINT8 targets must use a PowerOfTwo* method. Non-PoF2 scales are rejected at NPU
runtime — flag this as a hard constraint in the plan.
Available Algorithms
From quark/onnx/quantization/config/algorithm.py and examples/onnx/accuracy_improvement/:
Algorithm
Kind
Compatible presets
Description
CLE (Cross-Layer Equalization)
Pre
INT8 CNN configs
Folds BN, equalizes per-channel scales across consecutive Conv/Linear layers (Nagel et al., 2019)
BiasCorrection
Post
INT8 CNN
Post-hoc bias adjustment (Nagel et al., 2019)
AdaRound
Post (fast-finetune)
XINT8 / A8W8 / A16W8 / block formats
Adaptive rounding optimization; needs cal data + LR + iterations; GPU-accelerated
AdaQuant
Post (fast-finetune)
Same as AdaRound
Layer-wise calibration tuning; usually after AdaRound is not enough
AutoMixprecision
Post
Block formats / mixed-precision
Auto-selects sensitivity-based per-layer dtype; can do dual BFP16+MX hybrid
Algorithms compose: e.g. CLE (pre) + AdaRound (post) is a common XINT8 recipe. Combinations
beyond two algorithms are usually a red flag — flag them in risks.
Decision Guide
Help the user choose based on their priorities and the architecture from
model_analysis.json.model.onnx_specific.architecture_guess:
User intent
Architecture
Recommended starting plan
"Best accuracy, AMD GPU"
any
BF16 or BF16_MIXED_BFP16
"INT8 CNN, CPU/GPU deployment"
cnn
INT8_CNN_DEFAULT or A8W8; add CLE if accuracy drops
"INT8 CNN → Ryzen AI NPU"
cnn
XINT8 + CLE, calibration = PowerOfTwo_MinMSE; usually NHWC pre-conversion via quark.onnx.tools.convert_nchw_to_nhwc
"Ryzen AI NPU, accuracy-tight CNN"
cnn
XINT8_ADAROUND (then XINT8_ADAQUANT if still short)
"Block format MXFP4 / BFP16 experimentation"
any
BFP16 or MXFP4E2M1; require quark.onnx.operators.custom_ops to be compiled
"Hybrid mixed-precision for best size/accuracy"
any
BF16_MIXED_BFP16 or S16S16_MIXED_S8S8 + AutoMixprecision
Reject A8W8 / BFP16 / MX* / FP16 if user requests npu_cnn
npu_transformer (Ryzen AI Transformer)
enable_npu_transformer=True, INT8/INT16 QDQ on MatMul/Gemm
MinMax / Percentile typically
Reject CNN-only presets
cuda
onnxruntime-gpu present, CUDAExecutionProvider available
Any
Block-format presets require custom-ops
rocm
onnxruntime_rocm (ROCm 6.x) or CPU ORT on ROCm 7.x (tools/ci/install_onnxruntime.sh)
Any
Custom-ops library must compile for ROCm
cpu
Any
Any
Block-format presets work via CPU custom-ops; expect speed cost
If the deployment target conflicts with a requested preset, the plan must either (a) downgrade
to a viable preset and explain, or (b) leave it unset with a high-severity risk in quant_plan.json.
Op-Type / Node Include/Exclude Levers
Three commonly-used knobs the plan should expose:
op_types_to_quantize — restrict QDQ insertion to a subset, e.g. ["Conv"] for
CNN-only quantization.
nodes_to_quantize / nodes_to_exclude — surgical per-node control by graph node name.
Node names change after pre-processing (NCHW→NHWC, BN folding, etc.), so resolve them after
any pre-processing pass.
extra_options["OpTypesToExcludeOutputQuantization"] — keep certain op outputs in float
while still quantizing their inputs/weights.
Extra Options Commonly Set in quant_plan.onnx_specific.extra_options
AdaRound / AdaQuant tutorials and Auto-Search tutorials
Decision Table (MUST show to user)
ALWAYS present this table and WAIT for confirmation before finalizing. Fill the "Value"
column from the user's request, the model analysis, and the deployment-target gates above.
Decision
Value
Reason
backend
onnx
Fixed for this skill
deployment_target
(fill: cpu / cuda / rocm / npu_cnn / npu_transformer)
(from env or user)
preset
(fill: name from DefaultConfigMapping or "custom")
(fill: list e.g. ["CLE"], ["AdaRound"], ["AdaQuant"], ["BiasCorrection"])
(why)
onnx_specific.enable_npu_cnn
(fill: bool)
Hard-tied to deployment_target == "npu_cnn"
onnx_specific.enable_npu_transformer
(fill: bool)
Hard-tied to deployment_target == "npu_transformer"
onnx_specific.include_cle
(fill: bool)
CNN INT8 default true
onnx_specific.op_types_to_quantize
null or ["MatMul"] etc.
(why)
onnx_specific.use_external_data_format
true if model > 2 GB (from model_analysis.json)
Required for large models
onnx_specific.extra_options
(fill: dict)
(why — list each key)
evaluation_intent
smoke (default), mlperf, mAP
(why)
After showing the table, ask: "Confirm this plan? Any changes?"
Do NOT proceed until the user confirms.
Per-Layer Overrides
For fine-grained control, individual layers can override the global config via QLayerConfig
(see examples/onnx/yolo_quantization/quantize_yolo.py):
from quark.onnx import QConfig, QLayerConfig, XInt8Spec, CLEConfig
config = QConfig(
global_config=QLayerConfig(activation=XInt8Spec(), weight=XInt8Spec()),
algo_config=[CLEConfig()],
EnableNPUCnn=True,
exclude=[
# YOLOX-style: keep a specific subgraph in float
(["/_head/_modules_list.14/Transpose"], ["/_head/_modules_list.14/Concat_9"]),
],
)
Record any per-layer overrides under onnx_specific.subgraphs_to_exclude (list of
(start_nodes, end_nodes) tuples) or onnx_specific.nodes_to_exclude (flat name list).
Rules
Keep scope to plan creation only. Do not generate scripts, do not run quantization, do not
export. Those are separate skills.
Require a model analysis first. Without architecture, op histogram, opsets, and the
already-QDQ flags, you cannot make informed preset / op-type recommendations. If
model_analysis.json is missing, route back to quark-onnx-model-intake.
Honor hard deployment-target constraints. NPU CNN ⇒ XINT8 family + PowerOfTwo*
calibration + NHWC. NPU Transformer ⇒ INT8_TRANSFORMER_* / INT16_TRANSFORMER_*. Reject
conflicting preset requests with a clear explanation.
Custom-op preconditions for block formats. Any preset in {BFP16*, MX*, MXFP*, MXINT8*, BF16_*BFP*, BF16_*MXINT8*} requires the Quark custom-ops library to have compiled. If
onnx_install_result.json shows the compile failed, suppress these presets from
recommendations and emit a risk pointing back to quark-onnx-install.
External-data flag tracks model size. If model_analysis.json.model.estimated_size_gb > 2,
set onnx_specific.use_external_data_format = true automatically. Otherwise default false.
Always present the decision table before finalizing. The user must explicitly confirm.
Record risks. If the user picks a risky combination (e.g. XINT8 without PoF2 calibration,
MXFP4 on a model that hasn't been validated with AutoMixprecision), keep the user's choice
but record the risk in the plan.
Do not duplicate fields between model_analysis.json and quant_plan.json. The plan
references the analysis via analysis_ref.
Interaction Flow
Check prerequisites: Is model_analysis.json available? If not, route to
quark-onnx-model-intake first. Is onnx_install_result.json available? If a block-format
preset is on the table, require it.
Confirm deployment target: From session_context.json.constraints.deployment_target or
ask. Apply the hard gates from the table above.
Narrow presets: Start from model_analysis.json.quantization_targets.onnx_specific.preset_candidates,
filter by deployment target and custom-op availability, present the top 1–3 with rationale.
Pick calibration method: Default per target (MinMax for general; PowerOfTwo_MinMSE for
NPU CNN; Percentile for outlier-heavy transformers; consider LayerWisePercentile when
accuracy is critical). Decide data_size and batch_size.
Pick algorithms: Default per architecture and preset (CLE for INT8 CNN;
AdaRound→AdaQuant for accuracy-tight CNN; BiasCorrection as a cheap post-hoc fixup).
At most two algorithms unless justified.
Set extra options: From the table of common knobs, only set what's needed; document each.
Present the decision table: Show defaults, explain the tradeoffs, let the user adjust.
Confirm: Always required. Show the final plan summary before writing.
Emit: Write quant_plan.json. Surface any new constraints back to quark-onnx-router so
they land in session_context.json.
Recovery
If model_analysis.json shows has_qdq_already=true or has_quark_custom_ops=true — do
not produce a plan. Route back to quark-onnx-model-intake with a recommendation to
remove existing QDQ first (quark.onnx.tools.remove_qdq).
If the user picks a preset that conflicts with their deployment target — explain why it won't
work (e.g. "A8W8 uses non-PoF2 scales which the AMD NPU CNN runtime rejects"), suggest the
closest viable alternative (e.g. XINT8), and respect the user's choice if they insist —
recording the risk.
If model_analysis.json shows architecture unknown — produce a draft plan with
requires_confirmation: true, list the op histogram, and ask the user which of CNN /
transformer / hybrid path to follow.
If calibration data is unavailable — explain that PTQ requires representative inputs for
accurate scale estimation, and ask the user to supply a small calibration set (≥ a few dozen
samples) before the plan can be finalized.
If a block-format preset is requested but custom-ops compile is unverified — hand off to
quark-onnx-install to verify, then resume.
If the user picks an unusual combination (e.g. CLE + transformer, three or more algorithms)
— explain why it's atypical and suggest the standard recipe, but respect the user's choice
with a recorded risk.
