quark-onnx-debug

name	quark-onnx-debug
description	Diagnose failed Quark ONNX installation, calibration, quantization, custom-op compilation, or export attempts. Use when the user reports an error, stack trace, invalid artifact, missing dependency, ORT execution-provider mismatch, silent CPU fallback, OOM during calibration, custom-op load failure (BFPQuantizeDequantize / MXQuantizeDequantize / Extended*), or unexpected quantization results from the ONNX flow. Trigger for "Quark ONNX error", "onnxruntime error", "quantize_static failed", "calibration crashed", "CUDAExecutionProvider not available", "ROCMExecutionProvider not available", "custom op library load failed", "model.onnx larger than 2GB", "external data not found", "AdaRound diverged", "GPTQ ONNX failed", "QuaRot failed", "NPU power-of-2 scale", any Python traceback mentioning quark.onnx / onnxruntime / onnx, or when the user pastes an error message related to Quark ONNX workflows.
layer	l1-atomic
primary_artifact	validation_report.md
source_knowledge	["quark/onnx/quantization/quantize.py","quark/onnx/quantization/api.py","quark/onnx/quantization/input_check.py","quark/onnx/calibration/calibrators.py","quark/onnx/operators/custom_ops/build_custom_ops.py","quark/onnx/quantizers/registry.py","docs/source/install.rst","docs/source/onnx/basic_usage_onnx.rst","docs/source/onnx/gpu_usage_guide.rst"]

quark-onnx-debug

Purpose

Convert ONNX-flow failures into a structured diagnostic report with the smallest safe recovery path. Debugging Quark ONNX issues is tricky because errors can originate from many layers — Python environment, the onnx package version, the installed onnxruntime* variant, custom-op compilation, CUDA / ROCm / NPU EPs, calibration data plumbing, graph optimization, or the chosen quantization config itself. This skill systematically narrows down the root cause.

Inputs

• Error message and stack trace from a failing run
• The exact command or ModelQuantizer / quantize_static invocation that triggered it
• env_context.json, onnx_install_result.json, quark_install_result.json (optional, for environment and install state)
• The model path / size and (if relevant) the QuantizationConfig used

Outputs: validation_report.md

Diagnostic report with root cause, evidence, and the smallest safe fix.

Schema: validation_report.schema.json

# Debug Report

## Common Error Patterns

### Installation Errors

| Error | Likely Cause | Fix |
|-------|-------------|-----|
| `ModuleNotFoundError: No module named 'quark'` | Quark not installed or wrong Python env | `pip install amd-quark` or activate correct conda env |
| `ModuleNotFoundError: No module named 'onnxruntime'` | ONNX Runtime not installed | Hand off to `quark-onnx-install` |
| `ModuleNotFoundError: No module named 'onnx'` | `onnx` package not installed | `pip install "onnx>=1.21.0,<=1.22.0"` |
| `ImportError: quark.onnx.operators.custom_ops` / compile failure | Missing C++ compiler (Linux `g++`, Windows VS 2022) or `ROCM_PATH`/`CUDA_HOME` unset for GPU build | `apt install g++` (Linux), install VS 2022 (Windows), `export ROCM_PATH=/opt/rocm` or `export CUDA_HOME=/usr/local/cuda` |
| Both `onnxruntime` and `onnxruntime-gpu` (or `_rocm`) installed | Variant collision — wrong EP loads | `pip uninstall -y onnxruntime onnxruntime-gpu onnxruntime_rocm`, then reinstall the single correct variant via `quark-onnx-install` |
| `onnx` schema / `opset` errors at import | `onnx` outside `>=1.16.0,<=1.19.0` | Pin within the supported range |
| `onnxruntime` ABI / symbol errors when loading custom ops | ORT version outside `>=1.22.2,<=1.24.2` (custom-ops built against a different ABI) | Reinstall ORT within the supported range, then re-trigger custom-ops compile |

### ONNX Runtime / Execution Provider Errors

| Error | Likely Cause | Fix |
|-------|-------------|-----|
| `get_available_providers()` lacks `CUDAExecutionProvider` | CPU `onnxruntime` installed on a CUDA box | Reinstall `onnxruntime-gpu` via `quark-onnx-install` |
|  lacks  on ROCm 6.x | Wrong variant (need  from AMD Artifactory) | Reinstall  via  |
|  missing on ROCm 7.x |  — ROCm 7.x falls back to CPU  per  (build incompatibility) | Document the CPU-only fallback; do not attempt a ROCm wheel install |
| Silent CPU fallback (no GPU utilization during calibration) | EP not requested or unavailable | Pass  /  explicitly; verify with  |
|  | CUDA toolkit version mismatch with  build | Match  to the system CUDA major (CUDA 11 → Azure DevOps index; CUDA 12/13 → pypi default) |
|  | ROCm driver / library mismatch | Verify  matches the wheel's ROCm version |



| Error | Likely Cause | Fix |
|-------|-------------|-----|
|  | Custom-ops library was not compiled, or compiled against a different ORT version | Re-run  and inspect the compile output |
|  | The session was created without registering Quark's custom-op library | Use  / , or pass the custom-op / via  |
| Custom-op symbol-not-found on GPU but works on CPU | GPU kernel of the custom op was not built (missing  /  at compile time) | Set the env var, delete the cached /, re-import to recompile |



| Error | Likely Cause | Fix |
|-------|-------------|-----|
|  () | Wrong  path | Check absolute path to  |
|  | Model fails ONNX schema check | Run  to localize; rebuild the model with a compatible opset |
|  / ProtoBuf size error | Model > 2 GB without external data | Save with ; pass  to the quantizer |
|  |  moved but  / weight blobs left behind | Move the model directory as a whole, or re-export with external data adjacent |
| Shape-inference failure | Incomplete shapes in the model | Run  /  before quantization |



| Error | Likely Cause | Fix |
|-------|-------------|-----|
|  () | Custom  missing  | Implement  returning sample count |
|  () | Data reader yielded zero batches before exhaustion | Check reader's  returns at least one batch |
|  () | Custom calibrator subclass returned wrong type | Return  |
|  |  never called before  | Call  first, or use  which handles the order |
|  () |  with out-of-range constant | Use a value in , typically  |
|  () | Typo or unsupported  | Check  enum in  |
| OOM during calibration | Activation cache too large | Set  (disk cache) and/or ; reduce  if memory bound; reduce calibration sample count |



| Error | Likely Cause | Fix |
|-------|-------------|-----|
|  () | INT16/UINT16/INT32 with  | Switch to  |
|  () | AdaRound/AdaQuant + INT4 weights | Disable fast-finetune for INT4, or use GPTQ instead |
|  () | Tensor name in  not in the graph | Verify tensor name with  |
|  () | Per-channel override on a quant type that doesn't support it (e.g., FP types, block formats) | Remove per-channel from that override |
|  () |  with a file path | Load model first:  |
|  () | Passed a dict or wrong type | Build a  or  |
|  () | Typo in  preset name | Pick from  (XINT8, S8S8todevice='cuda:0'CUDAExecutionProvideroptimrateCLE` pre-processing first |
| GPTQ shape mismatch | Group-size doesn't divide the weight dimension | Pick a  that divides the weight's reduction dim (commonly 32, 64, 128) |
| SmoothQuant alpha range |  outside  | Use a value in  for LLMs |
| QuaRot rotation config invalid | Missing rotation pair definitions | Provide the rotation pairs in the algorithm config; see  |
|  () | Search invoked without a model | Pass the model path or  |
|  () | Search invoked without calibration data | Provide a  |
|  () | Sampler typo | Use  or  |



| Error | Likely Cause | Fix |
|-------|-------------|-----|
| Power-of-2 scale validation failure | Non-PoF2 scale produced for an NPU target | Use a PoF2 calibrator ( MinMSE / NonOverflow); confirm  |
|  but model has unsupported ops | NPU CNN backend doesn't support the op | Fold / replace via ; see  |
| NCHW vs NHWC mismatch | NPU expects NHWC but model is NCHW | Run  before quantization |
|  deprecated warning | Old API | Switch to  |



| Error | Likely Cause | Fix |
|-------|-------------|-----|
|  on output dir | No write permission | Check output directory permissions |
| Quantized model > 2 GB and external data not written |  on a large model | Set  |
| Inference using the quantized model errors on a custom op | Deployment env missing Quark's custom-ops library | Ship the compiled / and register via  |



  — capture the exact error message, full stack trace, and the command or
    /  call that triggered it.
  — installation / ORT runtime / custom-op / model loading / calibration /
   config validation / algorithm / NPU-specific / export.
  — Python version,  version, installed  variants
   and versions, available EPs, Quark version, accelerator (CUDA major / ROCm major), whether the
   custom-ops library compiled.
  — use the tables above.
  — smallest change that resolves the issue without side effects.







  Partial errors lead to wrong
  diagnoses. For ONNX, also ask for: the  / preset name, the model path and
  size, the chosen , and the calibration data reader class.
  Narrow down the root cause first, then propose a specific solution.
  Both cause silent CPU
  fallback. Check   the providers actually passed to the
  session.
  If the fix involves reinstalling , rebuilding the
  custom-ops library, or re-running an hour-long calibration, present the plan and get
  confirmation.
  Bumping  may force a custom-ops rebuild and may break
  ABI with an older  pin. Bumping  outside  may break Quark's QDQ insertion.
  Always check version constraints (, ).
  Do not propose a "fix" that downgrades to
  ROCm 6.x unless the user asks; instead document the CPU-fallback trade-off (see
  ).





`onnxruntime.capi.onnxruntimestate.RuntimeException: ... CUDAExecutionProvider is not in
the list of available providersModelQuantizer.quantizeavailableavailablerocm
pip install --no-cache-dir onnxruntime-gpu
python -c "import onnxruntime as ort; print(ort.getproviders())"   # expect CUDAExecutionProvider present
python -c "import quark.onnx.operators.custom

Interaction Flow

1. Gather evidence: error message, stack trace, exact call, model size, config, EPs.
2. Classify: install / custom-op / ORT runtime / model / calibration / config / algorithm / NPU / export.
3. Diagnose: match against known patterns and run diagnostic commands if needed.
4. Propose fix: present the smallest change that resolves the issue.
5. Confirm: get user approval before reinstalls, custom-ops rebuilds, or long recalibrations.
6. Verify: after the fix, re-run the failing step (or a smaller smoke variant) to confirm resolution.

Recovery

• If the fix requires an onnxruntime* or onnx package change (wrong variant, version mismatch, missing variant), hand off to quark-onnx-install with the specific requirement noted.
• If the fix requires an amd-quark package or shared-dependency change, hand off to quark-install with the specific requirement noted.
• If the fix requires a C++ compiler install or ROCM_PATH/CUDA_HOME setup, surface the environment gap and let the user resolve it before re-running custom-ops compile.
• If the root cause is upstream (ORT API change, onnx schema change), hand off to quark-doc-drift-check or quark-skill-sync.
• If the error is in a custom config or unsupported op for an NPU target, suggest the relevant pre-processing tool from quark.onnx.tools (e.g. convert_nchw_to_nhwc, convert_qdq_to_qop, convert_a8w8_npu_to_a8w8_cpu).
• If the issue cannot be reproduced from the supplied evidence, ask for the minimum repro (model, calibration data sample, config) before guessing.