model-builder

name

model-builder

description

QAI ModelBuilder. Tools and workflows for model conversion, inspection, operator patching, quantization, and inference validation of self-converted models on Qualcomm platform. Use this skill when working with custom ONNX/PyTorch models — export to ONNX, convert to QNN/SNPE DLC, FP16/FP32/INT8 quantization, operator patching, context binary generation, and inference validation of self-built models. NOT for AI Hub prebuilt packages — use model-hub skill instead.

Model Builder

How to use this SKILL (it is a thin dispatch layer):

1. Pass the Boundary Decision gate below first — it decides whether this skill even applies.
2. Use the Routing Table to load the ONE reference / sub-SKILL that matches your step or problem — do NOT read everything up front.
3. Follow the Core Workflow spine; open references/core_workflow.md for per-step commands.
4. The Blocking Conditions and Disciplines below are the only rules you must hold in mind the whole time.
5. Trust the docs: never run commands to re-verify facts already in this file or ${APP_ROOT}\data\config\qairt_env.json (torch/Python versions, tool paths). Don't read script source unless a reference doc is missing the detail (then update that doc).
6. x64 host + user wants to run inference locally on this machine? Read ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md FIRST (before Step 7). If HOST_OS is ARM64, or if inference will go via ADB, skip that file entirely.

---

🚨 Boundary Decision — pass BEFORE activating (MANDATORY GATE)

Answer three questions; if any veto holds, stop and switch skills.

#	Question	YES ->	NO ->
Q1	Does the model already have a prebuilt package on AI Hub (Zipformer, MobileNet, YOLO, …)?	❌ Stop -> model-hub skill	Q2
Q2	Is the file to download/use an AI Hub prebuilt artifact (QNN context binary .bin / .dlc)?	❌ Stop -> model-hub skill	Q3
Q3	Does the user have a custom ONNX/PyTorch model to convert, or to re-quantize/recompile into a custom .bin?	✅ Activate this skill	❓ Intent unclear — confirm first

Switch to model-hub on any of: "download from AI Hub" / "model on AI Hub" / "prebuilt package" / "on-device pre-exported package"; a model name + "download" where it already exists on AI Hub; "QNN_CONTEXT_BINARY" / "QNN_DLC". An AI Hub prebuilt package (.bin/.dlc) is loaded directly by model-hub via qai_appbuilder/QNNContext; this skill only converts/compiles custom ONNX/PyTorch. ("run inference" alone does NOT pick this skill — only when the target is a self-converted model.)

Dispatching a sub-agent to another skill (e.g. model-hub) — MANDATORY: a sub-agent inherits NOTHING from this skill (blank context). Once a vacuum-written prompt sent one recursively scanning C:\/C:\WoS_AI for .bin -> 30-min hang. So: (1) first read the target SKILL.md in full, write the prompt from it; (2) the prompt's first instruction MUST be "read the target SKILL.md in full before acting"; (3) the prompt MUST NOT carry any path/script/toolchain from this skill (run_pipeline.py, qnn-onnx-converter, …) — useless for prebuilt packages and lures a wrong disk scan; (4) reuse the target's Sub-Agent Dispatch Template if it has one.

⚠️ First action after activating (before Step 1): ask the DLC portability question (cross-platform vs SoC-optimised DLC) unless the user's request already signals it. Full wording + CLI mapping + skip-signals -> references/operations_reference.md § DLC Portability Question.

---

🧭 Routing Table — load only the ONE that matches (MANDATORY)

This SKILL is the main pipeline + index. For any specific step, error, or topic, load the single matching doc below — do NOT read them all. Troubleshooting sub-SKILLs are the canonical authority for errors (more complete than any inline summary). base ships in the external edition; advanced is internal-only (silently skip if the file is absent).

By problem / error (troubleshooting sub-SKILLs — canonical):

Symptom / error code	Load	tier
unsupported operator / 0xc26 / Einsum / Mod / Floor / ScatterND / dry-run false positive	${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/operator-patching/SKILL.md	base
Graph Compose failure / graph_names / Wrong number of Parameters 5 / loadRemoteSymbols 4000 / arch mismatch	${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/conversion-troubleshooting/SKILL.md	base
QNNContext crash / stale artifact / multi-model same-process / Linux HTP transport mismatch / NCHW-NHWC wrong	${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/inference-troubleshooting/SKILL.md	base
VCTargetsPath / CMake / import cv2·Pillow / qai_appbuilder import fails	${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/env-troubleshooting/SKILL.md	base
0-byte generator / WinError 193 / need to modify an SDK file	${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/sdk-integrity-recovery/SKILL.md	base
basicsr / functional_tensor / aux-branch ReshapeOp (ONNX export)	${APP_ROOT}/factory/chat_features/model-builder/troubleshooting/export-troubleshooting/SKILL.md	base

By topic (references):

Topic	Load
Core Workflow step details (commands + caveats)	references/core_workflow.md
Operations detail (flow selection, DLC-portability Q, guardrails, working-dir, project config, script index, pack export)	references/operations_reference.md
Environment setup (Windows)	references/win_qairt_setup.md
Export + ONNX validation	references/model_export_validation.md
Operator patching (full code library)	references/operator_patching.md
QNN conversion	references/qnn_conversion.md
SNPE conversion	references/snpe_conversion.md
Quantization (+ tool-param map)	references/model_quantization.md
Context binary	references/context_binary.md
Inference (NCHW/NHWC, API, templates)	references/inference.md
QNN inference routing (per-platform defaults + override keywords)	${APP_ROOT}/factory/chat_features/_shared/qnn-inference-routing.md
x64 host — local inference guide (opt-in; compatibility matrix, backend choice via question tool, B11, closing statement)	${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md
Quantization sensitivity (pre-conversion risk pre-flight)	references/quantization-sensitivity.md
Verification discipline	references/verification-discipline.md
Pack export & inference_manifest.json	references/pack_export.md
ADB device deployment	references/adb_execution.md
Remote (SSH) execution	references/remote_execution.md
Troubleshooting quick-index + Windows tips	references/troubleshooting.md

---

Core Workflow (8-step spine)

Per-step commands, caveats, and MANDATORY sub-requirements -> references/core_workflow.md — open it when you start executing. First (once, before Step 1): run Host OS Detection, write HOST_OS to plan.md (windows-arm64 / windows-x64 / linux-aarch64 / linux-x64; drives Step 3 backend + Step 7 path) -> core_workflow.md § Host OS Detection.

1. Export to ONNX — python_x64_venv, model.eval(), FP32 only (never FP16), opset_version=18; disable training-only branches (aux_logits/dropout). -> core_workflow.md § Step 1 / model_export_validation.md.
2. Inspect ONNX I/O — qai_inspect_onnxio.py. ⚠️ Do NOT gate on --dry_run (false positives) — go straight to Step 4. -> core_workflow.md § Step 2.
3. Operator patching — ONLY if actual conversion hits a hard op error. Patch in-memory, re-validate (checker -> real conversion -> cosine ≥ 0.95). Canonical -> operator-patching sub-SKILL.
4. Convert float model — run_pipeline.py (Flow A, default, all hosts). run_pipeline_legacy.py / qai_convert_fp.py / qai_convert_int.py are Flow C (DLL, windows-arm64 only; error out elsewhere). --precision fp16|fp32, --htp_version v73|v81 (default v73). -> core_workflow.md § Step 4 / qnn_conversion.md.
5. Quantization (optional) — run_pipeline.py --precision <p> --calib_list <list>. ⚠️ Real multi-class calibration data; ask user if none. -> core_workflow.md § Step 5 / model_quantization.md.
6. Context binary — run_pipeline.py emits .bin automatically. The .bin is a QNN context binary for the HTP backend; on ARM64 hosts it targets real HTP. For loading a .bin on an x64 host, see x64-host-notes.md. Portable across HTP backend builds (routing doc §4); use .dlc for cross-backend / late backend choice. -> core_workflow.md § Step 6 / context_binary.md.
7. Inference + validation — route by HOST_OS per ${APP_ROOT}/factory/chat_features/_shared/qnn-inference-routing.md (the routing doc): ARM64 hosts default to local HTP via Path A (qai_runner.py + , ); x64 hosts default to via Path B (), with opt-in local execution when the user asks — see . User override keywords can flip the default (routing doc §3). MANDATORY: save + ; any x64-local-execution run must emit the closing statement (see §4) in the user's language. Per-step details -> / .

Artifact checklist per Flow (A/B/C) -> references/expected_output_artifacts.md. Batch runs -> scripts/model_config.json.

---

🛑 Blocking Conditions (always STOP & ask — both modes)

#	Condition -> Action
B1	Required config var empty/placeholder -> stop, list missing, ask user.
B2	pip install needed -> stop, state package + reason, ask permission.
B3	Patch iterations exhausted, NO progress (same ops, no patterns left) -> stop, list attempts + logs, escalate.
B4	Operator patch would change model semantics -> stop, describe change, ask approval.
B5	Target device unavailable for context-bin gen / on-device test (incl. remote unreachable) -> stop, ask how to proceed.
B6	Accuracy < threshold after quant (cosine < 0.95) -> do NOT auto-fix. ① zero-cost diagnosis (is calibration one image / its augmentations? not diverse). ② STOP, report cosine + diagnosis, present options (each 1-line principle), ask which: (1) improve calib diversity; (2) --cle (+--per_channel); (3) --precision w8a16; (4) keep FP16 / try bf16; (5) accept if Top-K correct. Full flow -> model_quantization.md / quant-accuracy sub-SKILL.
B7	No known replacement pattern for an unsupported operator -> stop, document, escalate.
B8	Context binary gen fails on windows-arm64 / linux-aarch64 (real HTP hosts) -> stop (run_pipeline.py exits non-zero; NOT silently degraded). Return to operator patching; do NOT retry alternate generators (x86_64 build can't load an ARM64 DLL). 0-byte/corrupt generator = damaged SDK file -> qai_dev_gen_contextbin.py self-heals from the kept SDK zip; if none -> sdk-integrity-recovery sub-SKILL. Diagnose READ-ONLY. On x64 hosts see ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md for loading the .bin locally and applicable blocking rules (B11).
B9	Fixing would require modifying any file under $QAIRT_SDK_ROOT/$QNN_SDK_ROOT -> STOP IMMEDIATELY. Never edit/copy-over/rename/delete an SDK file (the C:\Qualcomm tree is tool-layer write-protected). Copy the file into the workspace and edit the copy, pointing tooling at it via documented overrides (--config_file, env, workspace-local ). Reading the SDK dir is fine. Genuinely missing/corrupt -> recover from kept zip (); ask explicitly and act only on a scoped yes naming the file.

---

Disciplines (hold these the whole run)

• Execution mode (MODE in config, default batch): batch = run all phases autonomously, apply safe defaults, log decisions, only stop on a Blocking Condition — do NOT ask "proceed to next phase?" / "which precision?" (use config) / "run onnxsim?" (always). interactive = confirm at each phase. Never silently fall back to ONNX/CPU when QNN/HTP fails — diagnose & fix, or stop & report; substituting CPU for a failed HTP run is never an acceptable fix.
• Inference results MUST come from actual execution. Never output Top-K/confidence/latency/cosine without first running the script via exec; every number traces to an exec log line. No guessing/estimating from model knowledge; no writing the report before running.
• Operator patching is exhaustive — patch ALL unsupported ops until no pattern remains; never fall back to CPU; no fixed iteration cap; escalate only on B7/B4/B3. Rules + code -> references/operator_patching.md.
• Working directory: all model artifacts under ${WORKSPACE}\<model_name>\ — NEVER under a QAIModelBuilder path, home/Downloads, or a CWD outside ${WORKSPACE}. Self-check every write. Bootstrap with qai_workspace_init.py. Tables + init diagnosis -> references/operations_reference.md § Working Directory.
• Wrappers only: conversion via run_pipeline.py (run_pipeline_legacy.py = Flow C, windows-arm64 only); inference via qai_runner.py/qai_appbuilder (never qnn-net-run). Wrappers handle --preserve_io, layout, PYTHONPATH, arch dirs, and host_arch routing.
• Trust known facts; never re-verify via shell (torch=2.x, Python x64=3.10/ARM64=3.13, tool paths — all in qairt_env.json). Timeouts: timeout=0 for all conversion commands. Benign HTP errors, os._exit crash, encoding, escalation, SDK read-only rules -> references/operations_reference.md § Guardrails.

---

⚠️ Python Environments (not interchangeable — gates B10)

Paths from ${APP_ROOT}\data\config\qairt_env.json (Setup.bat generates it). Never hardcode.

Env	Key	Python	Role
Conversion	python_x64_venv	x86_64 3.10	ONNX export, qairt-converter, qairt-quantizer, qnn-onnx-converter, qnn-model-lib-generator (all hosts).
Runtime	python_runtime_venv (fallback: python_arm64_venv)	3.13 — aarch64 on WoS (.venv_arm64_313), x86_64 on x64 Windows (.venv_x64_313)	qai_appbuilder, QNNContext, inference.
Ubuntu	python3_venv	x86_64 3.12	All Ubuntu ops (no ARM64 venv on Ubuntu).

Default for tools not listed here: python_x64_venv (most QAIRT tools link python310.dll); switch to python_runtime_venv only when the tool imports qai_appbuilder/QNNContext or runs inference on a .bin/.dlc; unsure -> B10. On x64 hosts, if the user asks to run inference locally, use python_runtime_venv (resolves to .venv_x64_313) and see ${APP_ROOT}/factory/chat_features/_shared/x64-host-notes.md for backend selection. Setup / pip / --index-url / opencv / PYTHONPATH -> references/win_qairt_setup.md; env broken -> env-troubleshooting sub-SKILL.