| name | model-management |
| description | Explains how model files, checkpoints, GGUF quantization, and the Model Manager work in HOT-Step CPP. Use when adding/converting/quantizing models, debugging "no GGUF models found" or missing-model/wrong-model failures, working on the model download service or Model Manager UI, or answering which component (LM/DiT/VAE/text-encoder) needs which file. |
Model Management — Models, Checkpoints & Quantization
All paths are repo-relative to the repo root (d:\Ace-Step-Latest\hot-step-cpp). All commands are Windows PowerShell (use ; to chain, never &&).
Terminology (read first)
- LM — the language model (Qwen3-based) that turns caption+lyrics text into audio codes. Files:
acestep-5Hz-lm-{0.6B,1.7B,4B}-*.gguf.
- DiT — Diffusion Transformer, the denoising model that generates audio latents. Files:
acestep-v15-*.gguf. One DiT GGUF is self-contained (also carries the condition encoder, FSQ tokenizer/detokenizer, silence_latent, null_condition_emb).
- Text encoder — Qwen3 embedding model that encodes the caption for the DiT. Files:
Qwen3-Embedding-0.6B-*.gguf.
- VAE — decodes latents to 48 kHz audio (and encodes audio to latents for cover/repaint/extend). Files:
vae*.gguf / scragvae*.gguf.
- PP-VAE — optional post-processing VAE ("polish" re-encode pass). Files:
pp-vae-*.gguf.
- GGUF — the GGML binary weight format the C++ engine loads (mmap'd). Each GGUF declares its role in the header key
general.architecture.
- Quant — reduced-precision weight encoding (Q4_K_M, Q8_0, MXFP4, ...) to shrink VRAM/disk. Produced from BF16 GGUFs by
engine/tools/quantize.cpp.
- Adapter — LoRA/LoKr fine-tune delta (
.safetensors), lives in adapters/, not models/.
- Model Manager — the "Get More Models" modal in the UI + Node download service that fetches curated files from HuggingFace.
When to use this skill
- Installing, converting, or quantizing model files; deciding where a file must live.
- Debugging: engine won't start,
/synth unavailable, model missing from dropdowns, crash-loops, corrupt downloads.
- Modifying the Model Manager (route
server/src/routes/modelManager.ts, service server/src/services/modelDownloadService.ts, UI ui/src/components/model-manager/), or the catalogue server/src/data/model-registry.json.
Golden rules
- Two independent registries exist — don't confuse them. The Node-side curated catalogue (
server/src/data/model-registry.json) controls what is downloadable; the C++ engine's startup scan (engine/src/model-registry.h) controls what is usable. A file can show "installed" in the Model Manager yet be invisible to generation dropdowns (and vice versa). WHY: they use different detection logic (filename presence vs GGUF-header architecture) and different directory depths.
- GGUF files must sit in the models root, not a subfolder. The engine scans only root-level
.gguf files (engine/src/model-registry.h:234-267), but the Node installed-check also scans one subdir level (modelDownloadService.ts:177-208). A GGUF in a subfolder = "installed" in the UI, dead to the engine. WHY: silent classic confusion — no error anywhere.
- Never hand-edit or hand-build a GGUF's tensor set for a recognized architecture. A recognized-arch GGUF with a missing tensor kills the whole ace-server process:
gf_load_tensor() prints [GGUF] FATAL: tensor 'x' not found and calls exit(1) (engine/src/gguf-weights.h:164-168). Node respawns it, so a bad model file mid-request looks like a random engine crash-loop.
- Quantize from BF16 sources only, and never quantize the VAE.
quantize.exe reads BF16 GGUF input; VAE-arch tensors and small/critical tensors (silence_latent, scale_shift_table, null_condition_emb, 1-D tensors, text-enc embed_tokens) are deliberately never quantized (engine/tools/quantize.cpp:89-109). WHY: quantizing these destroys audio quality or breaks generation outright.
- Rebuild rules apply here too: any change to
engine/src/model-registry.h, model-store.h, gguf-weights.h, etc. means rebuild via dev-rebuild.bat at repo root — never engine/build.cmd directly (you cannot reliably tell whether the app is running; Node auto-respawns ace-server, and killing it uncleanly causes an infinite respawn + file-lock loop). Never cmake --clean-first (20+ min CUDA recompile).
- Runtime DLLs do not go in
models/. Catalogue entries with role: "runtime" (cuBLAS, cudart, ONNX Runtime, cuDNN) install next to ace-server.exe (modelDownloadService.ts:110-118). WHY: missing DLLs there are the #1 cause of the "crashed 3 times within 30s — giving up" loop.
- Model/adapters directory paths are restart-required config.
ACESTEPCPP_MODELS / ACESTEPCPP_ADAPTERS env vars override defaults <repo>/models and <repo>/adapters (server/src/config.ts:53-54,100-101); the engine gets them as spawn-time --models/--adapters flags.
Directory layout (expected)
models/ # config.aceServer.models (default <repo>/models)
acestep-v15-*.gguf # DiT (arch "acestep-dit"): base/sft/turbo/merge x BF16/Q8_0/Q6_K/Q5_K_M/Q4_K_M/MXFP4/NVFP4...
acestep-5Hz-lm-{0.6B,1.7B,4B}-*.gguf # LM (arch "acestep-lm")
Qwen3-Embedding-0.6B-*.gguf # Text encoder (arch "acestep-text-enc")
vae-*.gguf, scragvae-*.gguf # VAE (arch "acestep-vae"); ScragVAE/Regrind = drop-in decoder variants
pp-vae-*.gguf # PP-VAE (arch "pp-vae")
vae-*.safetensors # safetensors VAE, classified by filename prefix only
vae-*.onnx # ONNX VAE — DECODER-ONLY (see failure table)
<name>/ # HF safetensors checkpoint dir: config.json + model.safetensors
# (or model.safetensors.index.json for sharded) — classified by config.json content
onnx/ # ONNX Runtime / TensorRT model dirs (config.aceServer.onnxDir)
supersep/*.onnx # stem-separation nets (Cover/Stem Studio)
whisper/ggml-*.bin # whisper.cpp models (config.ts:253-255)
adapters/ # config.aceServer.adapters
<name>.safetensors # ComfyUI single-file LoRA (alpha baked in)
<name>/adapter_model.safetensors # PEFT directory format
Which component needs which model
Verified in engine/src/model-store.h:68-81 (ModelKind comments):
| Component | Model file | Notes |
|---|
LM (MODEL_LM) | acestep-5Hz-lm-*.gguf | ONE shared instance for generate + ace-understand — enforced by identical ModelKey (model-store.h:17-22) |
Text encoder (MODEL_TEXT_ENC) | Qwen3-Embedding-*.gguf | |
| Cond-enc + DiT + FSQ tok/detok | the same acestep-v15-*.gguf | Self-contained; also holds silence_latent + null_condition_emb (model-store.h:112-118) |
| VAE encode + decode | vae*.gguf (has encoder.* and decoder.*) | |
| PP-VAE polish | pp-vae-*.gguf | Request flag pp_vae_reencode (engine/src/request.h:146); availability = GET /api/models/pp-vae scans for pp-vae*.gguf (server/src/routes/models.ts:50-65) |
| ORT/TRT acceleration | models/onnx/ subdirs | MODEL_*_ORT kinds |
| SuperSep stems | models/supersep/*.onnx | + ONNX Runtime/cuDNN DLLs beside ace-server.exe |
| Whisper transcription | models/whisper/ggml-*.bin | + tools/whisper/whisper-cli.exe (config.ts:254-255) |
Synth pipeline needs DiT + Text-Enc + VAE simultaneously. Missing any one → /synth unavailable warning (server stays up LM-only if an LM exists), or exit 1 if no LM either (engine/tools/hot-step-server.cpp:2600-2616 — the compiled server; the same block exists in the UNCOMPILED reference copy ace-server.cpp:1686-1711). Request-level selection: synth_model, lm_model, vae are filenames resolved against the engine's scanned registry; empty string = first matching entry (engine/src/request.h:125-142).
Procedure: quantize a BF16 GGUF
# From repo root. Binary lives at engine\build\Release\quantize.exe
.\engine\build\Release\quantize.exe <input-BF16.gguf> <output.gguf> <TYPE>
# Example:
.\engine\build\Release\quantize.exe models\acestep-v15-turbo-BF16.gguf models\acestep-v15-turbo-Q4_K_M.gguf Q4_K_M
- Valid TYPEs (case-insensitive,
quantize.cpp:8): Q2_K Q3_K_S Q3_K_M Q3_K_L Q4_K_S Q4_K_M Q5_K_S Q5_K_M Q6_K Q8_0 NVFP4 MXFP4. IQ3/IQ4 quants seen on disk are not producible by this tool.
- Mixed-precision policy mirrors llama-quantize: "important" tensors (
v_proj, down_proj; L variants add o_proj) bumped one tier; embed_tokens always Q6_K (Q8_0 for Q8_0/NVFP4/MXFP4) (quantize.cpp:40-54,74-86).
- Streaming write, low memory. Prints
Quantized N/M tensors + compression ratio.
- Output goes straight into
models\ root → picked up on next engine restart.
Procedure: convert HF safetensors → BF16 GGUF (engine/convert.py)
- No CLI args. Hardcoded: reads checkpoint dirs from
engine/checkpoints/, writes GGUFs to engine/models/ (convert.py:14-16). Neither directory exists in this working tree — create engine\checkpoints\, put the HF checkpoint dir inside, run it, then move the output GGUF to repo-root models\.
- Classification is by checkpoint directory name:
acestep-5Hz-lm* → LM, acestep-v15* → DiT, Qwen3-Embedding* → text-enc, and exactly vae → VAE (convert.py:55-64). Skips outputs that already exist.
- Alternative: the engine loads safetensors checkpoint dirs directly (drop
<name>/ with config.json + model.safetensors into models/) — conversion is optional. Sharded (model.safetensors.index.json) and diffusers (diffusion_pytorch_model.safetensors) layouts supported (engine/src/weight-source.h, engine/src/model-registry.h:328-375).
Procedure: convert ComfyUI int8 DiT safetensors → Q8_0 GGUF (engine/convert-comfy-int8.py)
For ComfyUI comfy_quant int8 DiT checkpoints (int8 .weight + F32 .weight_scale scalar or per-row + .comfy_quant JSON tensor per layer), including ConvRot files ("convrot": true + convrot_groupsize). Per-tensor and per-row int8 grids are exactly representable in Q8_0 (block scale = tensor/row scale), so weights are repacked bit-faithfully — no dequant/requant round trip. Needs a donor GGUF of the same architecture (any convert.py-produced acestep-v15-*.gguf) to supply silence_latent and the acestep.* config KVs, which ComfyUI files lack. Aborts on any tensor-shape mismatch vs the donor.
python engine\convert-comfy-int8.py <comfy.safetensors> models\<matching-donor>-BF16.gguf models\<out>-Q8_0.gguf --name <general.name>
ConvRot handling: rotated decoder.* weights stay rotated and are recorded in GGUF KV acestep.convrot_map (name:group;...); the engine applies the matching group-wise Hadamard rotation to that linear's activations at inference (dit.h load + dit-graph.h/dit-alignment-graph.h, commit 182faef). Rotated encoder/tokenizer/detokenizer weights are dequantized + unrotated to BF16 offline (run once per generation — not worth graph wiring). --no-runtime-rotation builds an all-BF16 unrotated reference GGUF of the same quantized model, used for same-seed A/B validation of the engine rotation path. Adapter merge mode is refused on ConvRot models (deltas are unrotated); runtime adapter mode works (deltas consume raw activations).
ConvRot cardinal rule — any code reading ConvRot base weights for unrotated-space math must unrotate them first (convrot_transform_rows in engine/src/convrot.h, fast radix-4, self-inverse). Violation signature: generation "succeeds" but output is garbled full-band noise. First instance: the runtime basin re-base nudged deltas with β·(S−T) using rotated T — fixed in adapter_runtime_rebase (commit 22820ae) by unrotating T for acestep.convrot_map tensors. Audit any future weight-reader (TRT export, distills, external merge scripts) against this.
Producing ConvRot files from a local checkpoint: pip install convert_to_quant (needs torch+CUDA, triton-windows) then
ctq -i <model.safetensors> -o <out.safetensors> --int8 --scaling_mode row --convrot --dynamic_convrot --comfy_quant --save-quant-metadata (~35 min for a 5B XL on an RTX 5090, learned rounding included).
First applied 2026-07-15: hrktxz xl_sft_turbo (plain int8) → acestep-v15-xl-sft-turbo-comfy-int8-Q8_0.gguf; merge-base-sft-turbo-xl-thirds self-quantized with real ConvRot → ...-convrot-Q8_0.gguf (+ ...-convrot-ref-BF16.gguf reference). Numerical parity: rotated-path error 0.9% vs original F32 weights; skipping rotation → ~140% (i.e. rotation is load-bearing).
Procedure: add a model manually
- Copy the
.gguf into models\ root (not a subfolder — golden rule 2).
- Restart the engine (
dev-rebuild.bat restarts everything, or restart the app). The scan runs only at ace-server startup.
- Check the newest
logs\<session>\ace_engine.log for [Registry] <file> -> DiT (or LM/VAE/...). A WARNING: skipping X (unknown architecture) means the GGUF header lacks a recognized general.architecture (acestep-lm|acestep-dit|acestep-text-enc|acestep-vae|pp-vae, model-registry.h:99-130).
- The file now appears in
GET /api/models (Node proxies engine /props; buckets lm, embedding, dit, vae — hot-step-server.cpp:2326-2329 — the compiled server, not the uncompiled ace-server.cpp).
Procedure: drive the Model Manager via API
Routes in server/src/routes/modelManager.ts, mounted at /api/model-manager:
Invoke-RestMethod http://localhost:3001/api/model-manager/registry # catalogue + installed flags
Invoke-RestMethod -Method Post -Uri http://localhost:3001/api/model-manager/download -ContentType 'application/json' -Body '{"fileId":"<id>"}'
# GET /downloads = SSE progress stream; POST /download/<jobId>/cancel | /resume; DELETE /files/<filename>
Download mechanics: HuggingFace URL https://huggingface.co/{repo}/resolve/main/{repoPath || filename}, resume via HTTP Range + .part file, 3 attempts (0/2s/5s), validation before rename (size ±5%, MZ header for .dll, GGUF magic for .gguf) — modelDownloadService.ts:352-473. Details and data shapes: reference.md.
Key files
| Path | Role |
|---|
engine/src/model-registry.h | Engine startup scan/classification of --models and --adapters dirs |
engine/src/model-store.h | Refcounted VRAM ownership; EVICT_STRICT (default) vs EVICT_NEVER (--keep-loaded); ModelKey caching incl. adapter extras |
engine/src/gguf-weights.h | mmap GGUF loader; truncation guard; FATAL exit on missing tensor |
engine/src/safetensors.h, engine/src/weight-source.h | safetensors parser + format-agnostic layer (GGUF/safetensors) |
engine/tools/quantize.cpp → engine/build/Release/quantize.exe | BF16 GGUF → K-quant/FP4 GGUF |
engine/convert.py | HF safetensors checkpoint dir → BF16 GGUF (hardcoded dirs) |
engine/tools/ace-server.cpp | Startup validation, /props endpoint |
server/src/config.ts | aceServer.models/adapters/onnxDir, keepLoaded, warm-on-startup, whisper paths |
server/src/services/modelDownloadService.ts | Download jobs, resume, validation, installed-check, variant filtering |
server/src/routes/modelManager.ts | /api/model-manager/* REST + SSE |
server/src/routes/models.ts | /api/models (proxies engine /props), /api/models/pp-vae |
server/src/data/model-registry.json | Curated catalogue: 132 files, 8 packs |
server/src/index.ts | ace-server spawn/respawn limiter (152-156, 284-308); first-launch CUDA DLL bootstrap (318-380) |
ui/src/components/model-manager/ | Modal UI: ModelManagerModal.tsx, ModelCatalogueTab.tsx (7 tabs), ModelRow.tsx, StarterPackCard.tsx, DownloadProgressBar.tsx, useModelRegistry.ts, useDownloadStream.ts |
Failure signatures
| Symptom | Cause | Fix |
|---|
[Server] ERROR: no GGUF models found + engine exit 1; Node retries 3x then gives up | Empty/wrong models dir (ACESTEPCPP_MODELS), or nothing classifiable | Point at the right dir / install models; restart |
[Registry] WARNING: skipping X (unknown architecture) | GGUF header lacks a recognized general.architecture | Convert via convert.py, or it's not an ACE-Step GGUF |
[Server] WARNING: /synth unavailable, missing: VAE (etc.) | Partial install — synth needs DiT+Text-Enc+VAE together | Download the missing role (Model Manager quick-start pack) |
[GGUF] FATAL: '<f>' is truncated or corrupt ... file is only N bytes | Interrupted download / prematurely renamed .part | Delete and re-download |
[GGUF] FATAL: tensor 'x' not found then process death | Recognized arch, wrong/incomplete tensor set — kills ace-server mid-request | Remove the bad GGUF |
Download "completes" then Invalid GGUF header — got "<!DO" | HuggingFace served an HTML error page (auth/rate-limit/404) | Retry; check the repo/path in the catalogue entry |
Size mismatch: expected X MB, got Y MB | Catalogue sizeBytes drift vs repo file, or corrupt transfer | Re-download; fix sizeBytes in model-registry.json if repo file changed |
| Crash-loop "3 times within 30s" + missing-DLL hint | cuBLAS/cudart DLLs absent beside ace-server.exe (CUDA variant) | Model Manager "CUDA Runtime" pack; first-launch bootstrap normally handles it |
| Model shows "installed" in Model Manager but absent from generation dropdowns | GGUF in a subdir (Node scans subdirs, engine scans root only), unknown arch, or engine down (aceServerDown: true) | Move to models root / check engine log |
| Cover/repaint fails while text2music works, ONNX VAE selected | ONNX VAEs are decoder-only; VAE encode requires a non-ONNX VAE (registry_find_non_onnx, model-registry.h:62-85) | Install a GGUF/safetensors VAE alongside |
| Download of DreamVAE / Regrind entries fails instantly | Those catalogue entries have no repo field → URL contains undefined | Treat as local/display-only entries (see below) |
Institutional knowledge
- VALIDATED — subdir-GGUF blind spot: engine scans root-only for
.gguf; Node installed-check scans one subdir level. This mismatch is a real, recurring "installed but not selectable" source (code cited in golden rule 2).
- VALIDATED — truncation guard exists for a reason: the byte-range check in
gf_load() (gguf-weights.h:132-150) was added specifically because truncated downloads used to segfault deep in cuMemcpyHtoDAsync. Keep it if touching the loader.
- VALIDATED — catalogue entries without
repo: vae-dreamvae-onnx, vae-regrind-v9b-bf16, vae-regrind-v9b-blend50-bf16 in model-registry.json have no repo field, so startDownload builds https://huggingface.co/undefined/... and fails. They exist locally on the dev machine; display-only.
- VALIDATED — installed-check is name-only: filename presence, no size/hash (
modelDownloadService.ts:177-208). A stale/partial file with the right name shows "installed".
- VALIDATED —
--keep-loaded trade-off: default EVICT_STRICT reloads models per request; ACESTEPCPP_KEEP_LOADED=1 → --keep-loaded (EVICT_NEVER) avoids ~17 s LoKr adapter precompute per request but pins ~13 GB VRAM (config.ts:124-132, index.ts:177-184). Warm-on-startup env vars (ACESTEPCPP_WARM_DIT/_VAE/_ADAPTER) only fire when keepLoaded is on.
- VALIDATED — DiT instances cache per adapter combo: ModelKey includes adapter path/scale, per-group scales, basin re-base (
rebase_source/rebase_beta), and multi-adapter adapter_stack signature (model-store.h:83-103) — different values = distinct cached DiTs, each costing VRAM under keep-loaded.
- VALIDATED —
/api/models fallback shape is not a faithful mirror: when the engine is down, Node returns buckets { dit, lm, vae, understand } + aceServerDown: true (models.ts:28-36), but the live engine /props sends { lm, embedding, dit, vae } (ace-server.cpp:1512-1515) — no understand bucket exists, and the fallback lacks embedding.
- VALIDATED —
cleanupJobs() doc-drift: its comment says "older than 60s" but it deletes all terminal jobs immediately, and no route calls it (modelDownloadService.ts:341-348).
- VALIDATED — speculative-decoding draft LM is DISABLED:
ACESTEPCPP_DRAFT_LM plumbing remains (config.ts:117-122) but GGML per-call overhead negated the speedup; auto-detect commented out.
- VALIDATED — "convrot" HF uploads may be mislabeled plain int8: all files in
hrktxz/ACE_Step_1.5_ComfyUI_int8_convrot (checked 2026-07-15) contain only {"format": "int8_tensorwise"} per-tensor-scale layers — no Hadamard rotation metadata anywhere, despite the repo name. Real ConvRot (QuaRot-style group-wise Hadamard, ComfyUI ≥0.27) would need "convrot": true + convrot_groupsize in the .comfy_quant JSON and runtime activation rotation. Our vendored GGML already ships FWHT kernels (CUDA/Vulkan/CPU) + GGML_HINT_SRC0_IS_HADAMARD (ggml.h:444, unused by engine code) if we ever want native support. Check the safetensors header before believing a quant-format claim.
- UNVALIDATED: provenance of on-disk IQ3/IQ4 quant files (not producible by quantize.cpp — presumably made with external tooling); whether
neural-codec/mp3-codec binaries take model paths (not inspected).
Deeper reading
- reference.md (this folder) — engine scan order/classification details, ModelKey/VRAM policy, download-service internals, catalogue statistics, API data shapes, quantize policy table.
engine/docs/ARCHITECTURE.md — engine internals, request JSON, generation modes (committed).README.md — install/build; states safetensors DiT/LM/Text-Enc/VAE all loadable and BF16 safetensors bit-perfect vs BF16 GGUF.docs/plans/2026-05-03-model-manager-design.md — original Model Manager design (gitignored, local-only, may be absent; code has drifted from it — trust code).
