| name | rdk-model-zoo |
| description | Run a ready-made, officially pre-compiled BPU model from the RDK Model Zoo on a board — pick the right branch (branch = board), download the matching .bin/.hbm, run the sample, read the per-board benchmark (latency/FPS/accuracy). Use whenever the user wants a precompiled model instead of quantizing their own, asks "does RDK have a converted YOLO/classification/segmentation/OCR .bin/.hbm", "how do I run a Model Zoo sample", "which branch for my board", or "where do I download the precompiled model". 触发词:Model Zoo、现成模型、预编译模型、官方转好的、有没有现成的 bin/hbm、模型仓、跑示例 sample、哪个分支、archive.d-robotics 下载、benchmark 帧率精度、YOLO11 哪块板能跑、模型性能对比。Routing — quantizing your OWN .pt/.onnx through the toolchain → rdk-device; wrapping a model as a TROS/ROS2 node → rdk-ros; conversational LLM/VLM (InternVL/SmolVLM chat) → rdk-llm-deployment; "can my board run model X" / model selection → rdk-ecosystem; embodied ACT/VLA/Pi0 policies → rdk-embodied-lerobot. |
RDK Model Zoo — Ready-Made BPU Models
The Model Zoo is the official collection of out-of-the-box, pre-compiled BPU models plus full-link conversion tutorials. The single most important fact: the branch you clone IS your board. Cloning the wrong branch is the #1 failure — the model artifact or the runtime API will not match your hardware.
Sources: official D-Robotics repos verified for this skill — rdk_model_zoo (rdk_x5 / rdk_x3 / rdk_s branches), rdk_model_zoo_s (s100 archive), and model_zoo_doc appendix benchmarks. Facts carry provenance; nothing is invented.
The one rule that matters most
Branch = board. Confirm the board first (cat /sys/class/socinfo/board_id, or rdkos_info for the full OS/board summary), then clone exactly that branch. A .bin (Bayes / Bernoulli2) and a .hbm (Nash) are never interchangeable, and the Python runtime import differs by branch. Do not recite directory or model names from memory — ls samples/ on the actual checked-out branch is the source of truth.
Branch → board cheat-sheet (the foundation)
| Board | Repo | Branch | Sample dir | Artifact | Python runtime |
|---|
| RDK X5 | rdk_model_zoo | rdk_x5 | samples/vision/<model>/ | .bin | hbm_runtime |
| RDK X5 (legacy) | rdk_model_zoo | rdk_x5_legacy | (old demos) | .bin | hobot_dnn / pyeasy_dnn |
| RDK X3 | rdk_model_zoo | rdk_x3 | demos/<task>/ (note: demos/, not samples/) | .bin | pyeasy_dnn / hobot_dnn |
| RDK S100 / S100P / S600 | rdk_model_zoo | rdk_s | samples/vision/<model>/ | .hbm | hbm_runtime |
| RDK S100 / S100P (archive) | rdk_model_zoo_s | s100 | samples/Vision/<Model>/ | .hbm | hbm_runtime |
Verified against the live rdk_x5 and rdk_s branch READMEs. The rdk_s branch is now the single current delivery branch for S100, S100P, AND S600 (its README: "Current branch. Primary delivery branch for RDK S100, S100P, and S600"). rdk_model_zoo_s/s100 is the historical archive — still complete and runnable, but rdk_s is the one to use for new work. A deterministic branch lookup is in scripts/branch_selector.py.
RDK Ultra is not a Model Zoo sample branch — there is no rdk_ultra branch and no Ultra appendix. Ultra users convert via the toolchain (march bayes, see rdk-device) rather than pulling Model Zoo precompiled artifacts.
Model format & runtime (not portable across families)
- X3 (Bernoulli2) →
.bin, classic pyeasy_dnn / hobot_dnn stack; lightweight models.
- X5 (Bayes-e) →
.bin. On the rdk_x5 branch the Python samples use hbm_runtime (the artifact is still .bin, NOT .hbm). Only the old rdk_x5_legacy branch uses pyeasy_dnn/hobot_dnn. C/C++ interfaces also ship.
- S100 / S100P / S600 (Nash) →
.hbm (not .bin!), Python hbm_runtime. The .hbm artifact is the most common point of confusion versus X3/X5's .bin.
- Even a BPU-centric model usually keeps CPU-side quantize/dequantize at the input/output, and any op that cannot map to the BPU falls back to CPU. This is expected, not a bug.
Workflows
Workflow 1 — Pick the branch and run a precompiled model (the core)
Use when: "how do I run a Model Zoo sample", "which branch", "where's the precompiled model".
- Confirm the board →
cat /sys/class/socinfo/board_id (or rdkos_info), then read the cheat-sheet row.
- Clone the matching branch (the whole point):
git clone -b rdk_x5 https://github.com/D-Robotics/rdk_model_zoo.git
git clone -b rdk_s https://github.com/D-Robotics/rdk_model_zoo.git
git clone -b rdk_x3 https://github.com/D-Robotics/rdk_model_zoo.git
- Download the precompiled artifact into the sample's
model/ dir. The download root is
https://archive.d-robotics.cc/downloads/rdk_model_zoo/<branch>/<MODEL_FAMILY>/<file>.
Filenames encode quantization + input layout, e.g. yolo11x_detect_bayese_640x640_nv12.bin
(bayese = Bayes-e quantized, nv12 = input layout).
- Run from the right CWD. Sample dirs are layered
conversion/ + evaluator/ + model/ + runtime/{cpp,python}/ + test_data/. The Python entry point is main.py — do NOT python3 *.py (the runtime dir holds main.py plus per-task scripts, so a glob hits the wrong file).
cd samples/vision/ultralytics_yolo/runtime/python
python3 main.py --task detect \
--model-path ../../model/yolo11n_detect_bayese_640x640_nv12.bin \
--test-img ../../../../../datasets/coco/assets/bus.jpg \
--img-save-path ../../test_data/inference_yolo11.jpg
main.py with no args runs the default (yolo11n + bus.jpg). Success = the output image is written.
- If slow / low FPS, confirm you are actually running the BPU
.bin/.hbm and not a raw .pt/.onnx (the latter runs CPU-only → 1–2 FPS; see rdk-device).
Workflow 2 — Answer "which models does board X have + how fast" (benchmark lookup)
Use when: the user asks whether a specific model runs on their board, or wants latency/FPS/accuracy figures.
- Map board → appendix chapter set (model_zoo_doc
docs/appendix/<board>/):
- X5 (6 chapters): classification, detection, segmentation, pose, OCR, matting — with Float vs Quant Top-1 and PyTorch AP vs Python (on-board) AP so you can judge quantization accuracy drop.
- S100/S100P (7 chapters): classification, detection, segmentation, pose, OCR, depth estimation, LLM — latency + single/dual-thread FPS.
- X3 (4 chapters): classification, detection, segmentation, OCR — no pose / matting / depth / LLM.
- S600 (1 chapter): LLM benchmark only in the appendix. S600 vision/speech models exist as runnable samples on the
rdk_s branch but have no per-model perf appendix yet.
- For the per-board, per-model tables (verified figures, branch paths), read per-board-model-catalog.md.
- If the appendix has no entry, that means "no published number," not "cannot run" — check the
rdk_s/rdk_x5 sample README.
Workflow 3 — Boundary: ready-made vs. convert-it-yourself
Use when: the user is unsure whether to pull a precompiled model or run the toolchain.
- The Model Zoo ships both precompiled artifacts (download and run) and full conversion tutorials (
conversion/ in each sample). Prefer the precompiled artifact when one exists.
- For a private/custom
.pt/.onnx with no Model Zoo match, the general quantization flow (hb_mapper for X-series / hb_compile for S-series, calibration images, march) lives in rdk-device. Use the sample's conversion/ dir as a worked template.
Worked examples
Example 1 — "我板子是 S600,Model Zoo 有现成的 YOLO11 吗?用哪个分支?"
Yes. Clone the rdk_s branch of rdk_model_zoo (it is the current delivery branch for S100/S100P/S600), get the .hbm from samples/vision/yolo11/model/, and run with hbm_runtime. The README lists YOLO11 as S100/S600-supported. Note: the model_zoo_doc appendix only has an LLM benchmark for S600, so for vision perf numbers read the S100 tables as a proxy and confirm on-board.
Example 2 — "我有一个在 X5 上转好的 .bin,能拷到 S100 上跑吗?"
No. X5 .bin is Bayes-e; S100 is Nash and needs a .hbm. They are cross-architecture incompatible. Pull the S-series precompiled model from the rdk_s branch (or rebuild via hb_compile --march nash-e, see rdk-device).
Example 3 — "Model Zoo 示例怎么跑?我下了一堆 .py 不知道跑哪个"
Run main.py, never python3 *.py. From the sample, cd runtime/python, then python3 main.py --task detect --model-path ../../model/<file>.bin. The other .py files in that dir are per-task helpers; the glob would hit the wrong one. The default (main.py with no args) runs yolo11n on bus.jpg as a smoke test.
Example 4 — "X5 上 yolo11n 量化后掉多少精度?帧率多少?"
Read the X5 detection appendix (per-board-model-catalog.md → RDK X5 → detection): yolo11n is ~8.2 ms single-thread, ~122 FPS, PyTorch AP 0.323 → Python (on-board, post-quant) AP 0.308. X5's appendix is the one place that gives Float-vs-Quant and PyTorch-vs-Python AP so you can quantify the drop.
Common pitfalls
| ❌ Don't | ✅ Do |
|---|
Clone main / rdk_x5_legacy / guess the branch | git clone -b <board-branch> — branch = board |
Treat rdk_model_zoo_s/s100 as the only S source | Use rdk_s branch of rdk_model_zoo for new S100/S100P/S600 work |
Copy a .bin onto an S-board (or .hbm onto X) | Pull the artifact for the matching branch (.bin ≠ .hbm) |
python3 *.py in the runtime dir | Run main.py with --task/--model-path |
| Assume "no appendix entry" = "can't run" | Check the sample README on the board's branch |
Expect an rdk_ultra Model Zoo branch | Ultra has none — convert via toolchain (rdk-device) |
| Recite sample/model names from memory | ls samples/ on the actual checked-out branch |
Reference map
| Read this | When |
|---|
| model-zoo-catalog.md | Branch strategy, directory layout, format×runtime table, download path, run checklist — the "how the repo is organized" reference |
| per-board-model-catalog.md | Per-board, per-model verified benchmark tables (latency / FPS / accuracy) and exact branch/sample paths — including the S600 LLM numbers and the S100/S600 sample matrix |
scripts/branch_selector.py | Deterministic board → repo / branch / sample-dir / artifact / runtime lookup |
