| name | rk-knowledge |
| description | Rockchip RK35xx single-platform reference — RK3588 / RK3576 NPU specs, the RKNN-Toolkit2 conversion workflow (.onnx → .rknn on an x86 PC), board-side inference with RKNN-Toolkit-Lite2 / librknnrt, multi-core NPU scheduling via core_mask, and the version-mismatch failures that dominate RKNN support. Use whenever someone asks about a Rockchip RK3588/RK3576 SBC (Radxa ROCK 5B/5 ITX, Orange Pi 5, Firefly), its NPU, RKNN model conversion, .rknn deployment, or "RKNN version mismatch / init runtime failed". 触发词:RK3588、RK3576、Rockchip、瑞芯微、ROCK 5B、Orange Pi 5、Firefly RK3588、RKNN、rknn-toolkit2、rknn_toolkit_lite2、librknnrt、.rknn 转换、NPU 几 TOPS、core_mask、三核 NPU、版本不匹配、init runtime failed。Routing — this skill covers the RK35xx platform itself only; RDK-vs-RK3588 / Jetson / Raspberry Pi selection and "买哪个" comparison → rdk-ecosystem (RDK-anchored); Jetson → jetson-knowledge; Raspberry Pi → rpi-knowledge; RKNN-LLM (LLMs on RK35xx) is a separate Rockchip SDK, out of scope here. |
Rockchip RK35xx Knowledge
A clean single-platform reference for Rockchip RK35xx AI SBCs (RK3588 and RK3576). What the NPU delivers, how to turn a trained model into a .rknn, how to run it on-board, and how to read the version-mismatch errors that cause most RKNN support tickets. The single most important thing to get right: the PC-side toolkit version and the board-side librknnrt version must match, or inference fails or returns garbage — this is the #1 RKNN failure.
Sources: official Rockchip / airockchip repos — airockchip/rknn-toolkit2 (README, rknpu2/runtime C API headers, rknn-toolkit-lite2 examples, doc/), airockchip/rknn_model_zoo, airockchip/rknn-llm, PyPI (rknn-toolkit2, rknn-toolkit-lite2), and Radxa / Orange Pi / Firefly product docs. Every version, API, and spec below is grounded in those; provenance is noted inline. Specs and "latest version" change — defer to the live repo / PyPI when in doubt.
The one rule that matters most: versions must match
The RKNN stack has two halves that must be the same version:
- PC side —
rknn-toolkit2 (x86_64 or aarch64 Linux): converts .onnx/.pt → .rknn, runs simulation and accuracy analysis.
- Board side —
librknnrt.so (C runtime) and/or rknn_toolkit_lite2 (Python): loads and runs the .rknn.
If you convert a model with toolkit2 v2.3.x but the board ships an older librknnrt (e.g. v1.6 / v2.1), the model either fails to load (rknn_init error), warns about a version difference, or silently produces wrong outputs. When someone reports "RKNN init runtime failed / load model failed / version mismatch / 推理结果全是乱的" — check the two versions first, before suspecting the model. Fix by upgrading librknnrt.so + the NPU kernel driver on the board to match the toolkit2 version you converted with (or re-convert with the toolkit2 version that matches the board).
Confirm the board runtime version: the C API exposes it via rknn_query(..., RKNN_QUERY_SDK_VERSION, ...); the SDK version is also printed when a sample runs. Confirm the PC version with python3 -c "from rknn.api import RKNN; print(RKNN().get_sdk_version())" or pip show rknn-toolkit2.
Platform cheat-sheet (the foundation)
Confirm the SoC first (cat /proc/device-tree/compatible or cat /proc/device-tree/model), then read the row. The NPU core count is what gates core_mask scheduling.
| SoC | NPU | NPU cores | core_mask ceiling | CPU | Process |
|---|
| RK3588 / RK3588S | 6 TOPS INT8 | 3 cores | NPU_CORE_0_1_2 (all 3) | 4× A76 + 4× A55 | 8nm |
| RK3576 | 6 TOPS INT8 (INT4/8/16/FP16/BF16/TF32) | 2 cores | NPU_CORE_0_1 (both) | 4× A72 + 4× A53 | 8nm |
The 6 TOPS is the whole NPU, not per core. On RK3588 that is 3 cores at ~2 TOPS each; a single model runs on one core by default (NPU_CORE_AUTO) and only uses the full 6 TOPS if you either split work across cores or run 2–3 model instances pinned to different cores. This per-core-vs-total distinction is the most common spec mistake.
RK3588 vs RK3588S: same NPU/CPU; "S" is the cost-reduced package (fewer PCIe/USB lanes, no second display pipeline). For AI/NPU purposes they are identical.
RKNN software stack (who does what)
| Component | Runs on | Role | Install |
|---|
| RKNN-Toolkit2 | x86_64 / aarch64 Linux PC | convert → .rknn, simulate, accuracy analysis, perf eval | pip install rknn-toolkit2 (PyPI, v2.3.x) |
| RKNN-Toolkit-Lite2 | the board (aarch64) | Python inference of a .rknn | pip install rknn-toolkit-lite2 (PyPI, aarch64 wheels) |
RKNN Runtime (librknnrt.so) | the board | C/C++ inference (rknn_api.h) | ships in the board image / RKNPU2 SDK |
| RKNPU kernel driver | the board | talks to NPU hardware | in the Rockchip kernel; check dmesg | grep -i rknpu |
- RKNN-Toolkit2 is NOT compatible with the old RKNN-Toolkit (for RK1808/RV1109/RV1126/RK3399Pro). Do not mix them.
- Toolkit2 supports Python 3.6–3.12. Wheels are published on PyPI 2.2.0, 2.2.1, 2.3.0, 2.3.2 (latest 2.3.2); the full set (matching the exact board runtime) also lives in the repo's
packages/ and the RKNPU2 SDK.
- Supported NPU platforms (toolkit2 v2.3.2): RK3588, RK3576, RK3566/RK3568, RK3562, RV1103/RV1106, RV1103B/RV1106B, RV1126B, RK2118. This skill focuses on the RK35xx AI SBCs (RK3588/RK3576).
Workflows
Workflow 1 — Identify the board and its NPU
- SoC:
cat /proc/device-tree/compatible (e.g. rockchip,rk3588) or cat /proc/device-tree/model (e.g. "Radxa ROCK 5B").
- OS / kernel:
cat /etc/os-release, uname -a — image generation gates which librknnrt and driver you have.
- NPU driver present:
dmesg | grep -i rknpu (should show the RKNPU kernel driver probing). lscpu / lsusb for the rest.
- Read the cheat-sheet row → know the core count (3 for RK3588, 2 for RK3576) before deciding any
core_mask strategy.
Workflow 2 — Convert a model to .rknn (PC side, the core loop)
Conversion runs on an x86_64 (or aarch64) Linux PC, never as the deployment step on the board. Full command reference: rknn-toolkit-workflow.md.
- Install the matching toolkit2 —
pip install rknn-toolkit2. If the board runtime is older than PyPI's latest, install the wheel that matches the board's librknnrt from the repo packages/ instead, so the two halves line up (see "the one rule").
- Export ONNX correctly — for YOLO and similar detectors, export from the airockchip fork (
airockchip/yolov5, airockchip/ultralytics_yolov8, airockchip/ultralytics_yolo11, etc.), which strips post-processing into an RKNN-friendly head. A vanilla ultralytics export converts but is slower / harder to quantize well.
- Convert with the Python API —
RKNN() → config(target_platform='rk3588', ...) → load_onnx(...) → build(do_quantization=True, dataset='dataset.txt') (≈ a few hundred calibration images) → export_rknn('model.rknn'). target_platform must match the board (rk3588, rk3576, …); a .rknn built for one platform will not run on another.
- Verify before deploying —
rknn.accuracy_analysis(...) to catch quantization drift, and rknn.eval_perf() for on-NPU latency (needs a connected board). For INT8 accuracy loss, try hybrid/mixed quantization or per-channel quant in config.
rknn_model_zoo has a ready convert script per model (convert.py <onnx> <platform> <dtype> <out.rknn>) — prefer it over hand-writing the API for the common detectors.
Workflow 3 — Run the .rknn on the board
Python (RKNN-Toolkit-Lite2):
from rknnlite.api import RKNNLite
rknn = RKNNLite()
rknn.load_rknn('model.rknn')
rknn.init_runtime(core_mask=RKNNLite.NPU_CORE_0)
outputs = rknn.inference(inputs=[img])
core_mask only matters on RK3576/RK3588 (multi-core). Default NPU_CORE_AUTO (=0) lets the runtime pick. To use the full 6 TOPS of an RK3588, run separate model instances pinned to NPU_CORE_0 / _1 / _2, or use NPU_CORE_0_1_2 for one large model.- C/C++ side mirrors this:
rknn_init → rknn_set_core_mask(ctx, RKNN_NPU_CORE_0_1_2) → rknn_run. Header: rknpu2/runtime/.../rknn_api.h.
Diagnose a failing run, in order:
- Versions match? (toolkit2 vs board
librknnrt — Workflow 0 of debugging; see "the one rule").
- NPU driver loaded?
dmesg | grep -i rknpu.
target_platform at convert time == this SoC?- Only then suspect the model / preprocessing (input layout NHWC vs NCHW, mean/std, color order).
Workflow 4 — Out of scope: LLMs on RK35xx
For large language models on RK35xx, Rockchip ships a separate SDK, RKNN-LLM (airockchip/rknn-llm, with rkllm-toolkit to convert to .rkllm and librkllmrt to run). It is not part of rknn-toolkit2. If the user asks "run Qwen/Llama/DeepSeek on RK3588", point them to RKNN-LLM and note this skill covers the vision/CNN RKNN path. Do not try to push an LLM through rknn-toolkit2.
Worked examples
Example 1 — "RKNN init runtime failed / load model 报版本不匹配怎么办?"
Lead with the version rule: "This is almost always a PC-toolkit-vs-board-runtime version gap. Check both: on the PC pip show rknn-toolkit2; on the board the librknnrt.so version (printed when a sample runs, or via rknn_query RKNN_QUERY_SDK_VERSION). Make them match — either re-convert the .rknn with the toolkit2 version that matches the board's librknnrt, or upgrade librknnrt.so + the RKNPU kernel driver on the board to the version you converted with. Only after they match should you suspect the model." Then point to Workflow 3's diagnosis order.
Example 2 — "RK3588 不是 6 TOPS 吗,为什么我跑一个模型只用了 1/3?"
"6 TOPS is the whole NPU, which is 3 cores (~2 TOPS each). One model with the default NPU_CORE_AUTO runs on a single core, so you see roughly 1/3. To use all of it: either run 2–3 model instances pinned to NPU_CORE_0 / _1 / _2, or set core_mask=NPU_CORE_0_1_2 for a single model that supports multi-core. RK3576 is the same idea but 2 cores."
Example 3 — "怎么把我的 YOLOv8 .pt 部署到 Orange Pi 5 (RK3588)?"
"Two stages. PC side: export ONNX from the airockchip/ultralytics_yolov8 fork (RKNN-friendly head), then pip install rknn-toolkit2 and convert with target_platform='rk3588' and a calibration dataset → model.rknn. Board side: pip install rknn-toolkit-lite2, then RKNNLite().load_rknn + init_runtime + inference. Make sure the toolkit2 version matches the board's librknnrt. The rknn_model_zoo has a ready yolov8 convert script." Route to Workflow 2 then 3.
Example 4 — "我想在 RK3588 上跑 Qwen 大模型,用 rknn-toolkit2 转吗?"
"No — rknn-toolkit2 is for CNN/vision models. LLMs use Rockchip's separate RKNN-LLM SDK (airockchip/rknn-llm): rkllm-toolkit converts the HF model to .rkllm, and librkllmrt runs it on the board. That's a different toolchain from everything above." (Out of scope here; just route.)
Common pitfalls
| ❌ Don't | ✅ Do |
|---|
| Quote 6 TOPS as if one model gets it all | 6 TOPS = 3 cores (RK3588) / 2 cores (RK3576); one model on AUTO uses one core |
| Assume "load model failed" means the model is broken | Check toolkit2 vs board librknnrt versions first |
| Mix RKNN-Toolkit2 with the old RKNN-Toolkit | They are incompatible; RK35xx uses Toolkit2 only |
Convert with target_platform='rk3588' and run on RK3576 (or vice-versa) | .rknn is platform-specific; convert per SoC |
| Export YOLO from vanilla ultralytics and wonder why it's slow | Export from the airockchip fork (RKNN-optimized head) |
| Push an LLM through rknn-toolkit2 | Use the separate RKNN-LLM SDK (.rkllm / librkllmrt) |
pip install rknn-toolkit-lite2 on the x86 PC for conversion | Lite2 is board-side (aarch64) inference; conversion is rknn-toolkit2 on the PC |
Reference map
| Read this | When |
|---|
| rknn-toolkit-workflow.md | Converting / deploying a model — full toolkit2 Python API call sequence, config/build params, quantization, board-side Lite2 + C API, core_mask, version-matching, and the RKNN-LLM out-of-scope note |
scripts/rk_npu_selector.py | Quick SoC → NPU TOPS / core count / core_mask ceiling lookup, so the per-core-vs-total fact never drifts |
