rk-knowledge

name

rk-knowledge

description

Rockchip RK35xx single-platform reference — RK3588 / RK3576 NPU specs, the RKNN-Toolkit2 conversion workflow (.onnx → .rknn on an x86 PC), board-side inference with RKNN-Toolkit-Lite2 / librknnrt, multi-core NPU scheduling via core_mask, and the version-mismatch failures that dominate RKNN support. Use whenever someone asks about a Rockchip RK3588/RK3576 SBC (Radxa ROCK 5B/5 ITX, Orange Pi 5, Firefly), its NPU, RKNN model conversion, .rknn deployment, or "RKNN version mismatch / init runtime failed". 触发词:RK3588、RK3576、Rockchip、瑞芯微、ROCK 5B、Orange Pi 5、Firefly RK3588、RKNN、rknn-toolkit2、rknn_toolkit_lite2、librknnrt、.rknn 转换、NPU 几 TOPS、core_mask、三核 NPU、版本不匹配、init runtime failed。Routing — this skill covers the RK35xx platform itself only; RDK-vs-RK3588 / Jetson / Raspberry Pi selection and "买哪个" comparison → rdk-ecosystem (RDK-anchored); Jetson → jetson-knowledge; Raspberry Pi → rpi-knowledge; RKNN-LLM (LLMs on RK35xx) is a separate Rockchip SDK, out of scope here.

Rockchip RK35xx Knowledge

A clean single-platform reference for Rockchip RK35xx AI SBCs (RK3588 and RK3576). What the NPU delivers, how to turn a trained model into a .rknn, how to run it on-board, and how to read the version-mismatch errors that cause most RKNN support tickets. The single most important thing to get right: the PC-side toolkit version and the board-side librknnrt version must match, or inference fails or returns garbage — this is the #1 RKNN failure.

Sources: official Rockchip / airockchip repos — airockchip/rknn-toolkit2 (README, rknpu2/runtime C API headers, rknn-toolkit-lite2 examples, doc/), airockchip/rknn_model_zoo, airockchip/rknn-llm, PyPI (rknn-toolkit2, rknn-toolkit-lite2), and Radxa / Orange Pi / Firefly product docs. Every version, API, and spec below is grounded in those; provenance is noted inline. Specs and "latest version" change — defer to the live repo / PyPI when in doubt.

The one rule that matters most: versions must match

The RKNN stack has two halves that must be the same version:

• PC side — rknn-toolkit2 (x86_64 or aarch64 Linux): converts .onnx/.pt → .rknn, runs simulation and accuracy analysis.
• Board side — librknnrt.so (C runtime) and/or rknn_toolkit_lite2 (Python): loads and runs the .rknn.

If you convert a model with toolkit2 v2.3.x but the board ships an older librknnrt (e.g. v1.6 / v2.1), the model either fails to load (rknn_init error), warns about a version difference, or silently produces wrong outputs. When someone reports "RKNN init runtime failed / load model failed / version mismatch / 推理结果全是乱的" — check the two versions first, before suspecting the model. Fix by upgrading librknnrt.so + the NPU kernel driver on the board to match the toolkit2 version you converted with (or re-convert with the toolkit2 version that matches the board).

Confirm the board runtime version: the C API exposes it via rknn_query(..., RKNN_QUERY_SDK_VERSION, ...); the SDK version is also printed when a sample runs. Confirm the PC version with python3 -c "from rknn.api import RKNN; print(RKNN().get_sdk_version())" or pip show rknn-toolkit2.

Platform cheat-sheet (the foundation)

Confirm the SoC first (cat /proc/device-tree/compatible or cat /proc/device-tree/model), then read the row. The NPU core count is what gates core_mask scheduling.

SoC	NPU	NPU cores	core_mask ceiling	CPU	Process
RK3588 / RK3588S	6 TOPS INT8	3 cores	NPU_CORE_0_1_2 (all 3)	4× A76 + 4× A55	8nm
RK3576	6 TOPS INT8 (INT4/8/16/FP16/BF16/TF32)	2 cores	NPU_CORE_0_1 (both)	4× A72 + 4× A53	8nm

The 6 TOPS is the whole NPU, not per core. On RK3588 that is 3 cores at ~2 TOPS each; a single model runs on one core by default (NPU_CORE_AUTO) and only uses the full 6 TOPS if you either split work across cores or run 2–3 model instances pinned to different cores. This per-core-vs-total distinction is the most common spec mistake.

RK3588 vs RK3588S: same NPU/CPU; "S" is the cost-reduced package (fewer PCIe/USB lanes, no second display pipeline). For AI/NPU purposes they are identical.

RKNN software stack (who does what)

Component	Runs on	Role	Install
RKNN-Toolkit2	x86_64 / aarch64 Linux PC	convert → .rknn, simulate, accuracy analysis, perf eval	pip install rknn-toolkit2 (PyPI, v2.3.x)
RKNN-Toolkit-Lite2	the board (aarch64)	Python inference of a .rknn	pip install rknn-toolkit-lite2 (PyPI, aarch64 wheels)
RKNN Runtime (librknnrt.so)	the board	C/C++ inference (rknn_api.h)	ships in the board image / RKNPU2 SDK
RKNPU kernel driver	the board	talks to NPU hardware	in the Rockchip kernel; check dmesg | grep -i rknpu

• RKNN-Toolkit2 is NOT compatible with the old RKNN-Toolkit (for RK1808/RV1109/RV1126/RK3399Pro). Do not mix them.
• Toolkit2 supports Python 3.6–3.12. Wheels are published on PyPI 2.2.0, 2.2.1, 2.3.0, 2.3.2 (latest 2.3.2); the full set (matching the exact board runtime) also lives in the repo's packages/ and the RKNPU2 SDK.
• Supported NPU platforms (toolkit2 v2.3.2): RK3588, RK3576, RK3566/RK3568, RK3562, RV1103/RV1106, RV1103B/RV1106B, RV1126B, RK2118. This skill focuses on the RK35xx AI SBCs (RK3588/RK3576).

Workflows

Workflow 1 — Identify the board and its NPU

1. SoC: cat /proc/device-tree/compatible (e.g. rockchip,rk3588) or cat /proc/device-tree/model (e.g. "Radxa ROCK 5B").
2. OS / kernel: cat /etc/os-release, uname -a — image generation gates which librknnrt and driver you have.
3. NPU driver present: dmesg | grep -i rknpu (should show the RKNPU kernel driver probing). lscpu / lsusb for the rest.
4. Read the cheat-sheet row → know the core count (3 for RK3588, 2 for RK3576) before deciding any core_mask strategy.

Workflow 2 — Convert a model to .rknn (PC side, the core loop)

Conversion runs on an x86_64 (or aarch64) Linux PC, never as the deployment step on the board. Full command reference: rknn-toolkit-workflow.md.

1. Install the matching toolkit2 — pip install rknn-toolkit2. If the board runtime is older than PyPI's latest, install the wheel that matches the board's librknnrt from the repo packages/ instead, so the two halves line up (see "the one rule").
2. Export ONNX correctly — for YOLO and similar detectors, export from the airockchip fork (airockchip/yolov5, airockchip/ultralytics_yolov8, airockchip/ultralytics_yolo11, etc.), which strips post-processing into an RKNN-friendly head. A vanilla ultralytics export converts but is slower / harder to quantize well.
3. Convert with the Python API — RKNN() → config(target_platform='rk3588', ...) → load_onnx(...) → build(do_quantization=True, dataset='dataset.txt') (≈ a few hundred calibration images) → export_rknn('model.rknn'). target_platform must match the board (rk3588, rk3576, …); a .rknn built for one platform will not run on another.
4. Verify before deploying — rknn.accuracy_analysis(...) to catch quantization drift, and rknn.eval_perf() for on-NPU latency (needs a connected board). For INT8 accuracy loss, try hybrid/mixed quantization or per-channel quant in config.
5. rknn_model_zoo has a ready convert script per model (convert.py <onnx> <platform> <dtype> <out.rknn>) — prefer it over hand-writing the API for the common detectors.

Workflow 3 — Run the .rknn on the board

Python (RKNN-Toolkit-Lite2):

from rknnlite.api import RKNNLite
rknn = RKNNLite()
rknn.load_rknn('model.rknn')
rknn.init_runtime(core_mask=RKNNLite.NPU_CORE_0)   # or NPU_CORE_AUTO / NPU_CORE_0_1_2
outputs = rknn.inference(inputs=[img])

• core_mask only matters on RK3576/RK3588 (multi-core). Default NPU_CORE_AUTO (=0) lets the runtime pick. To use the full 6 TOPS of an RK3588, run separate model instances pinned to NPU_CORE_0 / _1 / _2, or use NPU_CORE_0_1_2 for one large model.
• C/C++ side mirrors this: rknn_init → rknn_set_core_mask(ctx, RKNN_NPU_CORE_0_1_2) → rknn_run. Header: rknpu2/runtime/.../rknn_api.h.

Diagnose a failing run, in order:

1. Versions match? (toolkit2 vs board librknnrt — Workflow 0 of debugging; see "the one rule").
2. NPU driver loaded? dmesg | grep -i rknpu.
3. target_platform at convert time == this SoC?
4. Only then suspect the model / preprocessing (input layout NHWC vs NCHW, mean/std, color order).

Workflow 4 — Out of scope: LLMs on RK35xx

For large language models on RK35xx, Rockchip ships a separate SDK, RKNN-LLM (airockchip/rknn-llm, with rkllm-toolkit to convert to .rkllm and librkllmrt to run). It is not part of rknn-toolkit2. If the user asks "run Qwen/Llama/DeepSeek on RK3588", point them to RKNN-LLM and note this skill covers the vision/CNN RKNN path. Do not try to push an LLM through rknn-toolkit2.

Worked examples

Example 1 — "RKNN init runtime failed / load model 报版本不匹配怎么办?" Lead with the version rule: "This is almost always a PC-toolkit-vs-board-runtime version gap. Check both: on the PC pip show rknn-toolkit2; on the board the librknnrt.so version (printed when a sample runs, or via rknn_query RKNN_QUERY_SDK_VERSION). Make them match — either re-convert the .rknn with the toolkit2 version that matches the board's librknnrt, or upgrade librknnrt.so + the RKNPU kernel driver on the board to the version you converted with. Only after they match should you suspect the model." Then point to Workflow 3's diagnosis order.

Example 2 — "RK3588 不是 6 TOPS 吗,为什么我跑一个模型只用了 1/3?" "6 TOPS is the whole NPU, which is 3 cores (~2 TOPS each). One model with the default NPU_CORE_AUTO runs on a single core, so you see roughly 1/3. To use all of it: either run 2–3 model instances pinned to NPU_CORE_0 / _1 / _2, or set core_mask=NPU_CORE_0_1_2 for a single model that supports multi-core. RK3576 is the same idea but 2 cores."

Example 3 — "怎么把我的 YOLOv8 .pt 部署到 Orange Pi 5 (RK3588)?" "Two stages. PC side: export ONNX from the airockchip/ultralytics_yolov8 fork (RKNN-friendly head), then pip install rknn-toolkit2 and convert with target_platform='rk3588' and a calibration dataset → model.rknn. Board side: pip install rknn-toolkit-lite2, then RKNNLite().load_rknn + init_runtime + inference. Make sure the toolkit2 version matches the board's librknnrt. The rknn_model_zoo has a ready yolov8 convert script." Route to Workflow 2 then 3.

Example 4 — "我想在 RK3588 上跑 Qwen 大模型,用 rknn-toolkit2 转吗?" "No — rknn-toolkit2 is for CNN/vision models. LLMs use Rockchip's separate RKNN-LLM SDK (airockchip/rknn-llm): rkllm-toolkit converts the HF model to .rkllm, and librkllmrt runs it on the board. That's a different toolchain from everything above." (Out of scope here; just route.)

Common pitfalls

❌ Don't	✅ Do
Quote 6 TOPS as if one model gets it all	6 TOPS = 3 cores (RK3588) / 2 cores (RK3576); one model on AUTO uses one core
Assume "load model failed" means the model is broken	Check toolkit2 vs board librknnrt versions first
Mix RKNN-Toolkit2 with the old RKNN-Toolkit	They are incompatible; RK35xx uses Toolkit2 only
Convert with target_platform='rk3588' and run on RK3576 (or vice-versa)	.rknn is platform-specific; convert per SoC
Export YOLO from vanilla ultralytics and wonder why it's slow	Export from the airockchip fork (RKNN-optimized head)
Push an LLM through rknn-toolkit2	Use the separate RKNN-LLM SDK (.rkllm / librkllmrt)
pip install rknn-toolkit-lite2 on the x86 PC for conversion	Lite2 is board-side (aarch64) inference; conversion is rknn-toolkit2 on the PC

Reference map

Read this	When
rknn-toolkit-workflow.md	Converting / deploying a model — full toolkit2 Python API call sequence, config/build params, quantization, board-side Lite2 + C API, core_mask, version-matching, and the RKNN-LLM out-of-scope note
scripts/rk_npu_selector.py	Quick SoC → NPU TOPS / core count / core_mask ceiling lookup, so the per-core-vs-total fact never drifts