RKNN Model Zoo

RKNN Model Zoo is a collection of deployment routines for mainstream algorithms based on the RKNPU SDK toolchain. It provides complete workflows including exporting RKNN models and performing inference using both Python APIs and C APIs.

Supported Platforms:

• RK3562, RK3566, RK3568, RK3576, RK3588, RV1126B
• Partial support for RV1103, RV1106
• RV1109, RV1126, RK1808

librga is a user-space driver implementation for the RGA (Raster Graphic Acceleration Unit), which is an independent 2D hardware accelerator. It is used to accelerate common 2D graphics operations such as:

• Point/line drawing
• Image scaling
• Image rotation
• bitBlt (Bit Block Transfer)
• Alpha blending

libjpeg-turbo provides two distinct APIs for JPEG compression and decompression:

• TurboJPEG API: An easy-to-use interface for in-memory operations. It is ideal for generating planar YUV images and performing multiple simultaneous lossless transforms. The Java interface is built on top of this API.
• libjpeg API: The industry-standard API. It is more powerful and complex than TurboJPEG. libjpeg-turbo is API/ABI and mathematically compatible with libjpeg v6b. It can optionally be configured to emulate libjpeg v7 or v8.

To run the CLIP demo on an Android device, follow these steps:

1. Compile and Build

From the rknn_model_zoo root directory, set your NDK path and run the build script. You must specify the target platform, architecture (e.g., arm64-v8a), and the demo name (clip).

cd ../../
export ANDROID_NDK_PATH=<android_ndk_path>
./build-android.sh -t rk3588 -a arm64-v8a -d clip

Note: Use adb shell cat /proc/version to verify if your device is arm64-v8a or armeabi-v7a.

2. Push files to device

adb root
adb remount
adb push install/<TARGET_PLATFORM>_android_<ARCH>/rknn_clip_demo/ /data/

3. Run the demo

adb shell
cd /data/rknn_clip_demo
export LD_LIBRARY_PATH=./lib
./rknn_clip_demo clip_images_fp16.rknn model/dog_224x224.jpg clip_text_fp16.rknn model/text.txt

To achieve better inference performance on RKNPU, the PP-YOLOE model output structure is modified. These changes move the DFL (Distribution Focal Loss) structure to post-processing and add an extra output representing the sum of all category scores to accelerate candidate box filtering.

To apply these optimizations, you must patch the PaddleDetection repository (specifically the release/2.5 branch) using the provided patch file.

git clone https://github.com/PaddlePaddle/PaddleDetection
cd PaddleDetection
git checkout release/2.5
git apply ../ppyoloe_rknn_optimize.patch

When using the MMS-TTS demos, you must manually adjust the MAX_LENGTH parameter to match the input length of the encoder model being used. Failure to align this value with your specific encoder model's input length may result in incorrect processing.

• For Python Demos: Modify the MAX_LENGTH value in mms_tts.py.
• For C++ Demos: Modify the MAX_LENGTH value in process.h.

If you are distributing only libjpeg-turbo binaries (without source) or an application that statically links with libjpeg-turbo, you must follow these documentation requirements:

1. General Attribution: Your product documentation must include the following message: This software is based in part on the work of the Independent JPEG Group.
2. TurboJPEG API Attribution: If your binary distribution includes or uses the TurboJPEG API, your product documentation must also include the full text of the Modified (3-clause) BSD License.

To build the Android demo, you must be in the rknn_model_zoo root directory and have the Android NDK path exported.

Steps:

1. Export the NDK path: export ANDROID_NDK_PATH=<android_ndk_path>
2. Run the build script with the target platform, architecture, and demo name.

Architecture Query: To find the correct <ARCH> (e.g., arm64-v8a or armeabi-v7a), run:

adb shell cat /proc/version

Example:

cd ../../
export ANDROID_NDK_PATH=/path/to/ndk
./build-android.sh -t rk3588 -a arm64-v8a -d ppseg

# go back to the rknn_model_zoo root directory
cd ../../
export ANDROID_NDK_PATH=<android_ndk_path>

./build-android.sh -t <TARGET_PLATFORM> -a <ARCH> -d ppseg

Convert the ONNX models to RKNN format using the convert.py script.

Arguments:

• <onnx_model>: Path to the ONNX model.
• <TARGET_PLATFORM>: NPU platform name (e.g., rk3588, rk3576, rk3568, rk3566, rk3562, rv1126b).
• <dtype> (optional): i8 for quantization (default for yolo_world) or fp for no quantization (required for clip_text).
• <output_rknn_path> (optional): Path to save the resulting .rknn file.

Note: The coco_text_outp.npy used in quantization is obtained via the save_text_outputs parameter in yolo_world.py.

cd python
python convert.py ../model/yolo_world_v2s.onnx rk3588

1. Compile and Build

From the rknn_model_zoo root directory, run the build script. You can optionally set GCC_COMPILER if it is not in your path.

export GCC_COMPILER=<GCC_COMPILER_PATH>
./build-linux.sh -t <TARGET_PLATFORM> -a <ARCH> -d yolov5_seg

Example for RK3588: ./build-linux.sh -t rk3588 -a aarch64 -d yolov5_seg

2. Push to Device

Use adb push for USB-connected devices, or scp for other boards:

adb push install/<TARGET_PLATFORM>_linux_<ARCH>/rknn_yolov5_seg_demo/ /data/

3. Run Demo

adb shell
cd /data/rknn_yolov5_seg_demo
export LD_LIBRARY_PATH=./lib
./rknn_yolov5_seg_demo model/yolov5_seg.rknn model/bus.jpg

Results are saved as out.png. Pull them using adb pull /data/rknn_yolov5_seg_demo/out.png.

After compiling the C/C++ demo, push the files to your Linux-based board and execute the binary.

1. Push files to device:

• Via ADB (EVB board):

  adb push install/<TARGET_PLATFORM>_linux_<ARCH>/rknn_LPRNet_demo/ /userdata/

• Via SCP or other methods: Push files to /userdata.

2. Run the demo:

adb shell
cd /userdata/rknn_LPRNet_demo
export LD_LIBRARY_PATH=./lib
./rknn_lprnet_demo model/lprnet.rknn model/test.jpg

Note for RV1106/1103: You must use an absolute path for LD_LIBRARY_PATH:

export LD_LIBRARY_PATH=/userdata/rknn_LPRNet_demo/lib