DOCSEARCHbyAlgolia
Submit Docs

RDK Robotics Development Kit

website·Indexed 34 days ago

https://developer.d-robotics.cc/

Developer documentation for RDK X and S series hardware, including the RDK X3 and X5 modules. Covers hardware design guidelines, core module interfaces, BPU SDK APIs for model inference, and the Horizon XJ3 Open Explorer for quantization-aware training (QAT), FX Quantization, and model deployment on Bernoulli2 architecture.

Tokens
2.9M
Snippets
12.3K
Records
20.1K
Agent score
63%

What's inside RDK

  1. Overview of the Basic Sample Package

    The Basic Sample Package provides developers with a foundation for building applications using the dnn API. It is categorized into three main areas:

    1. dnn API know-how: Tutorials on memory, model loading, resizing, ROI inference, and tensors.
    2. Special Features: Demonstrations of advanced capabilities such as custom operators (e.g., CustomIdentity).
    3. Miscellaneous: Samples for models with non-NV12 input formats.

  4. Overview of Pi0 VLA Model and Hardware Performance

    The Pi0 Vision-Language-Action (VLA) model is an end-to-end system that generates robot actions from visual input and language instructions. On RDK S600 hardware, the Pi0 model achieves the following performance benchmarks:

    • Total latency: 98ms
    • Pre-process: 1.5ms
    • Vision: 18.5ms
    • Language: 39ms
    • Action: 38.5ms
    • Post-process: 0.05ms
    • Memory usage: 4.4 GB

  7. Overview of Adaround PTQ weight reconstruction

    Adaround is an advanced Post-Training Quantization (PTQ) method that improves model calibration accuracy by learning an optimal rounding paradigm (rounding up or down) for weights, rather than using traditional rounding. It optimizes Conv and Linear layers in topological order to minimize quantization error. It is particularly effective for classification, segmentation, and BEV tasks as a supplement to the standard calibration process.

  8. Overview of IPCF Inter-Processor Communication

    The IPC (Inter-Processor Communication) module enables communication between homogeneous and heterogeneous cores. It utilizes a buffer-ring for shared memory management and implements hardware interrupts via MailBox.

    Key components include:

    • IPCF: Used on the MCU side.
    • IPCFHAL: Used on the Acore side to wrap IPCF interfaces for data transfer between user space and kernel space.
    • RPMSG: An open-source protocol framework supporting communication between Acore and VDSP.

  9. Overview of RDK Model Zoo

    RDK Model Zoo is a collection of BPU (Brain Processing Unit) model examples and tools provided by D-Robotics for RDK series development boards. It supports various AI domains including image classification, object detection, instance segmentation, pose estimation, OCR, and multi-modal applications. The repository provides a complete reference implementation covering the workflow from original models (PyTorch/ONNX) through fixed-point quantization conversion, inference execution, result parsing, and example verification.
    GitHub Repository: https://github.com/D-Robotics/rdk_model_zoo

  10. Overview of EfficientNet_Det Object Detection

    EfficientNet_Det is an object detection example that uses images as input, performs inference on the BPU, and publishes algorithm messages containing object categories and bounding boxes. It is based on an ONNX model trained on the COCO dataset, supporting 80 categories (e.g., people, animals, vehicles). It is primarily used for vehicle detection in autonomous driving and smart home scenarios.

  11. Overview of get_isp_data sample

    get_isp_data is a sample program used to initialize the Camera Sensor, MIPI CSI, SIF, and ISP modules. It allows developers to capture video frame data (YUV format) from the ISP module to debug and verify the MIPI CSI, SIF, and ISP hardware. It supports both online and offline modes via command-line parameters.