DOCSEARCHbyAlgolia
Submit Docs

Paddle Lite Documentation

repository·develop·Indexed 27 days ago

https://github.com/paddlepaddle/paddle-lite

A high-performance, lightweight, and flexible deep learning inference framework designed for mobile, embedded, and edge computing platforms. Includes guides for precision evaluation on Android using benchmark_bin, building the Arm NEON accelerated arm_dnn_library for Android and Linux, and utilizing the JIT Kernel system for optimized x86 CPU computation.

Tokens
126.1K
Snippets
207
Records
601
Agent score
92%

What's inside Paddle Lite

  1. Overview of Intel OpenVINO support in Paddle Lite

    develop

    Paddle Lite supports predictive deployment on Intel OpenVINO for X86 servers using a subgraph integration method.

    How it works:

    1. The system performs online analysis of the Paddle model.
    2. Paddle operators are converted into unified NNAdapter standard operators.
    3. The network is constructed using the OpenVINO Graph API (API 2.0).
    4. The model is generated and executed online.

  2. Overview of Qualcomm QNN support in Paddle Lite

    develop

    Paddle Lite supports deployment on Qualcomm QNN via x86 (simulator) and ARM devices (e.g., SA8295P).

    It utilizes a subgraph接入 (subgraph接入) approach. The mechanism works by performing online analysis of the Paddle model, converting Paddle operators into unified NNAdapter standard operators, and then using the Qualcomm QNN networking API to construct and execute the network online.

  4. Overview of Paddle Lite

    develop
    Paddle Lite is an updated version of Paddle-Mobile, an open-source deep learning framework optimized for performing inference on mobile, embedded, and IoT devices. It is compatible with PaddlePaddle and can use pre-trained models from other sources. It is designed for high performance and lightweight deployment, focusing on reduced model/binary size, efficient inference, and low memory usage.

  5. Overview of NNAdapter Framework

    develop

    NNAdapter is a unified hardware adaptation framework for Paddle Lite inference. It acts as a bridge between deep learning inference frameworks and AI hardware (especially AI ASIC chips) by decoupling the framework from the hardware implementation.

    Key objectives include:

    • Lowering entry barriers: Hardware vendors do not need to understand the internal implementation of the inference framework; they only need to implement the NNAdapter standard interfaces.
    • Reducing adaptation code: Vendors focus on a thin Hardware Abstraction Layer (HAL), reducing the amount of code required for new hardware.
    • Lowering maintenance costs: Framework updates and operator upgrades are absorbed by the NNAdapter adaptation layer, leaving the hardware HAL code unaffected.

  6. Overview of OpenCL Support in Paddle Lite

    develop
    Paddle Lite uses the cross-platform OpenCL framework to map computations to the GPU, leveraging hardware acceleration to improve inference performance. During execution, Paddle Lite prioritizes GPU execution for operators; if a specific operator lacks a GPU implementation, it automatically falls back to CPU execution.

  7. Overview of Kunlunxin XTCL Support in Paddle Lite

    develop

    Paddle Lite supports Kunlunxin series chips and boards for prediction deployment on X86 and ARM servers.

    Integration Mechanism: It uses a sub-graph接入 (sub-graph接入) approach. The process involves:

    1. Online analysis of the Paddle model.
    2. Converting Paddle operators into the unified NNAdapter standard operator format.
    3. Using the XTCL (XPU Tensor Compilation Library) networking APIs to build the network.
    4. Online generation and execution of the model.

    XTCL is a graph compilation engine library designed for Kunlunxin hardware, providing graph-level analysis frameworks and acceleration optimization capabilities.

  9. Overview of Paddle Lite Technical Features

    develop
    Paddle Lite is designed to bridge the gap between Server-side requirements (comprehensive graph analysis and optimization) and Mobile-side requirements (lightweight deployment). It focuses on high performance and multi-hardware support through a unified architecture that balances framework clarity with hardware-specific scheduling optimizations.

  11. Paddle Lite Architecture Overview

    develop

    Paddle Lite's architecture is designed for multi-hardware support and mixed execution across different hardware types. It is divided into two main phases:

    • Analysis Phase: Includes the MIR (Machine IR) modules. This phase optimizes the model's computational graph for specific hardware through operator fusion and computational pruning.
    • Execution Phase: Focuses on Kernel execution. This phase can be deployed independently to support extremely lightweight deployments.

  12. MediaTek APU Support Overview

    develop

    Paddle Lite supports predictive deployment on MediaTek APU. The integration works by loading and analyzing Paddle models, then converting Paddle operators into the MediaTek Neuron adapter API (similar to Android NNAPI) to construct, generate, and execute the model online.

    Supported Chips

    • MT8168
    • MT8175
    • Other intelligent chips

    Supported Devices

    • MT8168-P2V1 Tablet

    Supported Models

    • mobilenet_v1_int8_224_per_layer
    • mobilenet_v1_int8_224_per_channel
    • resnet50_int8_224_per_layer
    • ssd_mobilenet_v1_relu_int8_300_per_layer

    Operator Support

    To check the latest support status for specific operators on new hardware, refer to the NNAdapter operator support list in the source code.