DOCSEARCHbyAlgolia
Submit Docs

Vitis AI Documentation

repository·master·Indexed 23 days ago

https://github.com/xilinx/vitis-ai

An Integrated Development Environment (IDE) used to accelerate AI inference on AMD adaptable platforms, including SoCs and Alveo Data Center accelerator cards. It provides tools, libraries, and optimized IP, such as the Deep Learning Processing Unit (DPU). The documentation covers Docker installation, target board setup, and technical reference designs (TRD) for the VCK190 DPUCVDX8G, including hardware/software requirements and kernel integration.

Tokens
170.4K
Snippets
449
Records
894
Agent score
82%

What's inside Vitis AI

  1. Overview of Vitis AI Source Code Repository

    master
    Vitis AI provides adaptable and real-time AI inference acceleration. This repository contains specific components of Vitis AI released as source code to provide developers with maximum flexibility for their AI workflows.

  2. Overview of OnBoard for AI Inference Analysis

    master

    OnBoard is an experimental tool introduced in Vitis AI 3.5.0 designed for analysis and debugging of WeGO users. It extends the TensorBoard web server to provide visualization of the inference flow for a model.

    Key capabilities include:

    • Image Data Visualization: Displaying images of inference results.
    • Graph Examination: Visualizing the PyTorch or TensorFlow graph structure both before and after WeGO transformation.
    • Platform Information: Displaying DPU platform-related information.

    Currently, OnBoard supports the PyTorch and TensorFlow 2 frameworks.

  4. What is Xilinx Intermediate Representation (XIR)?

    master

    Xilinx Intermediate Representation (XIR) is a graph-based representation of AI algorithms designed for efficient compilation and deployment on FPGA platforms (specifically the DPU). It serves as the foundation for Vitis AI tools like the quantizer, compiler, and runtime.

    Key characteristics:

    • Formats: XIR exists as an in-memory xir::Graph object and a file format called .xmodel. A Graph can be serialized to a .xmodel file and vice versa.
    • Unified Representation: It provides a unified set of operators (Op) that bridges the gap between different deep learning frameworks like TensorFlow, PyTorch, and Caffe.
    • PyXIR: A Python API (PyXIR) is available to allow Python users to access XIR functionality directly within Python environments.

  5. Overview of Vitis AI Profiler

    master

    Vitis AI Profiler is an application-level tool designed to detect bottlenecks in AI applications running on heterogeneous systems. It profiles pre-processing functions, post-processing functions, and DPU (Deep Learning Processor Unit) kernel execution status simultaneously.

    By identifying whether a bottleneck lies in the CPU (e.g., slow pre-processing) or the DPU, developers can decide whether to optimize code using HLS or OpenCL to reduce CPU workload or adjust DPU utilization.

  6. Overview of the Vitis AI solution components

    master

    AMD Vitis™ AI is an integrated development environment for accelerating AI inference on AMD platforms (such as Zynq UltraScale+ MPSoCs, Kria KV260, Versal, and Alveo cards). The solution is divided into three primary functional areas:

    1. Deep-Learning Processor unit (DPU): A programmable hardware engine optimized for deep neural network inference (supporting architectures like VGG, ResNet, YOLO, etc.).
    2. Model Development Tools: A toolchain used to compile and optimize ML models for the DPU.
    3. Model Deployment Libraries and APIs: Software components used to integrate and execute models on the DPU from an application.

    Delivery Methods:

    • AMD open download: Pre-built target images with the DPU integrated.
    • Vitis AI docker containers: Used for model development.
    • Vitis AI github repository: Contains deployment libraries, setup scripts, examples, and reference designs.

  7. Overview of Vitis Optimizer

    master

    Vitis Optimizer is used to prune neural network models by removing redundant kernels. This process reduces the overall computational cost for inference, helping models meet low-latency and high-throughput requirements.

    Workflow Note: Pruned models produced by Vitis Optimizer are intended to be passed to the Vitis Quantizer for further optimization.

  9. Overview of Vitis AI Runtime (VART)

    master

    Vitis AI Runtime (VART) provides a unified high-level API for AI inference acceleration across both data center and embedded applications, enabling seamless cloud-to-edge deployment.

    Key features include:

    • Asynchronous Execution: Asynchronous submission and collection of jobs to/from the accelerator.
    • Multi-language Support: Implementations available in both C++ and Python.
    • Concurrency: Support for multi-threading and multi-process execution.

    In version 3.5, VART is fully open source, with the exception of the Python interfaces and DPUCADX8G interfaces. The runtime is composed of four core modules:

    1. unilog: Logging utility.
    2. xir: Xilinx Intermediate Representation.
    3. target_factory: Manages target hardware configurations.
    4. vart: The core runtime engine.

  10. Overview of Once-for-All (OFA) NAS Examples

    master

    The examples/ofa directory provides an end-to-end tutorial for implementing Neural Architecture Search (NAS) using the Once-for-All (OFA) technique. Unlike traditional NAS, OFA decouples the training and searching processes, allowing the search algorithm to execute very quickly.

    This tutorial covers the complete workflow:

    1. Architecture Search: Searching for a neural network architecture using OFA techniques.
    2. Quantization: Preparing the searched model for hardware.
    3. Compilation: Compiling the model for target hardware.
    4. Deployment: Deploying the final model on target platforms.

    Supported Design Spaces:

    • MobileNetv3
    • Proxyless (similar to MobileNetv2)
    • ResNet50

  11. Overview of Vitis AI

    master

    AMD Vitis™ AI is an Integrated Development Environment (IDE) designed to accelerate AI inference on AMD adaptable platforms, including AMD adaptable SoCs and Alveo Data Center accelerator cards.

    It provides a complete ecosystem for AI acceleration, including:

    • Optimized IP and tools
    • Libraries and models
    • Example designs and tutorials