DOCSEARCHbyAlgolia
Submit Docs

Voyager SDK

repository·latest·Indexed 40 days ago

https://github.com/axelera-ai-hub/voyager-sdk

A high-performance toolkit for building and deploying machine learning inference applications on Axelera AI Metis hardware. It provides multiple abstraction layers, including the high-level InferenceStream API, the low-level axelera.runtime API, and the Pipeline Builder for rapid prototyping. The SDK includes a compiler for converting ONNX models to int8 binaries, a Model Zoo of pre-trained models, and tools for measuring throughput and accuracy on the Metis AIPU.

Tokens
176K
Snippets
458
Records
714
Agent score
51%

What's inside Voyager SDK

  1. Overview of the Voyager SDK Compiler

    latest

    The Voyager SDK Compiler provides tools and APIs to convert custom ONNX models into int8 binaries optimized for Metis hardware (AIPU). The compilation process transforms floating-point models into quantized, hardware-optimized formats suitable for deployment on Metis devices.

    Key components include:

    • Compiler CLI: A command-line tool for compilation, quantization, and calibration.
    • Compiler Python API: A programmatic interface for the two-step quantize → compile workflow.
    • Compiler Configuration: Settings for managing multi-core compilation modes (Batch-1 vs Batch-4) and resource allocation.
    • ONNX Operator Support: Guidance on which operators are accelerated by the AIPU (opsets 14–17) versus those that fall back to the CPU.

  2. Overview of Voyager SDK Reference Layers

    latest

    The Voyager SDK technical reference is organized into several functional layers that allow developers to manage the full lifecycle of ML deployment on Metis devices. Depending on your task, you will interact with different layers:

    • Tools: CLI utilities for installation, device management, and model execution (install.sh, axdevice, inference.py, deploy.py, axrunmodel).
    • Pipeline: Details on pipeline structure, model file formats, and the GStreamer operator reference.
    • APIs: Python interfaces including the high-level InferenceStream, the low-level axelera.runtime, and the Pipeline Builder.
    • Pipeline Builder: A Pythonic API for composing ML pipelines using axelera.runtime.op.
    • Models: Access to the Model Zoo and dataset adapters.
    • Compiler: Tools for model compilation via CLI or Python API, multi-core configuration, and ONNX support.
    • Measurement: Tools for performance metrics, bottleneck analysis, and accuracy.
    • System: Hardware architecture, virtual environments, environment variables, and thermal management.

  3. Overview of Voyager SDK

    latest
    The Voyager SDK is a production-ready toolkit designed for building high-performance inferencing applications on Axelera AI Metis devices. It provides multiple layers of abstraction for deploying models, managing pipelines, and integrating AI inference into applications via Python and C++ APIs.

  4. Overview of the Voyager SDK tutorial series

    latest

    The tutorial series is designed to take you from a raw model to a production-ready, optimized pipeline on Metis hardware. The learning progression covers:

    1. Model Deployment: Wrapping models in types.Model and implementing preprocess functions.
    2. Pipeline Inspection & Performance: Implementing postprocessing via AxOperator, inspecting CPU/AIPU hybrid pipelines, and measuring Model FPS.
    3. Inference Integration: Using AxTaskMeta, application.py, and FrameResult to integrate results into applications.
    4. Data & Accuracy: Using types.DataAdapter for datasets and measuring accuracy with evaluators.
    5. GStreamer Pipelines: Building low-code, high-performance end-to-end pipelines using AxOperator.
    6. C/C++ Decoders: Writing custom decoders that output AxMeta format and integrating them via make operators.
    7. Custom Metadata: Defining new AxTaskMeta structures for specialized vision tasks.
    8. Custom Metrics: Implementing types.Evaluator and types.EvalResult for custom accuracy measurement.
    9. Cascade Pipelines: Implementing detection-then-classification workflows (ROIs).
    10. Advanced Topics (Upcoming): Multi-Object Tracking (MOT), custom PreprocessOperator for YAML pipelines, and advanced AxTaskMeta visualization/object-views.

  5. Overview of Voyager SDK Command-line Tools

    latest

    The Voyager SDK provides several command-line tools for managing the full lifecycle of AI models on Metis hardware:

    • install.sh: SDK installer. Sets up the Python virtual environment and downloads model dependencies.
    • axdevice: Device management. List connected Metis devices, show firmware versions, and manage device configuration.
    • inference.py: Primary inference tool. Used for running models, displaying results, and general benchmarking.
    • deploy.py: Model deployment. Compiles and deploys models based on a YAML definition; essential for custom weights and new architectures.
    • axrunmodel: Advanced benchmarking. Runs a compiled model directly against the AIPU to measure peak hardware throughput.

  6. Overview of the Pipeline Builder API

    latest

    The Pipeline Builder API is a Pythonic interface designed for building ML inference pipelines on the Axelera Metis AIPU. It provides a programmatic alternative to YAML-based pipeline configurations by using composable Python operators.

    Note on Naming Conventions: Most Operator classes support both PascalCase and snake_case aliases. For example:

    • op.Seq can be used as op.seq
    • op.CenterCrop can be used as op.center_crop

  7. Overview of the Pipeline Builder API

    latest

    The Pipeline Builder API (currently in Alpha) allows you to construct complex AI workflows by composing various operators.

    Stability Status:

    • Stable: Core operators including detection, classification, pose, segmentation, and tracking.
    • In Development: Cascade operations (such as op.for_each and op.crop_roi) and streaming APIs.

    To build a complete pipeline, you will typically follow these steps:

    1. Model Compilation: Compile your model into the .axm format.
    2. Pipeline Construction: Use operators to define preprocessing, inference, and postprocessing.
    3. Execution: Run the pipeline on Metis devices.

  8. Use `axelera.runtime.op.transforms` for image preprocessing

    latest
    The axelera.runtime.op.transforms module provides a suite of image transformation operations used for preprocessing data before it is passed to a model. These operations include Resize, CenterCrop, Letterbox, and Normalize. These transforms are essential for ensuring input data matches the expected dimensions and format of specific neural network architectures (e.g., adjusting resolution or aspect ratios).

  9. Explore Axelera AI demo folders

    latest

    The examples/demos/ directory contains specific demo implementations used for conferences and onboarding. Each demo is contained within its own subdirectory and includes its own specific documentation and setup instructions.

    Key demos available include:

    • Fruit demo: Located in fruit_demo.
    • 8k demo: Located in 8k_demo.

  10. Explore Voyager SDK code examples

    latest

    The SDK provides several standalone, runnable scripts categorized by language to serve as starting points for application development.

    Python Examples

    • Basic Application: A minimal detection loop with display (the "hello world" of Voyager).
    • Extended Application: Includes hardware capability checks, frame rate control, and temperature monitoring.
    • Tensor Access: Demonstrates direct tensor output for custom postprocessing.
    • Classification: Image classification using generator-based input.
    • Multiple Pipelines: Running several models concurrently.
    • Cross-Line Counter: Vehicle counting utilizing tracker metadata.
    • Remote Monitor: A TCP broadcast server for remote cross-line monitoring.

    C++ Examples (AxInferenceNet API)

    • Basic Inference: A standard C++ detection loop using the AxInferenceNet API.
    • Cascaded Pipeline: A multi-model C++ pipeline (e.g., detect $\rightarrow$ classify).
    • Tensor Access: Accessing raw tensor output in C++ for custom processing.

  11. Use data types in `axelera.runtime.op.types`

    latest
    The axelera.runtime.op.types module defines the standard data structures used for operation outputs within the Voyager SDK pipeline. These types represent common computer vision outputs such as bounding boxes, detected objects, and classifications. Use these types to ensure type safety and consistent data handling when consuming outputs from various pipeline operations.