DOCSEARCHbyAlgolia
Submit Docs

Hailo-Apps Documentation

repository·main·Indexed 17 days ago

https://github.com/hailo-ai/hailo-apps

A collection of high-performance AI applications and pipelines optimized for Hailo accelerators, including Hailo-8 and Hailo-10H. The documentation provides a User Guide for installing and running pre-built GenAI, GStreamer, and standalone apps, a Developer Guide for building custom applications using the Python framework and C++ post-processing, and a detailed reference for the hailo_apps.config module and its Configuration Manager CLI.

Tokens
106.6K
Snippets
290
Records
438
Agent score
66%

What's inside hailo-apps

  1. Overview of Pose Estimation Application

    main

    This application performs pose estimation using Hailo-8, Hailo-8L, or Hailo-10H devices. It takes an input (image, video, or live camera), performs inference using a provided HEF file, and draws detection boxes, class types, confidence scores, keypoints, and joint connections on the output image.

    Supported Input Formats:

    • Images: .jpg, .jpeg, .png, .bmp
    • Video: .mp4
    • Live Camera Feed: USB cameras, Raspberry Pi cameras, or specific device paths (e.g., /dev/video0).

    Supported Models: This application requires pose estimation networks that support HailoRT-Postprocess. Supported models include:

    • yolov8m_pose
    • yolov8s_pose

  2. Overview of Hailo-Apps

    main

    Hailo-Apps provides high-performance AI applications designed for Hailo accelerators. The repository includes GStreamer pipelines, GenAI assistants, and standalone applications written in C++ and Python.

    Application Types

    • GenAI Apps: Optimized for LLM, VLM, and speech workflows on the Hailo-10H accelerator. Located in hailo_apps/python/gen_ai_apps/.
    • Pipeline Apps: Designed for real-time camera, RTSP, or video processing using GStreamer. Located in hailo_apps/python/pipeline_apps/.
    • Standalone Apps: Minimal per-app installs focused on HailoRT learning. Located in hailo_apps/python/standalone_apps/ and hailo_apps/cpp/.

  3. Overview of Paddle OCR Application

    main

    The Paddle OCR application performs end-to-end text detection and recognition using the PaddleOCR pipeline. It is accelerated by Hailo-8, Hailo-8L, or Hailo-10H devices.

    Capabilities:

    • Processes images, videos, folders, or camera input.
    • Detects text regions and decodes text using OCR models.
    • Optional FPS performance display.
    • Optional saving of inference output.

    Supported image formats: .jpg, .jpeg, .png, or .bmp.

  4. Overview of Python Standalone Examples

    main

    The standalone_apps directory contains several Python-based applications designed to run independently for specific AI tasks on Hailo hardware. These examples demonstrate various computer vision and audio capabilities including detection, segmentation, and recognition.

    | APP                             | Description                                                        |
|:--------------------------------|:-------------------------------------------------------------------|
| `object_detection`              | Object detection and tracking with YOLO, SSD, and CenterNet        |
| `instance_segmentation`         | Instance segmentation with yolov5_seg and yolov8_seg               |
| `lane_detection`                 | Lane detection using UFLDv2                                        |
| `pose_estimation`                | Pose estimation with yolov8                                        |
| `super_resolution`              | Super-resolution with espcnx4 and SRGAN                            |
| `paddle_ocr`                    | Text detection and recognition with PaddleOCR                      |
| `oriented_object_detection`     | Oriented object detection using YOLO11 OBB                         |
| `speech_recognition`            | Speech recognition for Hailo-8/8L/10H                              |
| `yolo26`                        | YOLO26 pose estimation/object detection with ONNX postprocessing   |

  5. Overview of C++ AI Applications

    main

    The hailo-apps repository provides several C++ based AI applications currently in Beta. These applications demonstrate various computer vision tasks optimized for Hailo hardware.

    Note: Because these are in beta, features and APIs are subject to change.

    Available C++ applications include:

    • classification: Image classification using ImageNet-trained models.
    • depth_estimation: Depth estimation using scdepthv3 and stereonet models.
    • instance_segmentation: Instance segmentation using yolov5_seg and yolov8_seg models.
    • object_detection: Generic and asynchronous object detection.
    • onnxruntime: Inference on Hailo devices with post-processing handled via ONNXRuntime.
    • pose_estimation: Pose estimation using yolov8.
    • semantic_segmentation: Semantic segmentation using Resnet18_fcn (trained on Cityscapes dataset).
    • zero_shot_classification: Zero-shot classification using clip_vit_l14.

  6. Overview of Hailo GenAI Utilities

    main

    The hailo_apps.python.gen_ai_apps.gen_ai_utils package provides shared utilities for building Generative AI applications on Hailo platforms. It is designed to encapsulate common functionality required for voice-enabled AI and Large Language Model (LLM) interactions.

    The package is divided into two primary modules:

    • voice_processing: Manages audio I/O, microphone recording, Speech-to-Text (Whisper), and Text-to-Speech (Piper TTS).
    • llm_utils: Manages LLM context windows, message formatting, response streaming, and terminal-based user interfaces.

  7. Overview of Developer Guides for Hailo Applications

    main

    The Hailo Developer Guide provides technical documentation for building custom AI applications, extending existing ones, and managing the AI lifecycle on the Hailo platform. Key developer tasks include:

    • Application Development: Building new applications using the Python framework.
    • Custom Post-Processing: Writing C++ post-processing functions for unsupported neural networks.
    • Model Management: Retraining models for specific use cases.
    • Pipeline Debugging: Using tools like GST Shark to debug GStreamer pipelines.
    • GStreamer Integration: Utilizing helper functions in the gstreamer_helper_pipelines.py module to simplify pipeline construction.

  8. Explore Hailo GenAI Applications and Resources

    main

    The gen_ai_apps directory contains various Generative AI applications and shared utilities for building AI-powered tools. You can explore specific application guides and utility documentation using the following resources:

    Application Guides

    • Agent Tools Example: A complete guide for the agent application (agent_tools_example/README.md).
    • VLM Chat: Documentation for Vision Language Model chat (vlm_chat/README.md).
    • Voice Assistant: Documentation for the voice assistant application (voice_assistant/README.md).
    • Simple LLM Chat: Guide for a basic LLM chat example (simple_llm_chat/README.md).
    • Simple VLM Chat: Guide for a basic VLM chat example (simple_vlm_chat/README.md).
    • Simple Whisper Chat: Guide for a basic Whisper chat example (simple_whisper_chat/README.md).
    • Hailo Ollama: Integration guide for Hailo Ollama (hailo_ollama/README.md).

    Shared Utilities

    • GenAI Utils: General documentation for shared utilities (gen_ai_utils/README.md).
    • Voice Processing: Documentation for the voice processing module (gen_ai_utils/voice_processing/README.md).

  9. Understand the Hailo-Apps repository structure

    main

    The Hailo-Apps repository is organized into several key directories that separate documentation, application logic, and system resources:

    • hailo_apps/: The core Python package. It contains python/pipeline_apps/ (GStreamer-based CLI tools like hailo-detect), python/gen_ai_apps/ (Generative AI apps like vlm_chat), python/standalone_apps/ (Python-based CV apps), and python/core/ (shared utilities and GStreamer components).
    • doc/: Contains user and developer guides.
    • scripts/: Contains shell scripts for installation and environment management.
    • resources/: A symbolic link to /usr/local/hailo/resources containing shared models and videos.
    • venv_hailo_apps/: The local Python virtual environment (generated during installation).
    • config/: Top-level configurations used by installers.

  10. Access the Hailo-Apps User Guide

    main
    The Hailo-Apps User Guide provides instructions for end-users to run pre-built applications and configure them. The guide is organized into several specialized modules covering installation, execution, repository structure, configuration, parallel application management, and AI-assisted development.

  11. Access Hailo Applications documentation

    main

    The Hailo Applications repository provides documentation organized into three main categories depending on your role:

    1. User Guide: For end-users looking to install and run pre-built AI applications. Includes installation, running applications, repository structure, and config.yaml configuration.
    2. Developer Guide: For developers building on the framework. Includes application development (Python framework), GStreamer helper pipelines, C++ post-processing, model retraining, and debugging with GST Shark.
    3. Applications Guide: Specific references for individual applications, categorized into Pipeline apps, GenAI apps, and Standalone apps.

  12. Run Oriented Object Detection with YoloV11 OBB

    main

    This application performs oriented object detection (OBB) using a YoloV11 model on Hailo-8, Hailo-8L, or Hailo-10H devices. It accepts HEF models and various input sources (images, video, or cameras) and outputs annotated media with rotated bounding boxes. This is ideal for aerial imagery, document analysis, or any scenario where objects appear at arbitrary angles.

    Supported Models: Currently supports only YoloV11-OBB models.

    python ./oriented_detection.py -n <model_path> -i <input_image_path> -l <label_file_path> -b <batch_size>