Hailo Application Code Examples

The runtime/python directory contains several pre-built application examples for running AI models on Hailo hardware using Python. These examples cover various computer vision and audio tasks.

### Available Python Applications

| APP | Description |
|:---|:---|
| `object_detection` | Object detection and tracking with YOLO, SSD, and CenterNet |
| `instance_segmentation` | Instance segmentation with yolov5_seg and yolov8_seg |
| `lane_detection` | Lane detection using UFLDv2 |
| `pose_estimation` | Pose estimation with yolov8 |
| `speech_recognition` | Automatic speech recognition with the Whisper model |
| `super_resolution` | Super-resolution with espcnx4 and SRGAN |
| `oriented_object_detection` | Oriented object detection using YOLO11 OBB |

When using camera as the input source (-i camera), you can configure the following:

• Camera Index: Set via the CAMERA_INDEX environment variable (defaults to 0).
• Input Resolution (--camera-resolution): Choose from sd (640x480), hd (1280x720), or fhd (1920x1080).
• Framerate (-f, --framerate): Override the camera input framerate.
• Output Resolution (--output-resolution): Set the output size using sd|hd|fhd or custom dimensions (e.g., --output-resolution 1920 1080).

CAMERA_INDEX=1 ./super_resolution.py -n model.hef -i camera

./super_resolution.py -n ./real_esrgan_x2.hef -i camera -f 20

To run multiple models on the same virtual device (vdevice) efficiently, use the AsyncModelInfer constructor with a shared group_id. Assigning the same group_id to models from different HEF files allows the runtime to treat them as part of the same group, enabling resource sharing and improved efficiency.

Implementation Pattern:

std::string group_id = "<group_id>";
AsyncModelInfer model1("<hef1_path>", group_id);
AsyncModelInfer model2("<hef2_path>", group_id);

std::string group_id = "my_shared_group";
AsyncModelInfer model1("model1.hef", group_id);
AsyncModelInfer model2("model2.hef", group_id);

This example requires HEF files that contain HailoRT-Postprocess. Consequently, it only supports detection models compatible with this post-processing method, including:

• YOLO family: YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv10, YOLOv11
• YOLOX
• SSD
• CenterNet

This proof-of-concept demonstrates how a Hailo device (such as H8, H15, etc.) can offload the computationally intensive 2D-convolutional components of a 3D object detection network operating on point clouds.

In the PointPillars (PP) architecture, the 2D backbone and detection head are mapped to the Hailo device. This represents the majority of the required TOPS, as the 3D components are relatively minimal. While 3D parts can also be mapped to Hailo, doing so requires significant task-specific effort to maintain efficiency.

In this implementation:

• Hailo: Handles the 2D backbone and detection head.
• PyTorch/CPU: Handles pre- and post-computation using native OpenPCDet code.
• CUDA: Required for the 3D-NMS (Non-Maximum Suppression) operation.

This example supports pose estimation networks that allow HailoRT-Postprocess. The supported models are:

• yolov8m_pose
• yolov8s_pose

A full list of supported detection models is defined in networks.json within the application directory.

This example supports object detection networks that allow HailoRT-Postprocess. Supported architectures include:

• YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv9, YOLOv10, YOLOv11
• YOLOx
• SSD
• CenterNet

The full list of supported detection models is defined in networks.json within the application directory.

The DSP example demonstrates a multi-stage image preprocessing pipeline using the Hailo DSP. The pipeline chains three distinct operations to transform a raw RGB input into a processed NV12 output:

1. Crop and Resize with Letterbox: Takes a raw RGB input, crops the top-left quarter (from (0,0) to (input_width/2, input_height/2)), and resizes it to the target dimensions (e.g., 640×640) using letterboxing to preserve the aspect ratio.
2. Format Conversion: Converts the resulting RGB image into NV12 format.
3. Affine Rotation: Applies a 30-degree rotation to the NV12 image.

This sequence illustrates how to compose multiple DSP operations into a single processing flow.