Hailo Raspberry Pi 5 Examples

The Detection Cropper application demonstrates a cascading TAPPAS pipeline using the Hailo cropper pipeline element. The pipeline follows this logic:

1. Object Detection: Uses a YOLO network to detect objects.
2. Cropping: If a person is detected, the application crops the bounding box of that person.
3. Depth Estimation: The cropped image is passed to a depth estimation network (supporting scdepthv3 relative depth).

This project also demonstrates how to modify and compile C++ post-processing scripts used to control how the hailocropper crops detections, which is a common pattern for native C++ post-processing in Hailo pipelines.

Additional community projects utilizing CLIP can be found in the hailo-CLIP repository:

• Ad Genie: A system that personalizes advertisements by matching outfits to individual styles in real time.
• BAIby Monitor: A smart baby monitor that detects cries and activity with real-time alerts.

The community_projects directory contains various examples of AI applications built using Hailo hardware and Raspberry Pi. These projects range from interactive games and robotics to biofeedback and smart monitoring systems.

To contribute your own creation, follow the guidelines in How to Add a Community Project.

ChessMate is an automated chess-playing system that integrates AI, robotics, and computer vision. The system architecture consists of the following components:

• Vision & AI: A USB camera captures images of the chessboard. These images are cropped into 64 individual cells and processed by the Xception neural network running on the Hailo8 AI accelerator to identify chess piece positions.
• Chess Engine: The detected board state is passed to Stockfish, which runs on the Raspberry Pi 5 to calculate the optimal move.
• Robotics: A 3D-printed, six-servo robotic arm controlled by an ESP32 chip executes the moves. The arm is controlled independently via HTTP over Wi-Fi.

To manage user-specific data and state across multiple frames, you should create a custom class that inherits from app_callback_class (provided by the hailo_apps_infra package).

This class is used to:

• Maintain state (e.g., frame counts).
• Manage frame data.
• Handle custom logic that needs to persist across the pipeline lifecycle.

Hailo pipelines use a callback method to process data from the GStreamer pipeline. A callback function is triggered whenever data is available from the pipeline. This function is responsible for extracting relevant information (like bounding boxes or keypoints), performing actions (like drawing on frames), or printing data to the terminal.

Critical Performance Note: The callback function is blocking. If the execution takes too long, the entire pipeline will stall. If you need to perform heavy processing, you must offload the data to a separate process (e.g., using a class like WLEDDisplay that runs its own process) to allow the pipeline to continue running smoothly.

The application uses a multi-process architecture to maintain high performance by separating heavy AI processing from game logic:

1. Main Process (Pose Estimation): Handles the GStreamer pipeline, runs the Hailo pose estimation model, extracts hand positions (left and right wrist landmarks), and injects fruit positions back into the video stream as Hailo detections via HailoOverlay.
2. Pygame Process (Game Logic): Runs the fruit physics simulation, collision detection, score management, and game rendering.

Communication Flow:

• Pose Estimation $\rightarrow$ Hand Positions (via queue) $\rightarrow$ Game Logic $\rightarrow$ Fruit Positions (via queue) $\rightarrow$ Hailo Detections $\rightarrow$ Video Overlay.

The hailo-rpi5-examples repository uses the Hailo Apps Infra repository as a core dependency.

• Hailo Apps Infra: Provides the underlying infrastructure, tools, and a simple API for creating custom pipelines and applications. It is installed as a pip package.
• hailo-rpi5-examples: Provides concrete, runnable demonstrations of AI capabilities built on top of that infrastructure.

The TAILO system tracks the following pet states during operation:

• PET_HOMING: Pet is not in the frame.
• PET_NOT_CENTERED: Pet is in the frame, but not centered.
• PET_ON_COUCH: Pet is sitting on the couch.
• PET_LOCKED: The camera is actively following the pet.

To set up and run a project following the template structure, you typically need to install Python dependencies and download required model resources or assets.

Example setup steps:

pip install -r requirements.txt
./download_resources.sh

Example execution:

python template_example.py

Before running examples, you must configure your environment and install dependencies. Perform these steps in order:

1. Source the environment: This sets required environment variables and activates the Hailo virtual environment.

   source setup_env.sh

2. **Install Python requirements**:
   ```bash
pip install -r requirements.txt

Note: If rapidjson-dev is missing, install it via sudo apt install -y rapidjson-dev. 3. Download resources: Download the necessary models and files.

./download_resources.sh

To download all available models, use the --all flag:

./download_resources.sh --all

To achieve optimal performance, you must manually set the PCIe speed to Gen3 if you are using the M.2 HAT (the AI HAT auto-detects Gen3).

1. Open the configuration tool: sudo raspi-config.
2. Navigate to 6 Advanced Options -> A8 PCIe Speed.
3. Select Yes to enable PCIe Gen 3 mode.
4. Finish and reboot the system.

sudo raspi-config
# Select: 6 Advanced Options -> A8 PCIe Speed -> Yes

sudo reboot