DOCSEARCHbyAlgolia
Submit Docs

MaixPy (v4) Documentation

repository·main·Indexed 21 days ago

https://github.com/sipeed/maixpy

A Python-based framework for AI-powered edge computing projects optimized for MaixCAM, MaixCAM Pro, and MaixCAM2 hardware. It integrates hardware peripheral control (UART, I2C), video streaming, AI vision/audio algorithms, and LLM/VLM support. The ecosystem includes the MaixPy-UI-Lib for lightweight UI development, MaixVision Workstation, and MaixHub for AI model training and deployment.

Tokens
207.8K
Snippets
470
Records
855
Agent score
73%

What's inside MaixPy

  1. Overview of the MaixPy ecosystem

    main

    MaixPy is an edge computing framework designed for easy AI project creation using Python. The ecosystem includes:

    • MaixVision: A dedicated workstation/IDE to accelerate development.
    • MaixHub: An online platform for free AI model training and one-click deployment to MaixCAM hardware.
    • Hardware Platforms: Optimized for MaixCAM, MaixCAM-Pro, and MaixCAM2.
    • C/C++ SDK: For production deployment, a C/C++ SDK is available with identical APIs to the Python version.

  2. Overview of YOLO11 OBB Application

    main

    This application implements the YOLO11 OBB (Oriented Bounding Box) algorithm, designed for precise object recognition, confidence assessment, and angle detection in real-time video streams. It is optimized for hardware like MaixCAM and provides a visual interface for interaction.

    Key Features:

    • Real-time Image Acquisition: Automatic camera capture without manual triggers.
    • Oriented Object Detection: Uses the YOLO11n_obb model to detect targets with rotated bounding boxes, which better fits irregular or rotated objects compared to standard rectangular boxes.
    • Visualized Detection: Displays target category, confidence (2 decimal places), and rotation angle (1 decimal place) in real-time, with red lines drawn for the oriented bounding boxes.
    • Touch Interaction: Includes a visual 'back' button on the screen for quick exit via touch.
    • Error Handling: Automatically captures and displays detailed error messages if an exception occurs during runtime.

  3. Overview of app_speech for MaixCAM2

    main

    The app_speech application is a speech-to-text tool developed for the MaixCAM2 hardware. It utilizes the SenseVoice speech recognition model to convert recorded audio into text.

    Key Features:

    • Voice Recording: Triggered by long-pressing a specific button; recording stops when the button is released.
    • Speech-to-Text: Automatically invokes the ASR (Automatic Speech Recognition) model upon stopping the recording.
    • Visual Interface: Provides real-time status updates (e.g., "Recording..", "Transcribing...") and displays the final transcription results.
    • Hardware Requirement: Specifically designed for the 4GB version of MaixCAM2 hardware.

  4. Overview of the Hand Gesture Classifier System

    main

    The Hand Gesture Classifier is a real-time embedded system designed for hand pose recognition and classification. It uses a camera to capture hand images, extracts 21 core hand keypoints using a pre-trained model, and performs classification using a Linear Support Vector Machine (LinearSVC).

    Key Features:

    • Real-time Detection: Automatically identifies hand regions and visualizes the 21-point skeletal structure.
    • Multi-Gesture Support: Recognizes up to 14 gestures, including numbers (1, 3, 4, 5, 6, 9, etc.) and command gestures (Like, OK, Heart, I Love You).
    • Dual Classification Modes: Supports a 'Simplified Mode' (4 gesture classes) for faster recognition and a 'Full Mode' (14 gesture classes) for broader coverage.
    • On-device Processing: All computation is performed locally on the device, ensuring low latency and high real-time performance without cloud dependency.

  5. Overview of Human Pose Classifier App

    main
    The Human Pose Classifier is a real-time system based on YOLO11 vision that detects human body keypoints and automatically classifies poses (e.g., standing, sitting, squatting, lying down). It provides real-time visual feedback including detection bounding boxes, skeletal keypoint connections, pose classification labels, and confidence scores.

  6. Overview of Human Pose Detection App

    main
    The app_human_pose application runs on MaixPy and implements human pose detection using either YOLOv8-Pose or YOLO11-Pose models. It detects 17 keypoints on the human body, captures real-time camera footage, and visualizes the results. It is designed for rapid human pose capture and simple motion analysis scenarios.

  7. Overview of the Offline Voice Interaction Application

    main
    The app_chat is an all-offline voice interaction application designed for MaixCam2 devices. It implements a complete local pipeline: Voice Input → Transcription → Intelligent Response without requiring an internet connection. It is intended for scenarios like quick on-site queries and offline voice assistants.

  8. Overview of Hand Landmarks Application

    main

    The Hand Landmarks application is a tool developed for Maix series hardware designed for real-time hand detection and keypoint localization. It visualizes hand morphology, keypoint trajectories, and hand angles, making it suitable for hand motion analysis and simple gesture interaction.

    Key Features

    • Real-time Hand Detection: Automatically identifies hands and displays category/confidence levels.
    • Keypoint & Skeleton Visualization: Draws precise keypoints and skeletal connections.
    • Trajectory Tracking: Records the last 10 frames of keypoint positions, including dual fingertip connection lines.
    • Angle Display: Shows real-time hand angles in degrees.
    • Model Switching: Allows rapid switching between different hand detection models to balance performance and accuracy.
    • Quick Exit: Provides a one-touch exit function.

  9. Overview of MaixCam Performance Benchmark App

    main

    The Performance Benchmark App is a tool designed for MaixCAM, MaixCAM Pro, and MaixCAM2 devices to evaluate hardware capabilities, verify firmware, and benchmark algorithms. It features a visual interface to load and execute pre-set test cases, providing real-time previews and automatic result saving.

    It functions as a benchmark framework that dynamically loads test modules from the benchmarks/ directory.

  10. Overview of the App VLM tool for MaixCam

    main

    App VLM is a multimodal application designed for MaixCam devices. It allows users to load lightweight Vision-Language Models (VLMs) to perform 'image-to-speech' tasks. The application captures images via the device's built-in camera and uses the selected model to generate natural language descriptions, which are then displayed on the device screen.

    Key Features:

    • Model Support: Choose between Qwen3-VL, InternVL, and SmolVLM.
    • Multilingual Output: Supports switching between Chinese (ZH) and English (EN) for compatible models.
    • Touchscreen Interaction: Full control via the touchscreen without needing a command line.
    • Automatic Environment Check: Validates memory availability and model presence before running.

  11. Overview of Face Emotion Recognition App

    main

    The app_face_emotion program implements real-time face capture and emotion state determination using a YOLOv8 face detection model and a specialized face emotion classification model. It is designed for quick verification of face emotion scenarios and simple human-machine interaction assistance.

    Key Features

    • Real-time Face Detection: Uses a lightweight YOLOv8 model to identify and locate multiple faces simultaneously from a camera feed.
    • Face Emotion Classification: Crops and preprocesses detected faces to output emotion labels and confidence scores.
    • Visual Results:
      • Displays a full confidence progress bar for the first detected face.
      • Annotates all detected faces with bounding boxes and the best-matching emotion label.
      • Color Coding: High confidence results are marked in green, while low confidence results are marked in red.
    • Touchscreen Interaction: Supports model switching and exiting via the touchscreen interface.

  12. Overview of Human Pose Detection Application

    main

    The app_human_pose application for MaixPy implements real-time human pose detection using YOLOv8-Pose or YOLO11-Pose models. It identifies 17 specific human key points and visualizes them by drawing bounding boxes, class labels, confidence scores, and connecting lines between body points on the real-time camera feed.

    Hardware Compatibility

    • MaixCam2: Uses the YOLO11 model.
    • MaixCam/Pro: Uses the YOLOv8 model.