DOCSEARCHbyAlgolia
Submit Docs

ESP-WHO Image Processing Platform

repository·master·Indexed 24 days ago

https://github.com/espressif/esp-who

An image processing development platform for Espressif chips that provides examples for face detection, recognition, and pedestrian detection using ESP-DL. It supports hardware such as ESP-EYE, ESP32-S3-EYE, ESP-S3-KORVO-2, and ESP32-P4-Function-EV-Board, and is compatible with ESP-IDF release versions v5.4, v5.5, v6.0, and v6.1.

Tokens
6.3K
Snippets
12
Records
39
Agent score
82%

What's inside ESP-WHO

  1. Overview of ESP32-S3-EYE v2.2

    master

    The ESP32-S3-EYE is an AI development board designed for image recognition and audio processing using the ESP-WHO framework. It is based on the ESP32-S3 SoC and features a 2-Megapixel camera (OV2640), an LCD display, and a digital I2S MEMS microphone.

    Key Hardware Specifications:

    • SoC: ESP32-S3 (supports Wi-Fi and Bluetooth 5 LE).
    • Memory: 8 MB Octal PSRAM and 8 MB Flash.
    • Camera: OV2640 (2MP, 66.5° FOV, max resolution 1600x1200).
    • Display: 1.3" LCD connected via SPI.
    • Audio: Digital I2S MEMS microphone.
    • Sensors: Three-axis accelerometer (QMA7981).
    • Storage: MicroSD card slot.
    • Connectivity: Micro-USB port for 5V power and communication via GPIO19 and GPIO20.

    Note: ESP32-S3-EYE has reached End of Life (EOL). This guide is for reference only. For new designs, consider other Espressif AI development boards.

  2. Overview of ESP-WHO

    master

    ESP-WHO is an image processing development platform based on Espressif chips. It provides practical development examples for tasks such as face detection, face recognition, pedestrian detection, and QR code recognition. The platform is built upon ESP-DL and can be integrated with various peripherals to implement complex vision-based applications.

    Key features include:

    • Support for the ESP32-P4 chip.
    • Asynchronous execution of camera and deep learning models for higher frame rates.
    • Support for LVGL (Light and Versatile Graphics Library) for custom GUI development.
    • New pedestrian detection models.

  3. Hardware Overview of ESP-EYE

    master

    ESP-EYE is an ESP32-based development board designed for AI applications such as face detection, face recognition, and speech recognition.

    Key Hardware Specifications:

    • Processor: ESP32 chip (2.4 GHz Wi-Fi and Bluetooth combo).
    • Memory: 8 MB PSRAM and 4 MB Flash.
    • Audio: Integrated digital microphone (MIC).
    • Vision: External 2-Megapixel camera module via a dedicated camera connector.
    • Connectivity: Wi-Fi and Bluetooth; supports external IPEX antenna or internal 3D PIFA antenna.
    • USB Interface: Micro USB port for power and debugging via the CP2102 USB-to-UART chip.
    • User Interface: Side tactile button (function key) and a top tactile button (Reset/Boot).

  4. Compare ESP32-S3-EYE with ESP-EYE

    master

    If you are transitioning from the older ESP-EYE (ESP32-based) to the ESP32-S3-EYE, note the following key differences:

    FeatureESP32-S3-EYEESP-EYE
    ChipESP32-S3ESP32
    PSRAM8 MB Octal PSRAM8 MB Quad PSRAM
    Flash8 MB flash4 MB flash
    LCD displayYesNone
    AccelerometerYesNone
    Power SupplyExternal battery (optional)None
    USB-to-UARTIntegrated via ESP32-S3 USB Serial/JTAGRequires external bridge
    AntennaIntegrated in ESP32-S3-WROOM-1External connector

  5. Hardware Reference: ESP32-S3-EYE GPIO and Power

    master

    GPIO Allocation

    Most GPIOs of the ESP32-S3-WROOM-1 module are pre-allocated to control specific components or functions on the board.

    • GPIO3: Can be used to configure LED statuses.
    • For custom pin configurations, refer to the official schematics.

    Power Supply Design

    The board uses independent power supplies for the camera and the remaining components to reduce audio signal noise and improve component performance.

  7. Hardware Reference: GPIO and Power

    master

    GPIO Allocation

    Most GPIOs on the ESP32-S3-WROOM-1 module are used for controlling board components or functions. GPIO3 is specifically available for configuring LED combination states. For custom pin configurations, refer to the provided schematics.

    Power Circuitry

    The ESP32-S3-EYE features independent power supply circuits for the camera and the ESP32-S3-WROOM-1 module/other components. This design reduces audio signal interference from the module circuitry on the camera circuit, improving overall performance.

  9. Use touch screen buttons on ESP32-P4-Function-EV-Board for Face Recognition

    master

    On the ESP32-P4-Function-EV-Board, use the touch screen buttons to control the face recognition process:

    • recognize: Trigger face recognition.
    • enroll: Enroll a new face.
    • delete: Delete the last enrolled feature.
    | btn       | operation         |
|-----------|-------------------|
| recognize | recognize         |
| enroll    | enroll            |
| delete    | delete last feat  |

  10. Build, Flash, and Monitor ESP-EYE (Linux Example)

    master

    Follow these steps to flash a program to the ESP-EYE board using a Linux environment.

    1. Connect the board: Plug the ESP-EYE into your PC via Micro USB.
    2. Identify the device: Run ls /dev/ttyUSB* to find the device path (e.g., /dev/ttyUSB0).
    3. Navigate to an example: cd esp-who/examples/single_chip/recognition_solution
    4. Configure the project:
      • Run make defconfig for default configuration.
      • Run make menuconfig to set the Default serial port under Serial flasher config (use the path found in step 2, e.g., /dev/ttyUSB0).
    5. Flash the software: Run make flash.
    6. Monitor logs: Run make monitor to view real-time logs (Note: this will restart the board).
    # 1. Check device
ls /dev/ttyUSB*

# 2. Navigate to example
cd esp-who/examples/single_chip/recognition_solution

# 3. Configure and Flash
make defconfig
make menuconfig
make flash

# 4. Monitor logs
make monitor

  11. Set up the ESP-WHO environment variable

    master

    To use ESP-WHO examples, you must set the IDF_EXTRA_ACTIONS_PATH environment variable to point to the tools/ directory within your ESP-WHO repository. This is required for the build system to function correctly.

    # Linux
export IDF_EXTRA_ACTIONS_PATH=/path_to_esp-who/tools/
echo $IDF_EXTRA_ACTIONS_PATH

# Win/powershell
$Env:IDF_EXTRA_ACTIONS_PATH="/path_to_esp-who/tools/"
echo $Env:IDF_EXTRA_ACTIONS_PATH

# Win/cmd
set IDF_EXTRA_ACTIONS_PATH=/path_to_esp-who/tools/
echo %IDF_EXTRA_ACTIONS_PATH%

  12. Connect a camera module to ESP-EYE

    master

    When using the ESP-EYE development board with a camera module, use the following pin mapping for the interface connections. Note that if you are using a different ESP32 module, the pin mapping will vary.

    | Interface | Camera Pin | ESP-EYE |
| :--- | :---: | :---: |
| SCCB Clock | SIOC | IO23 |
| SCCB Data | SIOD | IO18 |
| System Clock | XCLK  | IO4 |
| Vertical Sync | VSYNC | IO5 |
| Horizontal Reference | HREF | IO27 |
| Pixel Clock | PCLK | IO25 |
| Pixel Data Bit 0 | D2 | IO34 |
| Pixel Data Bit 1 | D3 | IO13 |
| Pixel Data Bit 2 | D4 | IO14 |
| Pixel Data Bit 3 | D5 | IO35 |
| Pixel Data Bit 4 | D6 | IO39 |
| Pixel Data Bit 5 | D7 | IO38 |
| Pixel Data Bit 6 | D8 | IO37 |
| Pixel Data Bit 7 | D9 | IO36 |
| Camera Reset | RESET | -1 |
| Camera Power Down | PWDN | -1 |
| Power Supply 3.3V | 3V3 | 3v3 |
| Ground | GND | GND |