ESP32 Camera Driver

repository·master·Indexed 25 days ago

https://github.com/espressif/esp32-camera

Driver providing support for various image sensors on ESP32, ESP32-S2, and ESP32-S3 SoCs. It includes tools for converting captured frame data into formats like BMP and JPEG, and supports a wide range of sensors including OV2640, OV3660, OV5640, and others. Compatible with ESP-IDF and Arduino environments, the driver features support for MJPEG streaming over HTTP and autofocus capabilities for OV5640 modules.

Tokens
2.1K
Snippets
6
Records
10
Agent score
33%

What's inside esp32-camera

  1. Install the ESP32 Camera Driver for ESP-IDF

    master

    To use the driver in an ESP-IDF project, add the espressif/esp32-camera component as a dependency. You can do this via the CLI or by manually editing your idf_component.yml file.

    Requirements:

    • Enable PSRAM in menuconfig.
    • Set both Flash and PSRAM frequencies to 80MHz.
    • Include esp_camera.h in your source code.

    These steps also apply to PlatformIO projects using framework=espidf.

    idf.py add-dependency "espressif/esp32-camera"
  2. Create a project from the camera example

    master

    You can quickly set up a working project by downloading the pre-configured camera_example using the ESP-IDF command line. This example includes the necessary menuconfig settings for the camera driver.

    idf.py create-project-from-example "espressif/esp32-camera:camera_example"
  3. Use Autofocus with OV5640

    master

    For OV5640 modules with AF-capable lenses, you can enable autofocus via menuconfig (Component configCamera configurationEnable autofocus (OV5640)).

    1. Include #include "esp_camera_af.h".
    2. Get the sensor instance using esp_camera_sensor_get().
    3. Initialize AF with esp_camera_af_init().
    4. Trigger AF with esp_camera_af_trigger() or wait for completion with esp_camera_af_wait().
    #include "esp_camera.h"
    #include "esp_camera_af.h"
    
    // After esp_camera_init(...)
    sensor_t *s = esp_camera_sensor_get();
    
    esp_camera_af_config_t af_cfg = {
        .mode = ESP_CAMERA_AF_MODE_AUTO,
        .timeout_ms = 2000,
    };
    
    ESP_ERROR_CHECK(esp_camera_af_init(s, &af_cfg));
    
    // Trigger a single AF cycle
    ESP_ERROR_CHECK(esp_camera_af_trigger(s));
    
    // Wait for completion
    esp_camera_af_status_t st;
    ESP_ERROR_CHECK(esp_camera_af_wait(s, 0, &st));
  4. Capture and stream JPEG via HTTP

    master

    The driver supports capturing single JPEG images or streaming a continuous MJPEG stream over HTTP.

    • For single captures: Use httpd_resp_send() with the buffer from esp_camera_fb_get().
    • For MJPEG streaming: Use httpd_resp_set_type() with multipart/x-mixed-replace;boundary=... and send chunks containing the boundary, headers, and JPEG data in a loop.
  5. Important Hardware and Performance Considerations

    master

    When using the ESP32 Camera Driver, keep the following in mind:

    • PSRAM Requirement: Except when using CIF or lower resolution with JPEG, the driver requires PSRAM to be installed and activated.
    • YUV/RGB Performance: Writing YUV or RGB data to PSRAM can be slow and may cause missing image data, especially if WiFi is enabled. For RGB data, it is recommended to capture in JPEG and convert using fmt2rgb888, fmt2bmp, or frame2bmp.
    • Frame Buffering Modes:
      • 1 Frame Buffer: The driver waits for VSYNC and starts I2S DMA. This provides more control but results in longer frame acquisition times.
      • 2+ Frame Buffers: I2S runs in continuous mode, pushing frames to a queue. This allows for higher frame rates (double the rate) but increases CPU/Memory strain. Use this mode only with JPEG.
    • PSRAM DMA Mode: On ESP32-S2 and ESP32-S3, you can enable PSRAM DMA mode via the Kconfig option CONFIG_CAMERA_PSRAM_DMA (defaults to false). You can also switch this mode at runtime using esp_camera_set_psram_mode().
  6. Install the ESP32 Camera Driver for Arduino

    master

    Installation depends on your development environment:

    Arduino IDE

    If you are using the arduino-esp32 core, no installation is required. The driver is available immediately.

    PlatformIO

    Add esp32-camera to the lib_deps section of your platformio.ini file:

    [env]
    lib_deps =
      esp32-camera

    After adding the dependency, you can include the header in your code:

    #include "esp_camera.h"

    Note: You must enable PSRAM in menuconfig or by adding CONFIG_ESP32_SPIRAM_SUPPORT=y to your sdkconfig file.

  7. Convert frames to BMP

    master

    If you need to serve BMP images, use the frame2bmp() function to convert the current camera frame buffer into a BMP format buffer.

    // Assuming fb is a valid camera_fb_t from esp_camera_fb_get()
    uint8_t * buf = NULL;
    size_t buf_len = 0;
    bool converted = frame2bmp(fb, &buf, &buf_len);
    
    if(converted) {
        // Use buf and buf_len
        free(buf);
    }
    esp_camera_fb_return(fb);
  8. Initialize the ESP32 Camera

    master

    To use the camera, you must define a camera_config_t structure specifying the pin mappings (XCLK, SIOD, SIOC, Data pins, VSYNC, HREF, PCLK, etc.) and sensor settings (pixel format, frame size, JPEG quality). Pass this configuration to esp_camera_init().

    #include "esp_camera.h"
    
    static camera_config_t camera_config = {
        .pin_pwdn  = CAM_PIN_PWDN,
        .pin_reset = CAM_PIN_RESET,
        .pin_xclk = CAM_PIN_XCLK,
        .pin_sccb_sda = CAM_PIN_SIOD,
        .pin_sccb_scl = CAM_PIN_SIOC,
    
        .pin_d7 = CAM_PIN_D7,
        .pin_d6 = CAM_PIN_D6,
        .pin_d5 = CAM_PIN_D5,
        .pin_d4 = CAM_PIN_D4,
        .pin_d3 = CAM_PIN_D3,
        .pin_d2 = CAM_PIN_D2,
        .pin_d1 = CAM_PIN_D1,
        .pin_d0 = CAM_PIN_D0,
        .pin_vsync = CAM_PIN_VSYNC,
        .pin_href = CAM_PIN_HREF,
        .pin_pclk = CAM_PIN_PCLK,
    
        .xclk_freq_hz = 20000000,
        .ledc_timer = LEDC_TIMER_0,
        .ledc_channel = LEDC_CHANNEL_0,
    
        .pixel_format = PIXFORMAT_JPEG,
        .frame_size = FRAMESIZE_UXGA,
    
        .jpeg_quality = 12,
        .fb_count = 1,
        .grab_mode = CAMERA_GRAB_WHEN_EMPTY
    };
    
    // Initialize the camera
    esp_err_t err = esp_camera_init(&camera_config);
  9. Supported SoC and Sensors

    master

    The ESP32 Camera Driver supports the following hardware:

    Supported SoCs

    • ESP32
    • ESP32-S2
    • ESP32-S3

    Supported Sensors

    modelmax resolutioncolor typeoutput format
    OV26401600 x 1200colorYUV(422/420)/YCbCr422, RGB565/555, 8-bit compressed, 8/10-bit Raw RGB
    OV36602048 x 1536colorraw RGB, RGB565/555/444, CCIR656, YCbCr422, compression
    OV56402592 x 1944colorRAW RGB, RGB565/555/444, CCIR656, YUV422/420, YCbCr422, compression
    OV7670640 x 480colorRaw Bayer RGB, Processed Bayer RGB, YUV/YCbCr422, GRB422, RGB565/555
    OV7725640 x 480colorRaw RGB, GRB 422, RGB565/555/444, YCbCr 422
    NT991411280 x 720colorYCbCr 422, RGB565/555/444, Raw, CCIR656, JPEG compression
    GC032A640 x 480colorYUV/YCbCr422, RAW Bayer, RGB565
    GC0308640 x 480colorYUV/YCbCr422, RAW Bayer, RGB565, Grayscale
    GC21451600 x 1200colorYUV/YCbCr422, RAW Bayer, RGB565
    BF3005640 x 480colorYUV/YCbCr422, RAW Bayer, RGB565
    BF20A6640 x 480colorYUV/YCbCr422, RAW Bayer, Only Y
    SC101IOT1280 x 720colorYUV/YCbCr422, Raw RGB
    SC030IOT640 x 480colorYUV/YCbCr422, RAW Bayer
    SC031GS640 x 480monochromeRAW MONO, Grayscale
    HM0360656 x 496monochromeRAW MONO, Grayscale
    HM10551280 x 720color8/10-bit Raw, YUV/YCbCr422, RGB565/555/444