DOCSEARCHbyAlgolia
Submit Docs

ESP-Claw AI Agent Framework

repository·master·Indexed 23 days ago

https://github.com/espressif/esp-claw

An AI agent framework for Espressif IoT devices that enables 'Chat Coding', allowing users to define device behavior through conversation. The framework transforms IoT hardware into active edge agents capable of local sensing, reasoning, and execution. It includes support and hardware specifications for various ESP32-S3 based boards, including the ESP32-S3 SparkBot, EDA AI GameBox, EDA Robot Pro, EDA Super Bear, and EDA TV Pro.

Tokens
125K
Snippets
248
Records
569
Agent score
82%

What's inside ESP-Claw

  1. Overview of ESP-Claw AI Agent Framework

    master

    ESP-Claw is an AI Agent framework designed for IoT devices, specifically optimized for the ESP32 series. It enables a "Chat Coding" workflow where device behaviors are defined through conversation. The framework moves the Agent Runtime directly onto the Espressif chips, transforming them from passive execution units into active decision-making centers.

    Core Capabilities

    • Chat Coding: Uses IM (Instant Messaging) combined with Lua dynamic loading, allowing users to define device behaviors via chat without manual programming.
    • Event-Driven Architecture: Any event can trigger the Agent Loop, enabling millisecond-level response times.
    • Structured Memory: Maintains organized memory locally on the device to ensure privacy (data does not leave the device unnecessarily).
    • MCP Communication: Supports the Model Context Protocol (MCP), allowing the device to act as both an MCP Server and Client to connect with standard MCP devices.
    • Extensibility: Modular design allows for trimming components or adding custom component adapters.

    Supported Platforms

    • Hardware: ESP32-S3, ESP32-P4, ESP32-C5, and ESP32-S31.
    • LLM Support: Supports OpenAI-style and Anthropic-style APIs. Compatible with GPT series, Qwen (via Alibaba Cloud), Claude (via Anthropic), DeepSeek, and custom endpoints.
    • IM Support: Supports Telegram, QQ, Feishu, and WeChat (extensible).

  2. Overview of mcp_server_point (ESP-Claw)

    master

    mcp_server_point is a lean ESP-IDF application designed as a trimmed-down version of the full edge_agent. It is optimized for a small capability set focused on Lua scripting and MCP (Model Context Protocol) server functionality.

    Key Characteristics:

    • Core Capabilities: Centered on cap_lua and cap_mcp_server.
    • Storage: Uses a wear-leveled FAT filesystem on a storage partition for Claw data and Lua scripts.
    • Networking: Wi-Fi configuration is handled via app_config (NVS-backed). There is no HTTP provisioning portal or serial CLI enabled by default.
    • Execution Model: MCP clients interact with the device by calling Lua tools (e.g., lua.run_script) rather than using the claw_event_router.

  3. Overview of Lilygo T-Display-P4 Board Support

    master

    The Lilygo T-Display-P4 board support is organized according to the ESP Board Manager v0.5.12 device model.

    Hardware Logic

    • Power & Reset: The XL9535 GPIO expander manages Power_EN_3V3, Power_EN_5V0, Screen_RST, Touch_RST, and Vcca_EN via YAML configuration.
    • Display Selection: The system performs a runtime probe for the GT9895 touch ID in setup_device.c.
      • If GT9895 is detected: The system selects the RM69A10 AMOLED path.
      • If GT9895 is NOT detected: The system falls back to the HI8561 LCD path.
    • Touch Initialization:
      • GT9895 (0xba/0x28) is created after display selection.
      • HI8561 (0xd0) touch is created only after the LCD panel is initialized, as its address must be read from the HI8561 controller.

    Device Mapping

    Device NameFunction
    gpio_expander_xl9535Manages power and reset pins via YAML.
    display_lcdSelects RM69A10 or HI8561 DSI timings based on GT9895 probe result.
    lcd_touchCreates GT9895 or HI8561 touch driver.
    lcd_brightnessControls backlight via ledc_ctrl (GPIO51 for HI8561) or DCS 0x51 (for RM69A10).
    audio_dacInitializes ES8311 playback path via audio_codec.
    audio_adcInitializes ES8311 recording path via audio_codec.

  4. Overview of cap_im_platform IM integration

    master
    The cap_im_platform component provides a unified interface for integrating various Instant Messaging (IM) platforms into ESP-Claw. It bundles support for Feishu, QQ, Telegram, WeChat, and a local Web interface into a single ESP-IDF component. While the build dependency is unified, the runtime surface is split by platform to maintain stable group IDs and tool names for existing configurations.

  7. Hardware specifications for RockBase-iot NM-CYD-C5

    master

    The RockBase-iot NM-CYD-C5 is an ESP32-C5 based board with the following key hardware features:

    • SoC: ESP32-C5-WROOM-1 (RISC-V @ 240 MHz)
    • Memory: 16 MB flash, 8 MB quad PSRAM
    • Display: 2.8 inch ST7789 TFT LCD (320x240 resolution)
    • Touch: XPT2046 resistive touch (shared SPI2 bus)
    • Storage: microSD slot (shared SPI2 bus)
    • RGB LED: Single WS2812 pixel on GPIO27
    • GPS: LP-UART P5 connector for NMEA modules
    • Button: BOOT button (GPIO0, active low)

  8. Lilygo T-Display-P4 Board Support Overview

    master

    The Lilygo T-Display-P4 board support follows the ESP Board Manager v0.5.12 device model. It uses an XL9535 GPIO expander to manage power rails (3V3, 5V0, Vcca) and reset lines (Screen, Touch).

    At runtime, the board performs a probe of the GT9895 touch ID to determine the display path:

    • GT9895 present: Selects the RM69A10 AMOLED path.
    • GT9895 absent: Falls back to the HI8561 LCD path. If this path is taken, the HI8561 touch driver is initialized after the LCD panel is ready, as its address is read from the controller.

  10. Hardware Specifications for EDA Super Bear

    master

    The EDA Super Bear is an ESP32-S3 based platform featuring 16MB QIO Flash, Octal PSRAM, I2S audio capabilities (microphone and speaker), and 6 SG90 servos.

    Key Hardware Components:

    • Chip: ESP32-S3
    • Flash: 16MB QIO 80MHz
    • PSRAM: Octal 80MHz
    • Microphone: INMP441 MEMS (I2S)
    • Speaker: MAX98357 Class-D (I2S)
    • Servos: 6x SG90
    • Buttons: Boot and Touch