DOCSEARCHbyAlgolia
Submit Docs

XiaoZhi ESP32 AI Chatbot

repository·main·Indexed 11 days ago

https://github.com/78/xiaozhi-esp32

An open-source, MCP-based AI chatbot for ESP32 hardware enabling voice interaction with LLMs like Qwen or DeepSeek. Features a decoupled AudioService architecture with AfeAudioEngine and LiteAudioEngine for various ESP32 targets, and supports multiple hardware configurations including DFRobot K10 and ESP32-S3 AI CAM.

Tokens
77.3K
Snippets
198
Records
378
Agent score
93%

What's inside XiaoZhi

  1. Overview of Waveshare ESP32-S3-Touch-AMOLED-2.16

    main

    The Waveshare ESP32-S3-Touch-AMOLED-2.16 is a high-performance, highly integrated microcontroller development board. It is designed for easy embedding into products due to its compact form factor and integrated peripherals.

    Key Hardware Features:

    • Display: 2.16-inch capacitive HD AMOLED screen.
    • Sensors: Six-axis sensor (includes a three-axis accelerometer and a three-axis gyroscope).
    • Audio: Low-power audio codec chip and an echo cancellation circuit.
    • Power & Timing: Highly integrated power management chip and an RTC (Real-Time Clock).

  2. Overview of ESP-SensairShuttle Hardware

    main

    The ESP-SensairShuttle is a development board co-developed by Espressif and Bosch Sensortec, designed for motion sensing and LLM-based human-computer interaction scenarios.

    Key Hardware Specifications:

    • Main Controller: ESP32-C5-WROOM-1-N16R8 module.
    • Wireless Connectivity: Dual-band 2.4 & 5 GHz Wi-Fi 6 (802.11ax), Bluetooth® 5 (LE), Zigbee, and Thread (802.15.4).

  3. Overview of Xiaozhi AI Chatbot

    main

    Xiaozhi AI is an MCP-based (Model Context Protocol) voice interaction chatbot. It leverages Large Language Models (LLMs) like Qwen or DeepSeek to provide AI capabilities and uses the MCP protocol to enable multi-terminal control.

    Key Features:

    • Connectivity: Supports Wi-Fi, Ethernet, USB RNDIS, and 4G (ML307/EC801E or NT26 Cat.1).
    • Voice Interaction: Offline voice wake-up via ESP-SR with custom wake words. Supports both traditional streaming (ASR + LLM + TTS) and Realtime end-to-end voice models using Opus audio streams.
    • Communication Protocols: Supports WebSocket and MQTT + UDP.
    • Hardware Capabilities: Voiceprint recognition, OLED/LCD display support (with expressions), power management, and camera visual input on specific hardware.
    • MCP Control:
      • Device-side MCP: Control local hardware like volume, lights, motors, and GPIO.
      • Cloud-side MCP: Extend LLM capabilities to control smart homes, PC desktops, perform knowledge searches, or send emails.

  4. Overview of Minsi-k08 (DUAL) Hardware

    main

    The Minsi-k08 (WiFi and ML307 variants) is an AI chatbot solution developed by Minsi Technology. It is based on the ESP32S3N16R8 chip and is built by modifying the K08 transparent mecha speaker.

    Key Hardware Components:

    • MCU: ESP32S3N16R8
    • Audio Amplifier: MAX98357
    • Microphone: INMP441 omnidirectional microphone module

  5. Overview of Xingzhi ABS 2.0 Hardware

    main

    The Xingzhi ABS 2.0 is a cost-effective AI voice interaction development board. Key hardware features include:

    • Dual Network Support: Supports switching between Wi-Fi and ML307R Cat.1 4G communication, allowing for use without Wi-Fi.
    • Display: 1.54-inch 240×240 LCD screen with optimized UI for square displays.
    • Physical Interaction: Features independent physical buttons (Boot, Volume+, Volume-) supporting single click, double click, long press, and five-click operations.
    • Expansion: Built-in Micro SD card slot for local storage and reserved support for a vibration motor for haptic feedback.
    • Power Management: Includes battery level ADC detection, charging status monitoring, and automatic sleep/deep sleep modes.
    • Compatibility: Fully compatible with the xiaozhi-esp32 firmware, supports models like Qwen/DeepSeek, and is compatible with MCP protocol device control.

  6. What is Xiaozhi AI Chatbot?

    main

    Xiaozhi AI Chatbot is an MCP-based (Model Context Protocol) chatbot that serves as an entry point for voice interaction. It leverages Large Language Models (LLMs) like Qwen and DeepSeek to enable multi-endpoint control through the MCP protocol.

    Key capabilities include:

    • Voice Interaction: Supports offline wake-up via ESP-SR and real-time full-duplex dialogue on AEC-compatible hardware.
    • Communication: Uses WebSocket or MQTT + UDP protocols.
    • Device Control: Supports device-side MCP (volume, brightness, actions) and cloud-side MCP (smart home control, PC desktop operation, knowledge search, email, etc.).
    • Visuals: Displays emojis and emotional expressions on OLED/LCD screens; some boards support camera input.
    • Connectivity: Supports Wi-Fi, Ethernet, USB RNDIS, and 4G (ML307/EC801E or NT26 Cat.1).

  7. Overview of Waveshare ESP32-C6-Touch-AMOLED-2.06

    main

    The Waveshare ESP32-C6-Touch-AMOLED-2.06 is a smartwatch-shaped development board designed for wearable application prototyping. It is built around the ESP32-C6 microcontroller and features a high-definition 2.06-inch capacitive touch AMOLED screen.

    Key integrated modules include:

    • Display: 2.06-inch capacitive touch high-definition AMOLED screen.
    • Sensors: Six-axis sensor for motion tracking.
    • Audio: Integrated audio codec chip.
    • System Components: Real-Time Clock (RTC) and power management modules.
    • Form Factor: Custom case designed to resemble a smartwatch.

  8. Overview of Waveshare ESP32-C6-Touch-AMOLED-1.8

    main

    The Waveshare ESP32-C6-Touch-AMOLED-1.8 is a highly integrated microcontroller development board based on the ESP32-C6. It is designed for compact, high-performance applications and features an onboard 1.8-inch capacitive AMOLED display.

    Key integrated hardware components include:

    • Display: 1.8-inch capacitive AMOLED
    • Motion Sensing: 6-axis IMU (3-axis accelerometer and 3-axis gyroscope)
    • Audio: Low power audio codec IC
    • Power Management: Highly integrated PMIC
    • Timing: RTC (Real-Time Clock)

  10. Overview of LVGL Image Conversion scripts

    main

    The scripts/Image_Converter directory contains two primary Python scripts for preparing assets for LVGL-based displays:

    1. LVGLImage.py: A core conversion script sourced from the official LVGL repository. It handles the logic of transforming standard image files into LVGL-compatible formats.
    2. lvgl_tools_gui.py: A wrapper script that provides a graphical user interface for LVGLImage.py. It is designed for batch processing and ease of use, making it ideal for users who need to modify device assets (like Xiaozhi's facial expressions) without using the command line.

  11. Overview of ESP-Hi AI Robot

    main
    ESP-Hi is a low-cost AI chatbot based on the ESP32-C3 developed by ESP Friends. It features a 0.96-inch color screen for displaying expressions and supports dozens of robotic dog movements. The software is optimized for resource-constrained environments, enabling wake-word detection and peripheral driver support with minimal memory and Flash usage. Hardware details are available on the 立创开源项目 platform.

  12. Overview of NULLLAB-AI-VOX3 Hardware

    main

    The NULLLAB-AI-VOX3 is a high-performance embedded development board designed for AI voice interaction. It is based on the ESP32-S3-R8 chip and is optimized for applications like AI chat, weather clocks, wireless walkie-talkies, MP3 players, and internet radio.

    Key Specifications

    • Processor: ESP32-S3R8 (Xtensa 32-bit LX7 dual-core, up to 240MHz).
    • Memory: 512 KB SRAM, 384 KB ROM, 8 MB PSRAM, and 16 MB Flash.
    • Connectivity: 2.4 GHz Wi-Fi (802.11 b/g/n) and Bluetooth 5 (LE) with onboard antenna.
    • Audio: Integrated ES8311 audio codec and 3W audio amplifier (NS4150B). Requires an external speaker. Features a dual-microphone design (supports analog microphones and single-mic interruption).
    • Display Support: Supports 1.54-inch 240×240 SPI LCD (ST7789) or OLED via reserved connectors.
    • Power: Type-C interface for programming, power supply, and lithium battery charging. Includes a 5V 2.4A boost circuit. Battery level can be monitored via IO18 ADC.
    • Storage: Onboard SD Card interface for expansion.
    • Expansion:
      • 8-pin GPIO header: GPIO43, GPIO44, GPIO42, GPIO48, GPIO4, GPIO3, GPIO2, GPIO1.
      • 4-pin PH2.0 interface for power or communication with other controllers.
      • Compatible with AI-VOX3 expansion boards and MD40 motor driver boards.
    • Development Frameworks: Supports ESP-IDF, Arduino IDE, and AilyBlockly.