Even Demo App

repository·main·Indexed 19 days ago

https://github.com/even-realities/evendemoapp

A reference implementation for interacting with Even smart glasses. It demonstrates dual-Bluetooth communication for AI features, real-time LC3 audio streaming, BMP image transmission (1-bit, 576*136 pixels), and text display. The documentation covers the Even AI workflow, TouchBar event commands, G1 Bluetooth architecture using Nordic UART-style GATT services, and the integration between Flutter and native Android/iOS layers via Method and Event Channels.

Tokens
9.5K
Snippets
38
Records
49
Agent score
67%

What's inside evendemoapp

  1. Identify G1 Device Advertising Names and Pairing

    main

    G1 devices advertise with names following a specific pattern that allows the app to pair the left and right ears.

    Naming Pattern

    Names follow the format: G1_{channel}_{side}_{id} (e.g., G1_45_L_92333 and G1_45_R_xxxx).

    Pairing Logic

    1. The app scans for devices matching the G1 prefix.
    2. It splits the name by underscores (_) to extract the channel number.
    3. A paired set is only recognized when both a Left (_L_) and a Right (_R_) device with the same channel number are found.
    4. When a pair is found, the native layer notifies Flutter with a map containing leftDeviceName, rightDeviceName, and channelNumber via the foundPairedGlasses method.
  2. Understand the EvenDemoApp BLE Architecture

    main

    The application uses a hybrid architecture where the Flutter (Dart) layer communicates with platform-specific native BLE implementations (Android/iOS) via MethodChannel and EventChannel.

    • Downlink (Dart → Native): Uses MethodChannel("method.bluetooth") to invoke methods like send, startScan, or connectToGlasses.
    • Uplink (Native → Dart): Uses EventChannel("eventBleReceive") (Android) or blueInfoSink (iOS) to stream binary notifications and business data.
    • Status Callbacks: Native code uses methodChannel.invokeMethod to trigger state changes in Dart (e.g., setMethodCallHandler).
    // Dart side entry point
    // lib/ble_manager.dart
    
    // Example of calling a native method
    await methodChannel.invokeMethod("send", {"data": data, "lr": lr});
    
    // Example of listening to incoming data
    eventChannel.receiveBroadcastStream().listen((data) => _handleReceivedData(data));
  3. Understand the G1 Bluetooth Architecture

    main

    The G1 device model uses two separate BLE peripherals (one Left _L_ and one Right _R_) that are paired by a shared channel number. The application manages this via a Flutter MethodChannel that communicates with native Android (BleManager.kt) and iOS (BluetoothManager.swift) implementations.

    Communication Channels

    • method.bluetooth (Bidirectional): Used for commands like startScan, stopScan, connectToGlasses, disconnectFromGlasses, and send.
    • eventBleReceive (Native → Dart): An EventChannel used for binary notifications and app payloads (containing lr, data, and type).
    • Status Updates: Native code uses methodChannel.invokeMethod to trigger state changes in Dart (e.g., setMethodCallHandler).
  4. How the Even AI workflow works

    main

    The Even AI feature follows a specific sequence of Bluetooth interactions between the app and the glasses:

    1. Activation: After dual Bluetooth connection, the user long-presses the left-side TouchBar. The glasses send command [0xF5, 0x17] to the app.
    2. Microphone Activation: The app must respond by sending command [0x0E, 0x01] to activate the right-side microphone.
    3. Audio Capture: The glasses stream real-time audio in LC3 format. The user holds the TouchBar while speaking (max 30 seconds).
    4. Processing: The app converts the LC3 audio to text and sends it to a Large Language Model (LLM).
    5. Result Transmission: The app sends the LLM response to the glasses via the Bluetooth protocol.
      • Automatic Mode (Default): Results are transmitted page by page.
      • Manual Mode: Triggered by a single tap on the TouchBar. Use the left TouchBar for page-up and the right TouchBar for page-down.
    6. Exit: A double-tap on the TouchBar exits the Even AI function.
  5. Identify G1 Paired Glasses via Bluetooth Advertising

    main

    G1 glasses consist of two separate BLE peripherals (one for the left ear, one for the right). They are identified and paired using a specific naming convention based on a channel number.

    Naming Format: G1_{channel}_{L/R}_{unique_id} (e.g., G1_45_L_92333 and G1_45_R_xxxx).

    Pairing Logic:

    • The system scans for devices matching the G1 prefix.
    • It groups devices by their channel number.
    • A pair is considered "found" only when both a Left (_L_) and a Right (_R_) device for the same channel are discovered.
    • Once paired, the Flutter layer receives a dictionary containing leftDeviceName, rightDeviceName, and channelNumber.
  6. Connect to G1 Glasses from Flutter

    main

    To connect to a pair of glasses, use the BleManager in Dart. The deviceName passed to connectToGlasses should follow the format Pair_{channelNumber}.

    Connection Lifecycle:

    1. Scan: Call BleManager.startScan().
    2. Discovery: Wait for the native layer to trigger foundPairedGlasses via the MethodChannel.
    3. Connect: Call connectToGlasses(deviceName).
    4. Success: The native layer will trigger glassesConnected once the connection criteria are met.
    5. Heartbeat: Upon successful connection, the app automatically starts sending heartbeats using Proto.sendHeartBeat() every 8 seconds.
    // 1. Start scanning
    bleManager.startScan();
    
    // 2. Listen for paired glasses discovery
    // (Triggered via MethodChannel callback)
    
    // 3. Connect using the pair name
    await bleManager.connectToGlasses("Pair_45");
  7. Transmit text to glasses

    main

    To display text on the glasses, follow these steps:

    1. Line Division: Divide input text into lines based on the glasses' display width (demo uses 488 pixels) and a chosen font size (demo uses 21).
    2. Packetization: Divide the lines into packets based on the number of lines per screen (demo uses 5) and the BLE packet size limit. For example, if 5 lines are displayed per screen, the first three lines might form one packet and the remaining two form another.
    3. Sequential Sending: Use the 0x4E command (see Text Sending protocol) to send multi-packet data screen by screen. The demo uses a timer to sequence the transmission of each screen.
  8. Implement the Flutter (Dart) Connection Flow

    main

    To manage the G1 connection from the Flutter layer, follow this lifecycle using the BleManager class:

    1. Scan: Call BleManager.startScan() (invokes startScan via MethodChannel).
    2. Handle Pairing: Listen for foundPairedGlasses via the method channel to receive the leftDeviceName, rightDeviceName, and channelNumber.
    3. Connect: Call connectToGlasses(deviceName). Note that deviceName should be passed as Pair_{channelNumber}. The native layer will strip the Pair_ prefix.
    4. Connection Established: Listen for glassesConnected. Once received, set isConnected to true and start the heartbeat via startSendBeatHeart().
    5. Data Reception: Listen to the eventBleReceive EventChannel to handle incoming binary data and commands (e.g., touchpad or AI events).
    // Example conceptual flow in Dart
    await bleManager.startScan();
    
    // When paired glasses are found
    // native calls 'foundPairedGlasses' with {leftDeviceName, rightDeviceName, channelNumber}
    
    await bleManager.connectToGlasses('Pair_45');
    
    // Listen for data
    eventChannel.stream.listen((event) {
      _handleReceivedData(event);
    });
  9. Transmit BMP images to glasses

    main

    Image transmission supports 1-bit, 576*136 pixel BMP images. The process requires three steps:

    1. Packetization: Divide the BMP data into 194-byte packets. Each packet must be prefixed with the 0x15 command and a syncID (the packet index).
      • First Packet: Must include the glasses' end storage address [0x00, 0x1c, 0x00, 0x00]. The format is [0x15, index & 0xff, 0x00, 0x1c, 0x00, 0x00, ...data].
      • Subsequent Packets: Do not include the address; use [0x15, index & 0xff, ...data].
      • Packets can be sent to the left and right BLE sides independently.
    2. End Command: After the last packet is sent, send the packet end command [0x20, 0x0d, 0x0e] to the dual BLE.
    3. CRC Check: Once the end command is acknowledged, send a CRC check command using command 0x16. The CRC must be calculated using Crc32Xz big endian, incorporating both the BMP picture storage address and the picture data.
    // First packet structure example
    [0x15, index & 0xff, 0x00, 0x1c, 0x00, 0x00, ...data]
    
    // Packet end command
    [0x20, 0x0d, 0x0e]
  10. Customize the iOS launch screen assets

    main

    To change the image displayed during the app's launch on iOS, you can either replace the image files directly in the ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory or use Xcode for a visual approach.

    Using Xcode:

    1. Open the iOS project workspace using: open ios/Runner.xcworkspace.
    2. In the Xcode Project Navigator, navigate to Runner/Assets.xcassets.
    3. Drag and drop your desired images into the asset catalog to replace the existing launch screen assets.
    open ios/Runner.xcworkspace
  11. How Even AI response streaming works

    main

    The Even AI response flow follows a specific sequence to ensure the glasses can display text correctly:

    1. Speech Capture: toStartEvenAIByOS() starts the microphone and listens for speech via an EventChannel named eventSpeechRecognize.
    2. Processing: When recordOverByOS() is called, the captured combinedText is sent to ApiDeepSeekService.
    3. Text Chunking: The response text is broken into a list of strings using EvenAIDataMethod.measureStringList, which calculates line breaks based on a maximum width (default 488) and font size (21).
    4. Paging Protocol:
      • The service first sends a packet with status: 0x30 to prepare the glasses.
      • It then sends a packet with status: 0x40 to trigger the actual display.
      • For long responses, a Timer periodically sends chunks of text every 5 seconds using status: 0x30 until the end of the list is reached, where it finishes with status: 0x40.
    5. Manual Override: If the user interacts with the TouchBar, the timer is cancelled, and the service uses status: 0x50 to send manually requested pages.
  12. Configure Bluetooth Method and Event Channels

    main

    The BleChannelHelper object manages the communication bridge between Flutter and Android for Bluetooth operations. To enable communication, you must initialize the channels using initChannel.

    Communication is split into two types:

    1. Method Channels: Used for Flutter to call native Android Bluetooth functions (e.g., scanning, connecting).
    2. Event Channels: Used for the native side to stream asynchronous updates back to Flutter (e.g., connection status, received data).

    Event Channel Tags:

    • eventBleStatus: Streams Bluetooth status updates.
    • eventBleReceive: Streams data received from the device.
    • eventSpeechRecognize: Streams speech recognition events.
    // In Android MainActivity/Context
    BleChannelHelper.initChannel(this, flutterEngine)