Claude Desktop Buddy

Any command sent by the desktop containing a cmd field requires a matching acknowledgment.

Ack Format: {"ack": "<original_cmd_name>", "ok": true, "n": 0}

If a command fails, set "ok": false and optionally include an "error": "..." string. The field n is a generic counter (e.g., bytes written for chunk acks, otherwise 0).

{
  "ack": "<same as cmd>",
  "ok": true,
  "n": 0
}

The device transitions between seven distinct states based on its connection to Claude and user interactions:

To connect your device to Claude on macOS or Windows, follow these steps:

1. Enable Developer Mode: In Claude Desktop, go to Help → Troubleshooting → Enable Developer Mode.
2. Open Hardware Buddy: Go to Developer → Open Hardware Buddy….
3. Connect: Click Connect in the Hardware Buddy window and select your device from the list.
4. Permissions: Grant Bluetooth permissions if prompted by macOS.

Once paired, the bridge automatically reconnects whenever both the desktop app and the device are awake. If the device is not discovered, ensure it is awake (press any button) and that Bluetooth is enabled in the device's settings menu.

When a user drops a folder into the Claude window, the desktop streams its flat contents (no recursion, no dotfiles) to your device. The transport is content-agnostic and sequential.

Workflow:

1. Start: Desktop sends char_begin. If you don't want files, do not ack this. char_begin.name is the folder name (or the name from a manifest.json inside the folder).
2. File Metadata: Desktop sends file with path and size.
3. Data Transfer: Desktop sends chunk with a base64-encoded payload. You must decode and append.
4. File End: Desktop sends file_end.
5. End Session: Desktop sends char_end.

Security Note: Always validate file.path to prevent writing to .. or absolute paths.

The BLE bridge is disabled by default. To enable it in Claude for macOS or Windows, follow these steps:

1. Go to Help → Troubleshooting → Enable Developer Mode. This adds a Developer menu to the menu bar.
2. Go to Developer → Open Hardware Buddy… to open the pairing window.
3. Click Connect and select your device from the scan list. The OS may prompt for Bluetooth permissions on the first use.

Once paired, the bridge automatically reconnects in the background. You only need the window open for initial pairing, viewing the stats panel, or using the folder drop target.

The firmware targets ESP32 with the Arduino framework and depends on the M5StickCPlus library. To flash the device, install PlatformIO Core and use the following commands:

To upload the firmware:

pio run -t upload

To wipe the device before flashing (recommended for previously-flashed devices):

pio run -t erase && pio run -t upload

If you need to perform a factory reset directly on the device, use the following sequence: hold A → settings → reset → factory reset → tap twice.

You can install the bufo character pack onto your Hardware Buddy using two different methods:

1. Drag and Drop: Drag the characters/bufo folder directly onto the Hardware Buddy window.
2. CLI Flashing: Use the provided Python tool to flash the character pack over a USB connection.

Note: The bufo pack consists of fifteen GIFs and a manifest that maps them to the seven device states. The idle state includes nine variants to provide varied animations (blinks and glances) rather than a single looping clip.

python3 tools/flash_character.py characters/bufo

You can replace the default ASCII pets with custom GIF characters by dragging a character pack folder onto the drop target in the Hardware Buddy window. The app will stream the pack over BLE and switch the device to GIF mode live.

To revert to ASCII mode, navigate to Settings → delete char on the device.

Character Pack Requirements

A character pack is a folder containing a manifest.json and several 96px-wide GIFs. The total folder size must be under 1.8MB.

To prepare your GIFs, use tools/prep_character.py to ensure all states are scaled consistently to 96px wide. You can also use gifsicle to reduce file size:

gifsicle --lossy=80 -O3 --colors 64

To skip the BLE transfer during development, use the staging tool:

python tools/flash_character.py characters/bufo

Because transcript snippets and tool-call hints flow over BLE, unencrypted links are vulnerable to sniffing. It is recommended to use LE Secure Connections bonding.

Implementation Steps:

1. Mark your NUS characteristics and the TX CCCD as encrypted-only.
2. Advertise DisplayOnly IO capability.
3. When the first GATT access occurs, the OS will trigger pairing. Your device should display a 6-digit passkey for the user to enter in the desktop app.
4. Once bonded, the link is AES-CCM-encrypted. Reconnects will reuse the stored LTK.
5. Protocol Integration:
   • Include "sec": true in your status response data once encrypted.
   • Handle {"cmd":"unpair"} by erasing your stored bonds so the next pairing generates a fresh passkey.

When the heartbeat snapshot contains a prompt object, your device must respond to the specific id provided. Send a JSON object with the cmd set to "permission" and the decision set to either "once" (approve) or "deny" (reject).

{"cmd":"permission","id":"req_abc123","decision":"once"}
{"cmd":"permission","id":"req_abc123","decision":"deny"}

When a turn is completed, the desktop fires a one-shot event containing the raw SDK content (text blocks, tool calls, etc.). Note that events larger than 4KB (UTF-8 bytes) are dropped by the desktop.

{
  "evt": "turn",
  "role": "assistant",
  "content": [{ "type": "text", "text": "..." }]
}