website·Indexed 33 days ago
https://docs.mentraglass.com/Developer documentation for building on-device apps for smart glasses using the Mentra Miniapp SDK. Covers core concepts including a two-layer architecture (background and UI), hardware integration for camera, IMU, LED, and microphone, as well as real-time transcription, translation, and navigation. Includes guides on the mentra-miniapp CLI, miniapp.json manifest, and Bluetooth SDK integration for Android.
The Mentra Miniapp SDK (@mentra/miniapp) allows developers to build on-device apps that run within the Mentra App on a user's phone to drive smart glasses.
Key Constraints & Status:
The @mentra/miniapp-cli provides the mentra-miniapp command for developing, packing, and releasing Mentra miniapps.
Critical Requirements:
bun or bunx. Do not use npx or Node.js.package.json scripts (e.g., "dev": "mentra-miniapp dev").mentra-miniapp <command>The Mentra SDK provides commands across several functional areas:
scan, startScan, stopScan, connect, connectDefault, disconnect, forgetrequestWifiScan, sendWifiCredentials, forgetWifiNetwork, setHotspotStaterequestPhoto, gallery mode, video recording defaults, camera field of viewstartStream, stopStreamsetMicState, VAD settings, audio callbacks, local transcription, media volumerequestVersionInfo, checkForOtaUpdate, startOtaUpdatergbLedControl (supported hardware only)session.phone.onBattery() to receive updates about the phone's battery status. This does not require any special permissions.session.phone.onBattery((data) => {
// data is of type BatteryData
console.log(`Battery Level: ${data.level}, Charging: ${data.charging}`);
});session object is the central handle for all interactions with the glasses hardware and system. It is provided to the registerMiniapp handler when a miniapp connects. All capabilities (display, audio, sensors, storage) are organized as modules on the session object.import { registerMiniapp } from "@mentra/miniapp/background";
registerMiniapp((session) => {
// Access modules via session
session.transcription.on((data) => {
session.display.render([
{ type: "text", id: "caption", box: { x: 0, y: 0, w: 576, h: 288 }, text: data.text },
]);
});
});The glasses use a queued drawing model where commands are accumulated in a pending batch and only rendered to the panel when a commit command is received. This ensures atomic updates and prevents the wearer from seeing half-applied frames.
Key rules:
clear_display wipes the current scene and frees all retained element IDs. A batch without clear_display amends the existing scene.intensity value from 0 (off/transparent) to 15 (brightest).clear_display // Wipe existing scene
draw_rect { ... } // Queue rectangle
draw_text { ... } // Queue text (paints over rect)
commit { } // Render all queued ops atomicallysession.storage is a phone-local, persistent key-value store scoped to the user and the specific miniapp. Data survives restarts and keys do not collide between different miniapps. Both keys and values must be strings. All methods are asynchronous as they round-trip to the phone. No special permissions or hardware are required to use this API.await session.storage.set("lastNote", "buy milk");
const note = await session.storage.get("lastNote"); // "buy milk"dev branch. The primary source of truth for setup, architecture, builds, and tests is the documentation located within each specific module's directory.miniapp.json configuration file and handled in the background layer. These actions map 1:1 to Model Context Protocol (MCP) tools, where id maps to name, description remains the same, parameters maps to inputSchema, and outputSchema is preserved.