MentraOS Documentation

Mentra Cloud Core Services (@mentra/cloud-core) is the proprietary central cloud product that powers the Mentra ecosystem. It hosts the shared application and developer ecosystem, authentication, and bundle storage.

Important Integration Note: OEMs (Original Equipment Manufacturers) do not self-host these services. Instead, they access them through the Cloud Proxy.

The OTA I/O package is organized into several functional layers:

• Interfaces: Defines the contracts for services (IOtaService), helpers (IOtaHelper), and callbacks (OtaProgressCallback).
• Core: Contains the central management logic (OtaManager), service factories (OtaServiceFactory), and base implementations (BaseOtaService).
• Services: Handles Android service lifecycle, notifications, and event bus integration (OtaService).
• Helpers: Provides robust implementation for version checking, downloading, and backup management (OtaHelper).
• Events: Manages progress updates via OtaEventBus using DownloadProgressEvent and InstallationProgressEvent.
• Utils: Provides utility functions for version management, file integrity (MD5), storage checks, and formatting (OtaUtils, OtaNotificationManager, OtaConstants).

MentraOS is an open-source operating system designed for smart glasses. It provides a unified abstraction layer that allows developers to build applications once and run them across multiple hardware platforms. MentraOS handles complex low-level tasks such as pairing, connection management, data streaming, and hardware access (displays, microphones, cameras, speakers), enabling rapid application development.

Key features include:

• Cross-Compatibility: One app runs on various supported smart glasses.
• Hardware Abstraction: Access all glasses' sensors and peripherals through a single API.
• Multi-App Execution: The Mentra Runtime allows multiple MiniApps (e.g., captions, AI notes, notifications) to run simultaneously over a single shared connection.
• Open Source: Licensed under MIT, providing full control over hosting and modification.

The v3-smoke-test is a minimal mini app designed to validate the MentraOS SDK v3 runtime and the Hono webview developer workflow. It is intended to exercise core v3 functionalities without the complexity of larger example applications.

Key components and features exercised by this app include:

• Runtime & Server: MiniAppServer and Bun fullstack /webview routes.
• Session & Connectivity: MentraSession management, transcription subscriptions, and reconnect callbacks.
• Authentication: @mentra/react auth initialization and createMentraAuthRoutes() for token exchange.
• API & UI: Authenticated Hono API routes, basic display output, and Bun Hot Module Replacement (HMR) for the frontend.

MentraOS Cloud provides two primary interfaces for interacting with the platform:

1. WebSocket APIs: Used for real-time, bidirectional communication between smart glasses and applications.
2. REST APIs: HTTP-based endpoints used for authentication, management, and configuration tasks.

The Audio Service is the stateful core of the MentraOS runtime. It manages the live transcription and translation path, converting user audio into transcripts and translations and pushing them back to the client via WebSocket.

Because live transcription sessions must maintain continuity without gaps during failures, the service includes specialized ownership and failover machinery. It operates on top of the runtime transport contract and relies on cloud-runtime tokens for authentication, extracting the user id and tenantId from the token claims to verify the audio path.

@mentra/cloud-client is a TypeScript library that serves as the phone's connection to Cloud V2. It is a headless library (no UI) responsible for opening connections, sending requests, and receiving events.

Key characteristics:

• Environment Agnostic: The same code runs on both the phone and a server. Platform-specific dependencies (like network sockets or secure storage) are injected via specific entry points: @mentra/cloud-client/react-native for mobile devices and @mentra/cloud-client/node for testing environments.
• V2 Exclusive: It is built specifically for the V2 cloud architecture and does not support or include V1 legacy messages or REST calls.
• Type Safety: It uses shared definitions from @mentra/cloud-runtime/protocol to ensure that subscriptions and events are perfectly synchronized with the cloud server's types, preventing runtime message mismatches.