DOCSEARCHbyAlgolia
Submit Docs

Meta Wearables Device Access Toolkit (DAT) for iOS

repository·main·Indexed 19 days ago

https://github.com/facebook/meta-wearables-dat-ios

A developer preview SDK enabling iOS applications to connect to Meta AI glasses, such as Meta Ray-Ban glasses. The toolkit provides capabilities for video streaming, photo capture, and interacting with display features. It includes modules like MWDATCore, MWDATCamera, and MWDATDisplay, and supports integration via Swift Package Manager.

Tokens
14.4K
Snippets
40
Records
68
Agent score
67%

What's inside Meta Wearables Device Access Toolkit (DAT) for iOS

  1. Use the Live DAT Debugging MCP skill for iOS

    main

    The live-debugging-mcp skill is designed for debugging iOS DAT (Device Access Toolkit) applications using local DAT Inspector MCP servers or DAT debug servers. It is intended for observing live app/device behavior, such as permission issues, registration failures, session/stream state, and communication problems between the app and glasses.

    When to use this skill

    • Debugging Meta AI app/device boundary issues.
    • Troubleshooting permissions, registration, or session/streaming states.
    • Investigating user reports regarding app-to-glasses communication.
    • Analyzing live device events and SDK logs.

    Key Constraints

    • Read-only by default: The agent should not attempt to mutate Meta AI app, device, account, permission, registration, or app state unless explicitly requested.
    • Boundary Diagnosis: Results from companion-app/device tools should be treated as boundary diagnosis from app-visible DAT events, not as direct access to companion app internals.
    • Fallback: If MCP tools are not configured, fall back to analyzing logs and using search_dat_docs (the documentation search tool).

  2. Map DAT symptoms to evidence sources

    main

    Use the following mapping to identify which data points to inspect based on the observed symptom:

    SymptomEvidence/API to Inspect
    ConfigurationApp logs around Wearables.configure()
    URL CallbacksWearables.shared.handleUrl(_:) return handling
    RegistrationWearables.shared.startRegistration() and wearables.registrationState
    Device AvailabilityDevice selectors, wearables.devices, device.compatibility, and device.properties
    Device Linkdevice.linkState
    PermissionsDAT permission APIs, device.permission, check_permissions, and get_permissions
    SessionDeviceSession creation/start, deviceSession.state, session.state, and session errors
    StreamStream start, stream.state, frame counters, and stream errors
    Transport (DAM/DWA)sdk.usesDam and DAM/DWA error values

  3. Use MCP for live documentation search

    main

    For live documentation search in MCP-compatible editors, connect to the remote HTTP MCP server. This server does not require authentication (no tokens, OAuth, or custom headers needed).

    Endpoint: https://mcp.developer.meta.com/wearables Tool: search_dat_docs

  4. Diagnose stream state issues

    main

    The DAT SDK stream follows a specific lifecycle. If your stream is not behaving as expected, check the current state against the expected flow.

    Expected flow: stopped $\rightarrow$ waitingForDevice $\rightarrow$ starting $\rightarrow$ streaming $\rightarrow$ stopped

    Common State Issues

    Stuck in waitingForDevice

    • The device is not in range or not connected.
    • The device is not reporting availability.
    • The DeviceSelector is not matching any available device.

    Unexpected stop

    • The device disconnected (out of range or battery died).
    • The channel was closed by the device.
    • An error occurred during frame processing.

  5. Understand the DAT SDK module architecture

    main

    The SDK is partitioned into four distinct modules to allow for granular imports:

    • MWDATCore: Handles device discovery, registration, permissions, and device selectors.
    • MWDATCamera: Provides functionality for streaming, VideoFrame handling, and photo capture.
    • MWDATDisplay: Manages display capabilities, UI components (FlexBox, Text, Button, Image, Icon), and video playback.
    • MWDATMockDevice: Contains MockDeviceKit for simulating hardware during testing without physical glasses.

  6. Use Swift patterns and concurrency correctly

    main

    When developing with the DAT SDK, follow these concurrency and architectural patterns:

    • Concurrency: Use async/await for all SDK operations. The SDK is fully asynchronous.
    • Observing Streams: Use AsyncSequence or the .listen {} publisher pattern to observe data streams.
    • UI Updates: Always annotate code that updates the user interface with @MainActor.
    • Threading: Never block the main thread with frame processing tasks.
    • Error Handling: Use standard do/catch blocks, as the SDK throws typed errors.

  7. Manage device session states in DAT SDK

    main

    The DAT SDK operates within sessions. There are two types of experiences: Device sessions (sustained access to sensors/outputs) and Transactions (short, system-owned interactions like notifications). Your application must observe session state changes, as the device controls the transitions.

    Session States

    StateMeaningApp action
    idleSession created but not startedCall start() when ready
    startingSession is connecting to the deviceShow connecting state
    startedSession active and ready for capabilitiesAdd or resume work
    pausedTemporarily suspended by the deviceHold work, may resume
    stoppingSession is cleaning upWait for terminal state
    stoppedSession inactive and terminalFree resources, create a new session to restart

    Observing session state

    Use session.stateStream() to asynchronously monitor state transitions.

    let session = try Wearables.shared.createSession(deviceSelector: AutoDeviceSelector(wearables: Wearables.shared))
try session.start()

Task {
    for await state in session.stateStream() {
        switch state {
        case .started:
            // Confirm UI shows session is live
        case .paused:
            // Keep connection, wait for started or stopped
        case .stopped:
            // Release resources, allow user to restart
        default:
            break
        }
    }
}

  8. Configure Info.plist for Display access

    main

    Display apps require specific Info.plist configurations for URL schemes, background modes, and network permissions.

    Required Keys & Values:

    • URL Scheme: Set CFBundleURLTypes and route callbacks to Wearables.shared.handleUrl(_:).
    • MWDAT Configuration: Set AppLinkURLScheme, MetaAppID, ClientToken, and TeamID under the MWDAT key. (Note: MetaAppID = 0 is for Developer Mode).
    • Accessory Protocols: Include UISupportedExternalAccessoryProtocols with com.meta.ar.wearable.
    • Background Modes: Include external-accessory and bluetooth-central. The sample also includes bluetooth-peripheral and processing.
    • Bluetooth: Include NSBluetoothAlwaysUsageDescription.
    • Local Network (High-bandwidth/Video): Include a non-empty NSLocalNetworkUsageDescription and NSBonjourServices to allow Display to acquire high-bandwidth link leases.

  12. Build and run the Display Access App sample

    main

    The Display Access App is a sample iOS application that demonstrates how to use the Meta Wearables Device Access Toolkit (DAT SDK) Display module to send visual content and interactive tutorials to Meta Ray-Ban Display glasses.

    Prerequisites

    • iOS: 17.2+
    • Xcode: 26.4+
    • Swift: 6.3+
    • Hardware: A Meta Ray-Ban Display glasses device
    • SDK: Meta Wearables Device Access Toolkit (included as a dependency)

    Build Steps (Xcode)

    1. Clone the repository.
    2. Open the project in Xcode.
    3. Select your target device.
    4. Build the project using Cmd+B or the Build button.
    5. Run the project using Cmd+R or the Run (▶️) button.
    # Clone the repository to begin
git clone https://github.com/facebook/meta-wearables-dat-ios.git