DOCSEARCHbyAlgolia
Submit Docs

mcp-remote-macos-use

repository·main·Indexed 19 days ago

https://github.com/baryhuang/mcp-remote-macos-use

An open-source Model Context Protocol (MCP) server that enables AI agents to remotely control macOS systems via Screen Sharing and VNC. It provides tools for screen observation, mouse control (move, click, scroll, drag-and-drop), keyboard input, and application launching via Spotlight. The server supports Apple Authentication (protocol 30) and can be deployed via Docker, with optional WebRTC support through LiveKit for low-latency streaming.

Tokens
5.8K
Snippets
16
Records
30
Agent score
66%

What's inside mcp-remote-macos-use

  1. Build and Publish the MCP Server Docker Image

    main

    Developers can build the image locally or use buildx for cross-platform publishing.

    Local Build

    docker build -t mcp-remote-macos-use .

    Cross-Platform Publishing (AMD64 & ARM64)

    1. Create a builder:
      docker buildx create --use
    2. Build and push:
      docker buildx build --platform linux/amd64,linux/arm64 -t buryhuang/mcp-remote-macos-use:latest --push .
    3. Verify:
      docker buildx imagetools inspect buryhuang/mcp-remote-macos-use:latest
    docker buildx build --platform linux/amd64,linux/arm64 -t buryhuang/mcp-remote-macos-use:latest --push .

  2. Install the Remote macOS MCP Server

    main

    To use this MCP server with Claude Desktop, you must first prepare the target macOS machine and then configure your local Claude Desktop environment to run the server via Docker.

    1. Prepare the Target macOS

    • Enable Screen Sharing: On the remote Mac, enable Screen Sharing in System Settings. (Note: If using a Mac from macstadium.com, this step can be skipped).
    • Network Access: Ensure you can connect to the remote Mac via VNC/Screen Sharing.

    2. Local Setup

    • Install Docker Desktop on your local machine.
    • Add the server to your Claude Desktop configuration by editing your claude_desktop_config.json file.
    {
  "mcpServers": {
    "remote-macos-use": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "-e",
        "MACOS_USERNAME=your_macos_username",
        "-e",
        "MACOS_PASSWORD=your_macos_password",
        "-e",
        "MACOS_HOST=your_macos_hostname_or_ip",
        "--rm",
        "buryhuang/mcp-remote-macos-use:latest"
      ]
    }
  }
}

  3. Configure Remote macOS Connection Environment Variables

    main

    The MCP server uses environment variables to establish the connection to the remote macOS machine. When configuring the Docker command in your MCP client (like Claude Desktop), you must provide the following variables:

    • MACOS_USERNAME: The username on the remote macOS machine.
    • MACOS_PASSWORD: The password for the remote macOS user.
    • MACOS_HOST: The hostname or IP address of the remote macOS machine.

    If using WebRTC via LiveKit for low-latency streaming, you must also configure the relevant LiveKit environment variables within the same Docker run command.

  4. Configure LiveKit for WebRTC support

    main

    To enable WebRTC support via LiveKit, provide the following environment variables. If these are set, the server will automatically initialize a LiveKitHandler and attempt to join the room remote-macos-room.

    # Example LiveKit configuration
LIVEKIT_URL=https://your-livekit-instance.com
LIVEKIT_API_KEY=your_api_key
LIVEKIT_API_SECRET=your_api_secret

  5. Configure VNC connection environment variables

    main

    The MCP server requires specific environment variables to establish a VNC connection to the remote macOS machine. These must be set before starting the server.

    Required Variables:

    • MACOS_HOST: The IP address or hostname of the remote macOS machine.
    • MACOS_PASSWORD: The VNC password for the remote machine.

    Optional Variables:

    • MACOS_PORT: The VNC port (defaults to 5900).
    • MACOS_USERNAME: The VNC username.
    • VNC_ENCRYPTION: Encryption preference (defaults to prefer_on).
    # Example .env file content
MACOS_HOST=192.168.1.100
MACOS_PORT=5900
MACOS_USERNAME=admin
MACOS_PASSWORD=your_secure_password
VNC_ENCRYPTION=prefer_on

  6. Use Remote macOS Control Tools

    main

    The server exposes several tools to allow an AI agent to interact with the remote macOS desktop. All tools automatically use the connection details provided via environment variables.

    Available Tools:

    • remote_macos_get_screen: Captures a screenshot of the remote desktop.
    • remote_macos_send_keys: Sends keyboard input.
    • remote_macos_mouse_move: Moves the cursor (includes automatic coordinate scaling).
    • remote_macos_mouse_click: Performs a single click.
    • remote_macos_mouse_double_click: Performs a double click.
    • remote_macos_mouse_scroll: Performs a mouse scroll.
    • remote_macos_mouse_drag_n_drop: Performs a drag-and-drop operation from a start point to an end point.
    • remote_macos_open_application: Opens or activates an application and returns its PID.

  7. Configure LiveKit environment variables

    main

    The LiveKitHandler requires three environment variables to be set for successful connection and operation. If any of these are missing, the handler will log a warning and fail to initialize correctly.

    Required environment variables:

    • LIVEKIT_URL: The URL of the LiveKit server.
    • LIVEKIT_API_KEY: Your LiveKit API key.
    • LIVEKIT_API_SECRET: Your LiveKit API secret.

  8. Configure `VNCClient` connection parameters

    main

    When initializing a VNCClient, you can specify the following parameters:

    ParameterTypeDefaultDescription
    hoststrRequiredRemote macOS hostname or IP address
    portint5900VNC port
    passwordstrNoneVNC password (required for Apple Auth)
    usernamestrNonemacOS username
    encryptionstr`

  9. Authentication and Security Limitations

    main

    The server has specific constraints regarding authentication protocols:

    • Supported Authentication: Only Apple Authentication (protocol 30) is supported.
    • Protocol Details: Protocol 30 uses the Diffie-Hellman key agreement protocol with a 512-bit prime. This is typically used when macOS 11 to macOS 12 communicates with OS X 10.11 or earlier clients.
    • Security Warning: Only use this tool with servers you trust and have explicit permission to access. Always use secure, authenticated connections.

  10. Troubleshoot Apple VNC Authentication failures

    main

    If connect() fails with Apple Authentication (type 30), check the following common causes:

    • Error Code 560513588: This often indicates:
      1. Password encryption/encoding mismatch.
      2. Screen Recording permission has not been granted to the service/app attempting the connection.
      3. Remote Management or Screen Sharing is not enabled in macOS System Preferences.
    • Missing Dependencies: If Diffie-Hellman key exchange fails, ensure the cryptography library is installed:
      pip install cryptography
    • Protocol Support: This client only supports Apple Authentication (protocol 30). Ensure the target macOS machine is configured to allow this.

  11. Use LiveKitHandler to manage data communication

    main

    The LiveKitHandler class provides an interface for connecting to a LiveKit room and communicating via data channels. It is designed to work with auto_subscribe=False to focus on data messaging rather than media streaming.

    Key Methods:

    • start(room_name: str, token: str) -> bool: Initializes the Room object and connects to the specified room using the provided token. Returns True if successful.
    • send_data(message: str, reliable: bool = True) -> bool: Encodes a string message as UTF-8 and publishes it to all participants. Use reliable=True for guaranteed delivery (RELIABLE) or reliable=False for faster, non-guaranteed delivery (LOSSY).
    • register_message_handler(message_type: str, handler: Callable): Registers a callback function to be executed when a specific string message is received.
    • stop(): Disconnects from the current LiveKit room.
    from mcp_remote_macos_use.livekit_handler import LiveKitHandler

handler = LiveKitHandler()

# Define a callback for a specific message type
async def my_callback(participant):
    print(f"Received custom message from {participant.identity}")

# Register and start
handler.register_message_handler("my_event", my_callback)

success = await handler.start("my-room", "my-access-token")
if success:
    await handler.send_data("my_event")
    # ... later
    await handler.stop()