SimDeck

repository·main·Indexed 40 days ago

https://github.com/nativescript/simdeck

A developer tool and macOS simulator control plane that allows developers to drive iOS simulators and Android emulators from IDEs or CLI. It includes a browser client, a Flutter Inspector package (simdeck_flutter_inspector), and a Swift-based Inspector Agent for exposing UIKit and SwiftUI view hierarchies.

Tokens
52.8K
Snippets
205
Records
278
Agent score
98%

What's inside SimDeck

  1. Use the SimDeck VS Code Extension

    main

    The SimDeck VS Code Extension allows you to open the local SimDeck browser client directly within a VS Code webview panel.

    By default, the extension attempts to connect to a service at http://127.0.0.1:4310. If a custom server URL is configured in your settings, it will use that instead. The extension manages the project service lifecycle by starting or reusing a service via simdeck service start only when required.

  2. Core workflows in SimDeck

    main

    SimDeck provides several categories of interaction for mobile simulators:

    • Visual Interaction: View live iOS Simulators or Android emulators in a browser. Perform actions like tapping, swiping, typing, pressing hardware buttons, rotating, and opening URLs.
    • Device Management: Install, launch, uninstall, boot, shut down, and erase devices.
    • Inspection & Capture: Capture screenshots, recordings, logs, pasteboard text, and accessibility trees. Inspect app UI via built-in accessibility or optional in-app inspectors.
    • Automation: Write JS/TS automation scripts using simdeck/test.
    • Collaboration: Share a paired browser session over your LAN or open the simulator view directly inside VS Code.
  3. Understand SimDeck data collection and privacy

    main

    SimDeck is designed to connect your iOS device to a SimDeck server of your choice. The privacy model is based on the following principles:

    Telemetry

    SimDeck uses a random app-generated identifier for telemetry. It does not use device advertising identifiers, analytics profiles, or tracking for ads/marketing.

    Excluded from telemetry:

    • Server addresses and pairing tokens
    • Simulator identifiers, names, or boot status
    • Device names or Apple IDs
    • IP-derived location or Tailscale node names
    • LAN hostnames, source code, project names, or file paths
    • User-entered text or payloads

    Local Data

    To facilitate reconnection, the app stores server addresses, pairing tokens, and connection preferences locally on your device. This data is not transmitted to SimDeck.

    Server Connections

    All requests, pairing credentials, simulator controls, and stream negotiation data are sent only to the specific SimDeck server you have selected or paired with. SimDeck does not operate an intermediate service that intercepts this data.

    Permissions

    • Camera: Used solely for scanning pairing QR codes to configure connections.
    • Local Network: Used to discover or connect to your SimDeck server.
    • Microphone: If requested for WebRTC functionality, audio is used only for the active connection and is not collected.
  4. How the Inspector source selection works

    main

    When using simdeck describe or the browser inspector, SimDeck attempts to use the highest-fidelity source available, falling back in a specific order:

    1. Framework runtime inspector: Uses specialized inspectors for frameworks like NativeScript, React Native, or Flutter.
    2. Swift in-app agent: Used for UIKit or SwiftUI apps.
    3. Native accessibility snapshot: The universal fallback if no specialized inspector is available.

    The response from the inspector includes metadata indicating which source was used and the reason for any fallbacks.

  5. How request flows work in SimDeck

    main

    Most user actions follow a centralized request flow through the service, which is why the long-lived service architecture provides better performance than calling low-level simulator tools directly:

    1. A command is sent from the Browser, CLI, or a test to the Local service.
    2. The service identifies the selected device and initiates a warm session if necessary.
    3. SimDeck executes the requested simulator or emulator action.
    4. The command returns data such as JSON, screenshots, recordings, logs, or updated stream state.
  6. Understand the SimDeck Inspector Protocol

    main

    The SimDeck Inspector protocol is a lightweight JSON-based protocol used by in-app inspectors to publish UI trees and handle debug actions. It is primarily intended for developers building or debugging an inspector runtime.

    Transport Mechanisms

    Depending on the runtime, the protocol is delivered via different transports:

    TransportUsed by
    TCP on 127.0.0.1:47370-47402Swift in-app agent
    WebSocket /api/inspector/connectNativeScript, React Native, Flutter, and other outbound runtimes
    Polling /api/inspector/poll plus /api/inspector/responseFallback when WebSocket is unavailable

    Protocol Envelope

    All messages follow a specific JSON structure for Requests, Success responses, Errors, and Events.

    Note: If the inspector was started with an auth token, all requests must include a matching top-level token field.

    // Request Example
    {
      "id": 1,
      "method": "View.getHierarchy",
      "params": { "maxDepth": 4 }
    }
    
    // Success Response Example
    {
      "id": 1,
      "result": { "roots": [] }
    }
    
    // Error Response Example
    {
      "id": 1,
      "error": {
        "code": -32004,
        "message": "No view was found for id view:0x1234."
      }
    }
    
    // Event Example
    {
      "event": "Inspector.connected",
      "params": { "protocolVersion": "0.1" }
    }
  7. Configure the advertised host for LAN and Tailscale

    main

    When using --bind 0.0.0.0, you must provide an --advertise-host value that the remote client can resolve.

    • Local Network: Use your machine's LAN IP (e.g., 192.168.1.50) or local hostname (e.g., my-mac.local).
    • Tailscale: Use your Tailscale IP or MagicDNS name. Note that Tailscale discovery does not use LAN broadcast across the tailnet, so explicit addressing is required.

    Warning: If you bind to 0.0.0.0 but advertise localhost, remote browsers will attempt to connect to themselves instead of your SimDeck host.

    simdeck --bind 0.0.0.0 --advertise-host my-mac.local --open
    simdeck --bind 0.0.0.0 --advertise-host 192.168.1.50 --open
    simdeck --bind 0.0.0.0 --advertise-host 100.101.102.103 --open
  8. How video streaming works in SimDeck

    main

    SimDeck uses a live WebRTC stream to show the device in the browser. The encoding process depends on the platform:

    • iOS: Frames are captured via the native display bridge and encoded on the Mac.
    • Android: Frames are captured from the emulator's -share-vid shared display surface and encoded on the Mac.

    SimDeck manages stream health by sending fresh frames and dropping stale ones if a client falls behind. Users can request refreshes via the UI.

  9. Debug React Native components with property edits and script evaluation

    main

    SimDeck provides two primary ways to interact with the running React Native app for debugging:

    • View.setProperty: A best-effort method to modify component properties. If the node resolves to a native host instance, the runtime uses setNativeProps(...). Note that the next React render cycle may overwrite these changes.
    • View.evaluateScript: Executes JavaScript within the app context. This is useful for diagnostics as it runs with useful React Native objects in scope.
  10. Secure SimDeck sessions with password protection

    main

    To protect simulator sessions, set a repository secret (e.g., SIMDECK_PASSWORD) and pass it to the action via the session_password input.

    When a password is provided, the action posts a SimDeck proxy link instead of the raw Cloudflare Tunnel URL. The daemon token is encrypted into the proxy payload, preventing access via the URL alone. To post raw tunnel links instead, set proxy_links: "false".

  11. Understand the SimDeck architecture and components

    main

    SimDeck is composed of several key pieces that work together to provide device inspection and control:

    • CLI: Starts the SimDeck service and provides scriptable commands.
    • Local service: The core engine that serves the browser UI, API, streams, metrics, and inspector routing.
    • Browser client: The web-based interface showing the live device, toolbar, inspector panes, and diagnostics.
    • Native bridge: Handles macOS-specific simulator operations.
    • Inspector runtimes: Optional app packages (like NativeScript, React Native, or Flutter) that provide rich UI tree data.
    • simdeck/test: A JS/TS wrapper designed for automation tasks.