maestro-runner Documentation

repository·main·Indexed 19 days ago

https://github.com/devicelab-dev/maestro-runner

A high-performance, JVM-free, open-source alternative to Maestro for UI test automation. It executes Maestro YAML flows on Android, iOS, Web, and cloud providers using a pluggable driver architecture including UIAutomator2, DeviceLab, WDA, and Appium. Features include parallel execution, visual regression testing via assertScreenshot, and integration with cloud Appium providers for automated reporting and artifact upload.

Tokens
28.9K
Snippets
83
Records
132
Agent score
64%

What's inside maestro-runner

  1. Overview of devicelab-ios-runner

    main

    The devicelab-ios-runner is an XCUITest-based iOS automation runner designed for use with the devicelab driver in the maestro-runner project. It serves as the iOS counterpart to the devicelab-android-driver.

    Its primary objectives are to provide a self-contained, owned XCUITest runner that eliminates runtime third-party downloads and ensures local parity with upstream Maestro flows, including matching retry profiles.

  2. What is WebDriverAgent

    main

    WebDriverAgent is a WebDriver server implementation for iOS and tvOS. It enables remote control of devices and simulators by linking XCTest.framework and calling Apple's APIs directly.

    Key capabilities include:

  3. Key design improvements over standard Maestro

    main

    maestro-runner addresses several architectural limitations found in the original Maestro runner to make it suitable for professional CI/CD and parallel execution:

    • Parallel Execution: Unlike Maestro, which uses hardcoded ports (e.g., 7001 for Android gRPC), maestro-runner uses dynamic ports, enabling multiple tests to run in parallel on the same machine.
    • Configurable Timeouts: While Maestro has hardcoded timeouts, maestro-runner allows you to configure timeouts per-flow and per-command.
    • Reliable Text Input: Replaces flaky character-by-character input with Appium Unicode IME or direct UIAutomator2 to ensure Unicode support and stability under load.
    • Cloud Support: Provides first-class support for remote device providers via Appium, including BrowserStack, Sauce Labs, LambdaTest, and TestingBot.
    • Modular Codebase: Replaces the monolithic Orchestra.kt orchestrator with small, focused, and decoupled components.
  4. Understand the devicelab-ios-runner Wire Protocol v1

    main

    The devicelab-ios-runner uses an internal HTTP+JSON protocol to communicate between the maestro-runner Go consumer and an on-device XCUITest runner.

    Key Design Principles:

    • Stateless app binding: Every command must include an appBundleId. The runner caches the application instance until the ID changes.
    • No element handles: Operations target coordinates or fresh selectors rather than persistent element IDs to prevent stale-element bugs.
    • Full hierarchy snapshots: The snapshot command returns the complete UI tree without filtering or occlusion logic. Selectors are processed on the Go side.
    • Inline binary payloads: Screenshots and recordings are returned as base64 encoded strings within the JSON envelope.
    • Hard timeouts: Every command has a 30s server-side timeout ceiling.

    Transport Details:

    • Protocol: HTTP/1.1 over TCP
    • Endpoint: POST /command
    • Content-Type: application/json; charset=utf-8
    • Health Check: GET /health returns 200 when the runner is ready.
  5. Use the DeviceLab driver for faster Android testing

    main

    The devicelab driver is an optimized Android driver that runs automation directly on the device via WebSocket, bypassing the UIAutomator2 HTTP layer. It is approximately 2x faster than the default UIAutomator2 driver and 5x faster than the standard Maestro CLI. It includes features like bounds stabilization for animated elements and improved special character handling.

    maestro-runner --driver devicelab --platform android test flows/
  6. Understand the iOS Runner wire protocol

    main

    The communication protocol between the maestro-runner Go driver and the iOS runner is based on the agent-device contract. This includes:

    • Command/Response shapes: Defined in RunnerTests+Models.swift.
    • Transport: HTTP/JSON transport defined in RunnerTests+Transport.swift.
    • Dispatch logic: Defined in RunnerTests+CommandExecution.swift.

    The Go driver located in pkg/driver/devicelab_ios/ is responsible for translating Maestro flow steps into these specific command shapes.

  7. How the maestro-runner architecture works

    main

    The maestro-runner operates through a three-stage pipeline to execute automation flows:

    1. YAML Parser (pkg/flow): Converts Maestro YAML flow files into typed Step structures.
    2. Executor (pkg/executor): Orchestrates the execution lifecycle and delegates specific commands to the active Driver implementation.
    3. Report Generator (pkg/report): Compiles execution data into JSON or HTML reports.

    This separation allows the runner to support multiple device backends (drivers) while maintaining a consistent flow definition and reporting format.

  8. How Cloud Provider Integration works

    main

    maestro-runner automatically detects and interacts with cloud Appium providers based on the --appium-url flag. The integration follows a three-step lifecycle:

    1. Detect: After parsing --appium-url, registered providers check if the URL matches their identifier (e.g., containing "saucelabs").
    2. Extract metadata: Once an Appium session is created, the provider reads session capabilities and stores provider-specific data (like jobID or session type) in a map[string]string.
    3. Report result: After all flows and reports are finished, the provider receives the TestResult and reports the pass/fail status to the cloud API.

    No additional flags are required for detection and reporting to function.

  9. How maestro-runner architecture works

    main

    maestro-runner is designed as a pluggable execution engine for Maestro YAML flow files. It decouples the test definition from the execution backend through three independent layers:

    1. YAML Parser (pkg/flow): Responsible for parsing Maestro YAML files into typed step structures. This layer is isolated so that changes to the YAML syntax do not affect the drivers.
    2. Driver (pkg/core, pkg/driver): A standardized interface that all backends must implement. This allows the runner to support multiple execution engines (like UIAutomator2, Appium, or WDA) interchangeably.
    3. Report (pkg/report): Consumes the results of the execution to generate outputs such as JSON or HTML reports. This layer is independent of both the parser and the driver.

    This separation allows you to add new drivers (e.g., Detox) or new report formats without modifying the parser or existing drivers.

  10. Set Sauce Labs job and test names

    main

    The name displayed in Sauce Labs for a job or session is determined by the following priority:

    1. sauce:options.name: If provided in your capabilities JSON, this value is used to name the session/job.
    2. YAML Flow Basename: If no name is provided in capabilities, the runner uses the filename of the Maestro flow (excluding the .yaml or .yml extension).

    Note: If multiple flows run sequentially within a single Appium session, the session name is taken from the first flow in that sequence.

    "sauce:options": {
      "name": "My regression suite",
      "build": "Maestro Android Run",
      "appiumVersion": "latest"
    }
  11. How iOS Runner handles SecureTextField typing

    main

    To prevent character loss in React Native SecureTextField components on iOS Simulators (where hasKeyboardFocus often remains false), the runner uses a synthetic typing path.

    When a target is identified as a .secureTextField, the runner uses DLSendSyntheticTyping (via SyntheticTyping.h/m) which bypasses standard keyboard-focus checks. For non-secure fields, the runner continues to use the standard app.typeText method. If the synthetic path fails, it falls back to app.typeText to ensure compatibility.