DOCSEARCHbyAlgolia
Submit Docs

Marathon Cloud Documentation

website·Indexed 33 days ago

https://docs.marathonlabs.io/

Cloud testing infrastructure and mobile development tools for running Android and iOS UI test suites. Features include support for XCTest, XCUITest, Patrol, KIF, and Maestro, as well as a CLI (v1.0.65), CI/CD integration with GitHub Actions, CircleCI, and Bitrise, and tools for batching, parallelism, and test filtering.

Tokens
24.8K
Snippets
103
Records
216
Agent score
65%

What's inside Marathon Cloud

  1. Overview of marathon-cloud CLI commands

    The marathon-cloud CLI is used to submit test runs, list device configurations, and download artifacts. Available commands include:

    • run android: Run Android instrumentation tests.
    • run ios: Run XCTest / XCUITest suites on iOS simulators.
    • run maestro android: Run Maestro flows on Android.
    • run maestro ios: Run Maestro flows on iOS.
    • devices android: List available Android device configurations.
    • download: Download artifacts and reports for a run.
    • completions: Generate shell completion scripts.

  2. Overview of Marathon Cloud

    Marathon Cloud is a platform for running Android and iOS UI test suites in parallel on a pool of virtual devices. It is designed to provide fast verdicts for gating pull requests by automating device provisioning, test distribution, and flaky test retries. The platform is based on the open-source Marathon test runner.

  3. Understand PID following and crash detection in Logcat

    Emu provides automatic process tracking and error detection:

    • PID following: Emu tracks your application's Process ID (PID) in real time. If the app restarts or crashes, Emu automatically detects the new PID and re-attaches the log stream.
    • Crash detection: When a FATAL EXCEPTION occurs, Emu captures and flags the stack trace. If capture is active, these crashes are also included in bug replay reports.

  4. Run Android tests using marathon-cloud

    The marathon-cloud run android command is used to execute Android instrumentation tests. It supports specifying application and test APKs, configuring the emulator environment (OS version, system image, and architecture), and managing device-specific settings like cameras and mock locations.
    # Example: Run Android tests with a specific OS version and system image
marathon-cloud run android \
  --application app-debug.apk \
  --test-application app-tests-debug.apk \
  --os-version 13 \
  --system-image google_apis

  5. Understand Marathon Cloud retry types

    Marathon Cloud retries individual tests (not whole suites) using three distinct mechanisms:

    1. Reactive Retries: Triggered after a test fails. The test is rescheduled on a different device. If it eventually passes, it is marked as 'flaky'; if it fails all attempts, it is marked as 'failed'.
    2. Preventive Retries: Based on historical instability, the scheduler may run a test multiple times from the start to reach a verdict faster, without waiting for an initial failure.
    3. Uncompleted Executions: Tests that produce no result (e.g., device crash, process killed, or timeout) are treated as 'uncompleted'. These use a separate retry quota so infrastructure issues do not exhaust the quota for actual test failures.

  6. Ensure test run isolation in Marathon Cloud

    Marathon Cloud provides device isolation to prevent state leakage between users or organizations. Each test run uses a freshly provisioned virtual device that is destroyed immediately upon completion, ensuring no data persists between runs.

  7. Understand Marathon Cloud parallelism and batching defaults

    Marathon Cloud automatically optimizes test runs based on suite history:

    • Parallelism: The scheduler estimates total execution time and sizes the device pool to target a total run time of approximately 15 minutes.
    • Batching: Tests are grouped into batches sent to a single device. Tests within a batch run sequentially in the same app process. This amortizes the overhead of transferring artifacts and receiving new tests. Because tests in a batch share a process, state (e.g., logged-in sessions, files) left by one test is visible to subsequent tests in that same batch.

  8. Configure Android emulator architecture

    Android emulators default to amd64 (x86_64). Use --arch arm64 if your app bundles native libraries without an x86_64 ABI or to match production device architecture.

    Requirements:

    • amd64: Requires google_apis or google_apis_playstore. Supported on any Android version.
    • arm64: Supported on Android 9 or later. Compatible with all system images, including default.

    Note: default images are only available for arm64. The combination --system-image default --arch amd64 is rejected.

  9. Understand key architectural differences between BrowserStack and Marathon Cloud

    Before migrating, be aware of these fundamental differences:

    • Virtualization: BrowserStack uses real devices. Marathon Cloud uses Android emulators and ARM64 iOS simulators. Tests relying on OEM-specific behavior or physical hardware must be reviewed.
    • Parallelism Logic: Marathon Cloud automatically sizes the test pool based on historical duration to target a 15-minute run, rather than requiring a fixed number of parallel slots.
    • Network Access: Marathon Cloud does not have a tunnel equivalent to 'BrowserStack Local'. All test backends (staging/test environments) must be reachable over the public internet.
    • Pricing Model: BrowserStack bills per parallel slot per month. Marathon Cloud bills by device time per second ($2/hour for Android, $3/hour for iOS).

  10. Set up Live Mirror for a device

    To start mirroring a device screen at native resolution with GPU acceleration, follow these steps:

    1. Connect a physical device via USB or start an emulator.
    2. Select the device from the sidebar list.
    3. The live mirror will appear in the main workspace.

  12. Integrate Android tests into CI gating

    The marathon-cloud command returns exit code 1 if any test fails and 0 otherwise. Use --result-file for a JSON summary and --output to save JUnit XML and videos.
    marathon-cloud run android \
  --application app-debug.apk \
  --test-application app-debug-androidTest.apk \
  --result-file result.json \
  --output ./results