Maestro

Maestro Studio Desktop is a lightweight, visual IDE for designing and executing tests without using the terminal.

Capabilities:

• Visual Flow Builder & Inspector: Record interactions, inspect UI elements, and build flows visually.
• Platform Support: Available as a native app for macOS, Windows, and Linux.

Note: Maestro Studio is a free tool but is not an open-source project.

The demo_app is a cross-platform Android and iOS application built with Flutter, specifically designed for testing the Maestro framework.

In the Maestro E2E (End-to-End) pipeline, the app follows this lifecycle:

1. Binaries are built.
2. Binaries are uploaded to a Google Cloud Storage (GCS) bucket.
3. Links to these binaries are maintained in e2e/manifest.txt.
4. During E2E pipeline execution, the binaries are downloaded from GCS and the flows are executed within the e2e/workspaces/demo_app workspace.

Maestro Cloud provides dedicated infrastructure to run large test suites in parallel, which can reduce execution times by up to 90%.

Features:

• Parallel Execution: Run hundreds of tests simultaneously.
• Deterministic Environments: Consistent testing environments.
• Debugging Tools: Includes built-in notifications and complete debugging capabilities.

To ensure that LLMs can correctly call new tools and interpret their outputs, use the Maestro evals framework.

1. Add test cases to ./maestro-cli/src/test/mcp/maestro-evals.yaml.
2. Run the evaluation suite using the following command, providing your Anthropic API key:

ANTHROPIC_API_KEY=<your_key> ./maestro-cli/src/test/mcp/run-mcp-server-evals.sh

To publish a new version of the Maestro CLI, trigger the publish-cli.yaml GitHub action. After the action completes, verify the installation by running the official install script and checking the version.

Verification Steps:

1. Install the CLI using the curl script.
2. Run maestro --version to ensure the version matches the intended release.

# Install the CLI
curl -Ls "https://get.maestro.mobile.dev" | bash

# Verify version
maestro --version

Maestro uses a human-readable YAML syntax to define UI and end-to-end tests called Flows. Flows are interpreted (not compiled), allowing for fast iteration.

Key features include:

• Cross-platform coverage: Test Android, iOS, and web apps (including React Native and Flutter) on emulators, simulators, or real devices.
• Resilience: Built-in flakiness tolerance and automatic waiting handle dynamic UIs without requiring manual sleep() calls.
• Simple Commands: Express interactions using commands like launchApp, tapOn, inputText, and assertVisible.

# flow_contacts_android.yaml

appId: com.android.contacts
---
- launchApp
- tapOn: "Create new contact"
- tapOn: "First Name"
- inputText: "John"
- tapOn: "Last Name"
- inputText: "Snow"
- tapOn: "Save"

To change the launch screen image for the iOS version of your Flutter app, you can use either of the following methods:

1. Direct File Replacement: Replace the existing image files located in the ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory with your own assets.
2. Xcode Interface:
   • Open the iOS project in Xcode using the command: open ios/Runner.xcworkspace.
   • In the Xcode Project Navigator, navigate to Runner/Assets.xcassets.
   • Drag and drop your desired images into the asset catalog.

The maestro-ai-demo executable is used to run AI-powered tests on screenshots to identify defects.

Single Screenshot Test

To run a test for a single screenshot that contains defects:

maestro-ai-demo foo_1_bad.png

Batch Test with Verbose Output

To run tests for multiple screenshots (e.g., using a wildcard pattern) and inspect the AI's reasoning, use the --model, --show-prompts, and --show-raw-response flags:

maestro-ai-demo \
  --model gpt-4o-2024-08-06 \
  --show-prompts \
  --show-raw-response \
  test-ai-fixtures/uber_*_bad.png

maestro-ai-demo --model gpt-4o-2024-08-06 --show-prompts --show-raw-response test-ai-fixtures/uber_*_bad.png

To update the local sample files and flows, run the provided update script from the e2e directory.

./update_samples

The Maestro Viewer is a React application located in maestro-cli/mcp-viewer. It uses npm, Vite, and Tailwind CSS v4.

To iterate on the UI locally without rebuilding the entire CLI, you can run the development server:

1. Navigate to the viewer directory.
2. Install dependencies.
3. Run the dev command.

The viewer communicates via HTTP: it accepts events at POST /api/events and streams them via GET /api/events/stream.

cd maestro-cli/mcp-viewer
npm install
npm run dev