Marionette MCP

Flutter allows only one WidgetsBinding per process. Since MarionetteBinding is a binding, calling MarionetteBinding.ensureInitialized() while another binding (like AutomatedTestWidgetsFlutterBinding or IntegrationTestWidgetsFlutterBinding) is already active will throw an error.

Avoiding conflicts during flutter test

If your tests run in debug mode, they might trigger the Marionette initialization. Use one of these two strategies:

Option A: Skip Marionette when running tests Check for the FLUTTER_TEST environment variable:

import 'dart:io' show Platform;

final isFlutterTest = Platform.environment.containsKey('FLUTTER_TEST');
if (kDebugMode && !isFlutterTest) {
  MarionetteBinding.ensureInitialized();
} else {
  WidgetsFlutterBinding.ensureInitialized();
}

Option B: Use a separate test entrypoint Keep MarionetteBinding in your production lib/main.dart and create a separate file (e.g., lib/main_test.dart) for tests that does not initialize MarionetteBinding.

To avoid degrading the experience for screen-reader users (VoiceOver/TalkBack) while still providing agents with necessary information, follow these patterns:

• Pattern A (Clean Text): Use label for human-friendly strings and let Text.rich.toPlainText() handle the rendered version. The agent sees both Type: Semantics and Type: Text.
• Pattern B (Stateful Controls): Use label for what the control is and value for its current state (e.g., label: 'Volume', value: '70%').
• Pattern C (Machine Selectors): Use Semantics.identifier for stable handles used by agents to target elements (e.g., identifier: 'submit_button'). The identifier is never announced by screen readers, making it the recommended home for machine-readable selectors.

If you need an agent to see raw markup that would be annoying for a screen reader to announce, use a custom widget with extractText configuration instead of Semantics.label.

When you provide an inputSchema to registerMarionetteExtension, the MCP server promotes the extension to a first-class, individually-named tool. This provides the agent with argument hints, enum autocomplete, and automatic validation. Invalid arguments are rejected by the MCP server before reaching your app.

Schema Rules:

• Flat Scalars Only: Schemas are restricted to flat scalar parameters because arguments travel over the VM service as a Map<String, String>. Nested objects or arrays cannot be reliably round-tripped.
• Supported Types: Use ExtensionParam with .string(enumValues: [...]), .integer, .number, or .boolean.
• Default Values: If a property declares a defaultValue, that value (as a string) is automatically populated in the params map if the caller omits it.
• Parameter Parsing: The params map in your callback always arrives as Map<String, String>. You must manually parse non-string values (e.g., converting 'true' to true).

registerMarionetteExtension(
  name: 'appNavigation.goToPage',
  description: 'Navigates to a page by name.',
  inputSchema: ExtensionInputSchema(
    properties: {
      'page': ExtensionParam.string(
        description: 'The page to navigate to.',
        enumValues: ['home', 'profile', 'settings'],
      ),
      'animate': ExtensionParam.boolean(
        description: 'Whether to animate the transition.',
        defaultValue: true,
      ),
    ,
    required: ['page'],
  ),
  callback: (params) async {
    final page = params['page'];
    if (page == null) {
      return MarionetteExtensionResult.invalidParams('Missing required parameter: page');
    }
    // params is Map<String, String>; parse non-string values manually
    final animate = params['animate'] != 'false';
    navigateTo(page, animate: animate);
    return MarionetteExtensionResult.success({'page': page});
  },
);

To enable AI agents (like Claude Code, Cursor, or Gemini CLI) to interact with your running Flutter application, follow these five steps:

1. Prepare your app: Add the marionette_flutter package and initialize MarionetteBinding in your main.dart. It is recommended to only initialize it in kDebugMode to avoid overhead in production.
2. Install the bridge: Activate the MCP server globally using dart pub global activate marionette_mcp.
3. Configure your AI tool: Add the Marionette MCP server to your agent's configuration (e.g., using claude mcp add).
4. Run your app in debug mode: Execute flutter run and copy the VM service URI (e.g., ws://127.0.0.1:9101/ws) from the console.
5. Connect and interact: Provide the VM service URI to your AI agent and instruct it to start driving the app.

Note: Standard Material widgets work out of the box. If you use a custom design system, you must follow the Configuration guide so the agent can identify and interact with your custom elements.

# 1. Add the package
flutter pub add marionette_flutter

# 2. Activate the MCP server
dart pub global activate marionette_mcp

# 3. Example: Add to Claude Code
claude mcp add --transport stdio marionette -- marionette_mcp

While Text.rich and RichText are automatically flattened via toPlainText(), certain content types are invisible to agents. Use the Semantics widget to provide an explicit label or value for:

• Custom-painted text: Content rendered via CustomPaint or TextPainter.
• Lossy WidgetSpan content: Text or icons embedded within WidgetSpan children.
• Raw-source preservation: When you want the agent to see the markdown/markup source instead of rendered plaintext.

Marionette surfaces these in get_interactive_elements as Type: Semantics, text: "<label>". If both label and value are provided, Marionette joins them as 'label: value' in the discovery output.

// Structural label + dynamic value.
// Screen reader: "Volume, 70 percent"
// Agent: Type: Semantics, text: "Volume: 70%"
Semantics(
  label: 'Volume',
  value: '70%',
  child: CustomPaint(painter: _VolumeBarPainter(level: 0.7)),
)