Terminator Documentation

The examples/ directory provides a variety of scripts demonstrating the automation capabilities of the Terminator framework across different platforms and use cases. You can use these as templates for your own automation tasks.

Available Example Categories:

• Windows Applications (Python): Automating native Windows tools like win_calculator.py (Calculator), notepad.py (Notepad), mspaint.py (MS Paint), and snipping_tool.py (Snipping Tool).
• Cross-Platform (Python): Platform-agnostic automation including monitor_example.py (monitor/UI info), element_screenshot.py (UI element screenshots and OCR), and vlc_auto_player.py (VLC media player control).
• Platform-Specific (Python): Linux-specific automation (e.g., gnome-calculator.py) and macOS-specific automation (e.g., macos_calculator.py).
• Web Automation (Python): Browser-based tasks such as gmail_automation.py.
• Complex Projects: Advanced implementations including pdf-to-form/ (PDF to web forms), recaptcha-resolver/ (reCAPTCHA solving), ai-explorer/ (AI UI exploration), and nextjs-workflows/ (Next.js integration).

When using engine mode (JavaScript or Python) via the run_command tool, data flows automatically between steps.

How it works:

1. Automatic Injection: env and variables are automatically injected into all scripts.
2. Auto-merging: Non-reserved fields returned from a script are automatically merged into the env object for subsequent steps.
3. Direct Access: Valid environment fields are available both as env.field_name and as direct variables (e.g., field_name).

Reserved Fields

The following fields are reserved and will not auto-merge into env:

• status
• error
• logs
• duration_ms
• set_env

Note: Data passing only works with engine mode (JavaScript/Python), not with shell commands.

steps:
  # Step 1: Return data directly
  - tool_name: run_command
    arguments:
      engine: "javascript"
      run: |
        const filePath = 'C:\\data\\report.pdf';
        const fileSize = 1024;

        return {
          status: 'success',
          file_path: filePath,      // Becomes env.file_path
          file_size: fileSize       // Becomes env.file_size
        };

  # Step 2: Access data automatically
  - tool_name: run_command
    arguments:
      engine: "javascript"
      run: |
        console.log(`Processing: ${env.file_path} (${env.file_size} bytes)`);
        console.log(`Direct access: ${file_path}`);

        const elements = await desktop.locator('role:button').all();

        return {
          status: 'success',
          file_processed: env.file_path,
          buttons_found: elements.length
        };

Data can be passed between steps using two primary methods:

1. Tool Result Storage

Any tool that defines an id field automatically stores its output in the environment. These are accessible in subsequent steps (especially in JavaScript run_command steps) using the following naming convention:

• {step_id}_result: The tool's return value (content, element info, etc.).
• {step_id}_status: Either "success" or "error".

2. Script Return Values

In a run_command step using the javascript engine, you can pass data to the environment by returning an object with a set_env key. In subsequent steps, these variables are available via direct access (no prefix required).

Example:

// Step 1: Set environment variables
return {
  set_env: {
    file_path: "C:/data/input.json",
    total_debit: "100.50",
  },
};

// Step 2: Access variables directly
const filePath = file_path; 
const debit = total_debit;

The recorder detects double clicks based on a 500ms time threshold and a 5-pixel distance tolerance. Double clicks are emitted as WorkflowEvent::Mouse events with the MouseEventType::DoubleClick variant.

To capture the UI element associated with a double click, ensure capture_ui_elements: true is set in your WorkflowRecorderConfig.

let config = WorkflowRecorderConfig {
    capture_ui_elements: true,  // Enable to capture UI elements on double clicks
    // ... other settings
};

// ... inside event loop ...
match event {
    WorkflowEvent::Mouse(mouse_event) => {
        match mouse_event.event_type {
            MouseEventType::DoubleClick => {
                println!("Double click at ({}, {})",
                    mouse_event.position.x,
                    mouse_event.position.y);

                if let Some(element) = &mouse_event.metadata.ui_element {
                    println!("Element: {} ({})",
                        element.name_or_empty(),
                        element.role());
                }
            }
            _ => {}
        }
    }
    _ => {}
}

Terminator supports several workflow formats for defining automation steps.

Direct Workflow (workflow.yml)

Uses a list of steps with tool_name and arguments.

steps:
  - tool_name: navigate_browser
    arguments:
      url: "https://example.com"
  - tool_name: click_element
    arguments:
      selector: "role:Button && name:Submit"
stop_on_error: true
include_detailed_results: true

Workflow with Conditional Jumps (workflow_with_jumps.yml)

Allows logic-based navigation between steps using jumps.

steps:
  - tool_name: validate_element
    id: check_logged_in
    arguments:
      selector: "role:Button && name:Logout"
    jumps:
      - if: "check_logged_in_status == 'success'"
        to_id: main_app
        reason: "User already logged in - skipping authentication"

  - tool_name: click_element
    id: login_flow
    arguments:
      selector: "role:Button && name:Login"
  # ... more login steps ...

  - tool_name: click_element
    id: main_app
    arguments:
      selector: "role:Button && name:Dashboard"

Tool Call Wrapper (workflow.json)

Wraps a single tool execution sequence.

{
  "tool_name": "execute_sequence",
  "arguments": {
    "steps": [
      {
        "tool_name": "navigate_browser",
        "arguments": {
          "url": "https://example.com"
        }
      }
    ]
  }
}

When using the run_command tool with a JavaScript engine (nodejs or quickjs), you have access to powerful desktop automation APIs via the desktop object.

Available APIs:

• Discovery: desktop.locator(selector).all(), desktop.locator(selector).first()
• Interaction: element.click(), element.type('text'), element.setToggled(true)
• Properties: element.name(), element.bounds(), element.enabled()
• Utilities: log('message'), sleep(ms)

Engines:

• nodejs: Full Node.js runtime with desktop APIs.
• quickjs: Lightweight JavaScript engine (default).

- tool_name: run_command
  arguments:
    engine: "javascript"
    run: |
      const submitButton = await desktop.locator('role:Button && name:Submit').first();
      if (await submitButton.enabled()) {
        await submitButton.click();
        return { action: 'submitted' };
      }
      return { action: 'disabled' };