RustAutoGUI

RustAutoGUI is a crate designed for automating interactions with other applications by controlling the mouse and keyboard. It is compatible with Windows, Linux, and macOS.

Core capabilities include:

• Screen Operations: Capturing the screen and finding images on the screen.
• Mouse Control: Moving to pixel coordinates, clicking buttons, dragging, and scrolling.
• Keyboard Control: Inputting strings, executing commands, and handling multiple key presses.
• Image Matching: Finding images on screen and moving the mouse to their location.

RustAutoGUI supports different MatchMode algorithms. Choosing the right one depends on your template and region:

General Advice: If you minimize template size and search regions, Segmented will generally perform faster.

Available MatchModes:

• MatchMode::Segmented
• MatchMode::FFT
• MatchMode::SegmentedOcl (Requires opencl feature)
• MatchMode::SegmentedOclV2 (Requires opencl feature)

pub enum MatchMode {
    Segmented,
    FFT,
    SegmentedOcl, // Only with opencl feature enabled
    SegmentedOclV2, // Only with opencl feature enabled
}

The choice between V1 and V2 algorithms affects performance and reliability.

V1 — Robust & Consistent

• Characteristics: Less sensitive to GPU performance variations; generally slower but more reliable across different hardware.
• Best Use Case: Use with non-_custom functions where the threshold is automatically determined. Recommended for general use.

V2 — Fast & Flexible

• Characteristics: Highly sensitive to the template and search images. Can be faster than V1 only when the threshold is perfectly tuned.
• Best Use Case: Use with _custom functions where you manually set the threshold.
• Warning: Not intended for low-tier or integrated GPUs. A bad threshold can result in performance worse than V1.

RustAutoGUI's underlying implementation varies by operating system:

• Windows: Interacts via winapi.
• Linux: Uses x11 (Note: Wayland is not supported).
• macOS: Uses the core-graphics crate.
• OpenCL: Utilized through the ocl crate.

You can install RustAutoGUI using cargo add or by manually adding it to your Cargo.toml.

Standard Installation

cargo add rustautogui

Feature Variants

• OpenCL Support: Enables OpenCL for potentially faster template matching. rustautogui = { version = "2.5.0", features = ["opencl"] }
• Lite Version: Includes only keyboard and mouse functions; excludes all template matching code. rustautogui = { version = "2.5.0", features = ["lite"] }
• Dev Version: Includes OpenCL and allows access to private modules. rustautogui = { version = "2.5.0", features = ["dev"] }

OS-Specific Setup

• Linux: You must install the X11 development libraries:

  sudo apt-get update
  sudo apt-get install libx11-dev libxtst-dev

- **macOS**: You must grant necessary permissions in your system settings for automation to work.

When performing image matching (available in non-lite versions), you can specify the MatchMode to determine the algorithm used.

Available modes:

• Segmented: Segmented correlation.
• FFT: Fourier transform correlation.
• SegmentedOcl: Segmented correlation using OpenCL (if opencl feature is enabled).
• SegmentedOclV2: An alternative OpenCL segmented correlation mode (if opencl feature is enabled).

// MatchMode is an enum used to select the correlation algorithm
pub enum MatchMode {
    Segmented,
    FFT,
    #[cfg(feature = "opencl")]
    SegmentedOcl,
    #[cfg(feature = "opencl")]
    SegmentedOclV2,
}

You can disable warnings emitted by the library using environment variables or directly in your code.

Via Environment Variables

Windows (PowerShell): $env:RUSTAUTOGUI_SUPPRESS_WARNINGS="1" (to turn off) $env:RUSTAUTOGUI_SUPPRESS_WARNINGS="0" (to turn on)

Windows (CMD): set RUSTAUTOGUI_SUPPRESS_WARNINGS=1 (to turn off) set RUSTAUTOGUI_SUPPRESS_WARNINGS=0 (to turn on)

Linux/macOS: export RUSTAUTOGUI_SUPPRESS_WARNINGS=1 (to turn off) export RUSTAUTOGUI_SUPPRESS_WARNINGS=0 (to turn on)

Via Code

let mut rustautogui = RustAutoGui::new(false).unwrap();
rustautogui.set_suppress_warnings(true);

Perform mouse scrolling in four directions.

rustautogui.scroll_up().unwrap();
rustautogui.scroll_down().unwrap();
rustautogui.scroll_left().unwrap();
rustautogui.scroll_right().unwrap();

Interact with the keyboard using strings or specific key commands. Note: Currently, only the US keyboard layout is fully implemented. Other layouts may cause incorrect character input.

Methods

• keyboard_input(string): Simulates a sequence of key presses for the provided string.
• keyboard_command(command): Presses a specific keyboard button (e.g., "backspace").
• keyboard_multi_key(key1, key2, Option<key3>): Executes multiple key presses simultaneously (e.g., combinations like Shift+Control+T).
• key_down(command): Presses a key down without releasing.
• key_up(command): Releases a pressed key.

rustautogui.keyboard_input("test!@#24").unwrap();
rustautogui.keyboard_command("backspace").unwrap();
rustautogui.keyboard_multi_key("shift", "control", Some("t")).unwrap();
rustautogui.key_down("backspace").unwrap();
rustautogui.key_up("backspace").unwrap();

To use the library, import the crate and create a new instance of RustAutoGui. The constructor accepts a boolean argument for debug mode.

use rustautogui;

let mut rustautogui = rustautogui::RustAutoGui::new(false); // arg: debug