uiautomation-rs

The MatcherFilter trait is the core abstraction for defining criteria to select specific UIElements. You can implement this trait yourself or use the built-in filter types to build complex matching logic for use with UIMatcher.

Built-in Filters

• NameFilter: Matches an element based on its name.
  • value: The string to match.
  • casesensitive: Whether the match should respect casing.
  • partial: If true, performs a substring match (contains); if false, performs an exact match.
• ClassNameFilter: Matches an element by its exact class name.
  • classname: The string representing the class name.
• ControlTypeFilter: Matches an element by its ControlType.
  • control_type: The ControlType enum value to match.
• FnFilter<F>: Allows you to wrap a custom closure as a filter.
  • filter: A closure with the signature Fn(&UIElement) -> Result<bool>.
• AndFilter: Combines two filters; returns true only if both pass.
• OrFilter: Combines two filters; returns true if either passes.

Process Filtering (Requires process feature)

• ProcessIdFilter: Matches elements belonging to a specific process or its child processes.
  • pid: The target Process ID.
  • sub_progress: If true, the filter will automatically discover and include all child processes of the target PID.

```rust
// Example of using built-in filters
let name_filter = NameFilter {
    value: "Submit".to_string(),
    casesensitive: false,
    partial: true,
};

let type_filter = ControlTypeFilter {
    control_type: ControlType::Button,
};

// Combining filters with AndFilter
let combined = AndFilter::new(
    Box::new(name_filter),
    Box::new(type_filter)
);
```埋

The uiautomation_derive crate provides procedural macros to automatically implement common Windows UI Automation patterns for your custom types. By deriving these patterns, your types gain access to high-level methods that interact with the underlying UIAutomation patterns (like Invoke, Window, Value, etc.).

When you derive a pattern, the macro generates code that:

1. Retrieves the appropriate pattern from the element using .get_pattern()?.
2. Delegates the method call to that pattern.

Commonly derived patterns include:

• Invoke: For triggering actions.
• Window: For controlling window state (maximize, minimize, close).
• Value: For getting or setting text values.
• Transform: For moving, resizing, or rotating elements.
• Selection: For managing selected items in a collection.
• Scroll: For scrolling within a view.
• ExpandCollapse: For expanding or collapsing elements.
• Toggle: For toggling states (like checkboxes).
• Grid and Table: For interacting with structured data.
• Text: For advanced text manipulation and range selection.

The Variant struct is a safe Rust wrapper around the Windows VARIANT type, used for handling heterogeneous data types in COM. It provides methods to inspect the type, convert to native Rust types, and perform arithmetic operations directly on the variant data.

Key capabilities:

• Type Inspection: Use get_type() to retrieve the VARENUM or is_string() / is_array() / is_null() for quick checks.
• Type Conversion: Use get_value() to convert the variant into a high-level Value enum, or use TryInto to convert directly to specific Rust types like String, bool, i32, f64, etc.
• Arithmetic: Perform operations like add, subtract, multiply, divide, mod_by, negate, not, and, or, and xor directly on Variant instances.
• Conversion to/from Value: The Value enum represents the possible data types held by a variant, including primitives, strings, arrays, and COM objects (UNKNOWN, DISPATCH).

use uiautomation::variants::{Variant, Value};

// Converting a Rust type to a Variant
let my_variant: Variant = 42.into();

// Converting a Variant to a Rust type
let val: i32 = my_variant.try_into().unwrap();

// Using the high-level Value enum
let value: Value = my_variant.get_value().unwrap();
match value {
    Value::STRING(s) => println!("It's a string: {}", s),
    Value::INT(i) => println!("It's an int: {}", i),
    _ => println!("Other type"),
}

// Arithmetic
let sum = my_variant.add(&other_variant)?;

The Screenshot struct provides several ways to capture visual data from the Windows desktop or specific UI elements.

Capture Methods

• Screenshot::capture_desktop(): Captures the entire virtual screen, including all connected monitors. It handles negative coordinates for multi-monitor setups.
• Screenshot::capture_rect(rect: Rect): Captures a specific rectangular region defined by a Rect. Coordinates are in screen pixels and support negative values for secondary displays.
• Screenshot::capture_element(element: &UIElement): A convenience method that retrieves the bounding rectangle of a UIElement and captures only that area.

Pixel Formats

By default, captured data uses the PixelFormat::BGRA format, which is raw GDI output (4 bytes per pixel, bottom-up row order). To use the data with most modern image libraries, convert it to top-down RGBA using to_rgba().

Saving Screenshots

• save_bmp(path): Saves the screenshot as a BMP file.
• save_png(path): Saves the screenshot as a PNG file. Note: This requires the png feature to be enabled in your Cargo.toml. It automatically converts the image to RGBA before encoding.

use uiautomation::screenshots::Screenshot;
use uiautomation::types::Rect;

// Capture a specific area
let rect = Rect::new(0, 0, 800, 600);
let screenshot = Screenshot::capture_rect(rect)?;

// Save as BMP
screenshot.save_bmp("screenshot.bmp")?;

// Or convert to RGBA for processing
let rgba_screenshot = screenshot.to_rgba();
println!("Width: {}, Height: {}", rgba_screenshot.width(), rgba_screenshot.height());

Use Process::create to launch an application and matcher to find specific windows. Once found, you can use send_keys to simulate keyboard input and try_into() to convert a UIElement into a specialized control like WindowControl.

use uiautomation::core::UIAutomation;
use uiautomation::processes::Process;

fn main() {
    Process::create("notepad.exe").unwrap();

    let automation = UIAutomation::new().unwrap();
    let root = automation.get_root_element().unwrap();
    let matcher = automation.create_matcher().from(root).timeout(10000).classname("Notepad");
    if let Ok(notepad) = matcher.find_first() {
        println!("Found: {} - {}", notepad.get_name().unwrap(), notepad.get_classname().unwrap());

        notepad.send_keys("Hello,Rust UIAutomation!{enter}", 10).unwrap();

        let window: WindowControl = notepad.try_into().unwrap();
        window.maximize().unwrap();
    }
}