DOCSEARCHbyAlgolia
Submit Docs

AXorcist

repository·main·Indexed 43 days ago

https://github.com/openclaw/axorcist

A Swift wrapper for macOS Accessibility APIs and a companion CLI tool (axorc) for automating, testing, and inspecting macOS user interfaces. It provides a type-safe, chainable API for macOS 14.0+ and a stable JSON protocol for programmatic UI automation, including features for hierarchy navigation, element matching, and real-time UI observation.

Tokens
6.5K
Snippets
22
Records
28
Agent score
56%

What's inside AXorcist

  1. Overview of AXorcist features

    main

    AXorcist is a Swift framework designed for macOS accessibility automation, enabling automated testing and UI tooling. Its core capabilities include:

    • Element Discovery: Locating UI elements via search criteria.
    • Action Execution: Performing interactions like clicks and key presses.
    • Attribute Access: Reading and modifying accessibility attributes.
    • Batch Operations: Efficiently executing multiple commands.
    • Permission Management: Handling macOS accessibility permissions.

  2. Navigate UI hierarchies using Path Navigation

    main

    You can navigate through UI hierarchies using path_from_root. This allows you to define a sequence of steps to reach a target element. Each step in the path can specify an attribute, a value, and a maximum search depth.

    Example JSON structure for path navigation:

    {
  "path_from_root": [
    {"attribute": "role", "value": "AXWindow", "depth": 1},
    {"attribute": "identifier", "value": "main-content", "depth": 3},
    {"attribute": "role", "value": "AXButton"}
  ]
}

  3. How AXorcist architecture and thread safety work

    main

    AXorcist is built around several core components: AXorcist (orchestrator), Element (AXUIElement wrapper), ElementSearch (traversal engine), and AXObserverCenter (notification management).

    Threading Model

    • MainActor Isolation: Accessibility element operations are isolated to the MainActor. Configuration and payload formatting also occur on the main actor.
    • Background Workers: Native observer work is handled by bounded background workers.
    • Logging: Global logging and clearing helpers enqueue requests on the main actor. Methods like axGetLogEntries and axGetLogsAsStrings return immutable snapshots.
    • Timeouts: AXTimeoutHelper.withTimeout runs async operations concurrently. It returns the first result, a timeout, or a cancellation. Note that timed-out or cancelled work may continue in the background and will not be automatically undone.
    • Synchronous Bounds: Use Element.withMessagingTimeout to bound synchronous native Accessibility messages.

  4. AXorcist framework component map

    main

    The AXorcist framework is organized into several functional areas:

    Essentials

    Core orchestration and response types: AXorcist, AXPermissionHelpers, AXCommandEnvelope, and AXResponse.

    Commands

    Instructions for the framework: AXCommand, QueryCommand, PerformActionCommand, and GetAttributesCommand.

    Elements

    Representations of UI components: Element, ElementSearch, and SearchCriteria.

    Utilities

    Supporting tools: GlobalAXLogger, AXLogEntry, and ProcessUtils.

    Error Handling

    Error types: AccessibilityError and AXError.

  5. Configure element matching strategies

    main

    When searching for elements, you can specify a match_type to control how the value is compared against the attribute. Supported strategies include:

    • exact: Exact string match (default)
    • contains: Case-insensitive substring match
    • regex: Regular expression match
    • containsAny: Matches if any comma-separated value is contained
    • prefix: String starts with the expected value
    • suffix: String ends with the expected value

  6. Verify Homebrew installation integrity

    main

    After installing axorc via Homebrew, you should perform three checks to ensure the installation is byte-for-byte identical to the published artifact and maintains its Developer ID signature:

    1. Byte-for-byte comparison: Compare the installed binary against the original artifact.
    2. Strict signature verification: Verify the Developer ID signature.
    3. Requirement inspection: Ensure the designated requirement contains anchor apple generic. An ad-hoc requirement is considered a release blocker.

    Note: The binary must be a Developer ID-signed universal binary. Do not use artifacts produced with the --adhoc flag, as they lack a stable macOS Accessibility identity.

    # Compare the original artifact with the installed binary
cmp axorc "$(brew --prefix axorc)/bin/axorc"

# Verify the signature strictly
codesign --verify --strict "$(brew --prefix axorc)/bin/axorc"

# Inspect the signing requirements
codesign -d -r- "$(brew --prefix axorc)/bin/axorc"

  8. Request accessibility permissions with AXPermissionHelpers

    main

    Before performing any accessibility automation, you must ensure your application has the necessary macOS accessibility permissions. Use AXPermissionHelpers.requestPermissions() to prompt the user or check current status. This is an asynchronous method.

    import AXorcist

let hasPermissions = await AXPermissionHelpers.requestPermissions()
if hasPermissions {
    let axorcist = AXorcist.shared
    // Begin accessibility operations
}

  9. Install AXorcist via Swift Package Manager

    main

    To use the AXorcist Swift API in your macOS project, add it as a dependency in your Package.swift file. Note that AXorcist requires macOS 14.0 or later.

    dependencies: [
    .package(url: "https://github.com/openclaw/AXorcist.git", from: "0.1.10")
]

  10. Use the AXorcist JSON Protocol for automation

    main

    AXorcist provides a stable JSON protocol for programmatic UI automation. Every command requires a command_id and a command name. Commands can be sent via the axorc raw subcommand using standard input, a file, or a JSON string argument.

    Supported Protocol Commands: ping, query, getAttributes, describeElement, getElementAtPoint, getFocusedElement, performAction, batch, observe, collectAll, stopObservation, isProcessTrusted, isAXFeatureEnabled, setFocusedValue, and extractText.

    Note on CLI vs JSON: The JSON protocol command names differ from the human-readable CLI. For example, the CLI tree command corresponds to the JSON collectAll command.

    # Using standard input
echo '{"command_id":"ping","command":"ping"}' | axorc raw --stdin

# Using a file
axorc raw --file command.json

# Using a JSON argument
axorc raw --json '{"command_id":"health","command":"ping"}'

  11. Install the AXorc CLI via Homebrew

    main

    You can install the signed, notarized universal command-line tool using Homebrew:

    brew install openclaw/tap/axorc

    After installation, run axorc permissions to ensure macOS has granted the necessary Accessibility permissions for inspection and automation.

  12. Verify notarization of the `axorc` executable

    main

    Because the axorc zip archives cannot be stapled, you must verify the notarization ticket of the downloaded executable online. Use the codesign command with the -R="notarized" flag to check the notarization status of the standalone binary.

    Note: Avoid using spctl --assess --type execute for standalone executables, as it may reject valid binaries that are not packaged as .app bundles. Instead, follow Apple's official guidance for testing notarized products and ensure networking is enabled during verification.

    codesign -vvvv -R="notarized" --check-notarization axorc