DOCSEARCHbyAlgolia
Submit Docs

hmdriver2 Documentation

repository·master·Indexed 19 days ago

https://github.com/codematrixer/hmdriver2

A non-intrusive UI automation framework for HarmonyOS NEXT (version 1.0.0) that allows developers to control devices, manage apps, and interact with UI elements using Python scripts without a pre-installed test runner app. It provides capabilities for app lifecycle management, device power and navigation control, HDC shell command execution, file operations, screen recording, and complex touch gestures. Includes support for XPath selectors, a UI Inspector (ui-viewer), and a toast watcher for notification monitoring.

Tokens
10.8K
Snippets
46
Records
55
Agent score
65%

What's inside hmdriver2

  1. Use UI Inspector for element discovery

    master
    The hmdriver2 ecosystem includes a UI Inspector (ui-viewer) which is a visual tool for inspecting the UI control tree. It allows you to view the hierarchy of controls and retrieve specific details for elements, which is essential for writing accurate selectors in your automation scripts.

  2. How UI element selectors (By) work

    master

    The uitest protocol uses several On methods to locate UI elements on the device. These selectors allow you to find components based on their properties or their relationship to other elements.

    Available selectors include:

    • On.text: Find an element by its displayed text.
    • On.id: Find an element by its unique ID.
    • On.key: Find an element by its key.
    • On.type: Find an element by its component type.
    • On.isAfter: Check if an element appears after a specific target element.
    • On.isBefore: Check if an element appears before a specific target element.

  3. Perform multi-pointer (gesture) actions with PointerMatrix

    master

    For complex gestures involving multiple fingers, use the PointerMatrix API to define points and then inject them via the Driver.

    1. Create a matrix: Use PointerMatrix.create(fingerCount, pointCount).
    2. Set points: Use PointerMatrix.setPoint(index, pointIndex, {x, y}) to define the coordinates for each finger.
    3. Inject action: Use Driver.injectMultiPointerAction(pointerMatrixId, duration) to execute the gesture.
    // 1. Create matrix with 1 finger and 104 points
// API: PointerMatrix.create(fingerCount, pointCount)
// Result: "PointerMatrix#0"

// 2. Set points
// API: PointerMatrix.setPoint(fingerIndex, pointIndex, {x, y})
// PointerMatrix.setPoint(0, 0, {"x": 65536630, "y": 984})

// 3. Inject
// API: Driver.injectMultiPointerAction(matrixId, duration)
// Driver.injectMultiPointerAction("PointerMatrix#0", 2000)

  4. Quick Start with hmdriver2

    master

    To use hmdriver2 for HarmonyOS NEXT UI automation, follow these steps:

    1. Configure HDC Environment: Download the Huawei Command Line Tools and add the toolchains directories to your PATH. You must also set HDC_SERVER_PORT (e.g., 7035).
    2. Connect Device: Connect your phone via USB, enable USB debugging, and verify connection using hdc list targets.
    3. Install Library: Install the core library via pip.
    4. Run Script: Initialize the Driver and start automating.

    Note: If you need screen recording functionality, install the opencv-python extra dependency.

    # 1. Configure environment (macOS example)
export HM_SDK_HOME="/Users/develop/command-line-tools/sdk/default"
export PATH=$PATH:$HM_SDK_HOME/hms/toolchains:$HM_SDK_HOME/openharmony/toolchains
export HDC_SERVER_PORT=7035

# 2. Install core library
pip3 install -U hmdriver2

# Optional: Install for screen recording support
pip3 install -U "hmdriver2[opencv-python]"

  5. Connect to a remote HDC Server

    master

    To perform automation on a remote device via an HDC Server, set the following environment variables before running your Python script:

    • HDC_SERVER_HOST: The IP address of the remote host.
    • HDC_SERVER_PORT: The port of the remote HDC server.

    To remove these settings, use unset HDC_SERVER_HOST and unset HDC_SERVER_PORT.

    export HDC_SERVER_HOST=127.0.0.1
export HDC_SERVER_PORT=8710

  6. Monitor AppState for application lifecycle

    master

    The AppState class provides integer constants representing the current lifecycle state of an application:

    • INIT (0): Initializing
    • READY (1): Initialized and ready
    • FOREGROUND (2): In the foreground
    • FOCUS (3): Focused (reserved/not currently supported)
    • BACKGROUND (4): In the background
    • EXIT (5): Exited

  7. Manage UI interaction with SwipeDirection and DisplayRotation

    master

    Use SwipeDirection to specify movement directions for gestures and DisplayRotation to manage screen orientation. DisplayRotation includes a from_value class method to convert integer values into the appropriate enum member.

    from hmdriver2.proto import SwipeDirection, DisplayRotation

direction = SwipeDirection.UP
rotation = DisplayRotation.from_value(1)  # Returns DisplayRotation.ROTATION_90

  8. Control device power and navigation

    master

    Use the following methods for basic device interaction:

    • d.go_home(): Go to Home screen
    • d.go_back(): Go back
    • d.screen_on(): Turn screen on
    • d.screen_off(): Turn screen off
    • d.unlock(): Unlock the screen
    • d.press_key(KeyCode.KEY_NAME): Press a hardware key (e.g., KeyCode.POWER)
    from hmdriver2.proto import KeyCode

d.go_home()
d.screen_on()
d.unlock()
d.press_key(KeyCode.POWER)

  9. Capture screenshots and screen recordings

    master

    Capture the device screen using screenshot() or screenrecord.

    Screen Recording: It is highly recommended to use the context manager (with statement) for screen recording to ensure resources are cleaned up automatically if the script crashes.

    Note: Requires opencv-python installed via pip3 install -U "hmdriver[opencv-python]".

    # Screenshot
d.screenshot("path/to/save.png")

# Recommended Screen Recording
with d.screenrecord.start("test2.mp4"):
    # do something
    time.sleep(5)

  10. Perform file operations (Push/Pull)

    master

    Transfer files between the local computer and the device:

    • d.pull_file(rpath, lpath): Download from device (rpath) to local (lpath).
    • d.push_file(lpath, rpath): Upload from local (lpath) to device (rpath).
    # Download from device to local
d.pull_file("/sdcard/test.txt", "C:/Users/Desktop/test.txt")

# Upload from local to device
d.push_file("C:/Users/Desktop/test.txt", "/sdcard/test.txt")

  11. Perform device touch gestures

    master

    Interact with the screen using various touch methods. Coordinates can be absolute pixels or relative percentages (0.0 to 1.0).

    Basic Gestures:

    • d.click(x, y): Single click
    • d.double_click(x, y): Double click
    • d.long_click(x, y): Long press
    • d.swipe(x1, y1, x2, y2, speed): Swipe from start to end. speed range: 200-40000 (default 2000).

    Advanced Gestures:

    • d.swipe_ext(direction, scale, box): Directional swipe. direction can be a string ("up", "down", etc.) or SwipeDirection enum. scale is distance percentage (0.1-1.0). box is the area (x1, y1, x2, y2).
    • d.gesture: Complex sequences of start(), move(), pause(), and action().
    # Click at 50% width, 50% height
d.click(0.5, 0.5)

# Swipe up in a specific area
d.swipe_ext("up", scale=0.8, box=(0.2, 0.2, 0.8, 0.8))

# Complex gesture (chaining)
d.gesture.start(0.5, 0.5).move(0.7, 0.7).pause(1).move(0.2, 0.2).action()

  12. Initialize the Driver

    master

    The Driver class is the main entry point for all automation tasks. You can initialize it without arguments to use the first device returned by hdc list targets, or provide a specific serial number obtained from hdc list targets to target a specific device.

    from hmdriver2.driver import Driver

# Use the first available device
d = Driver()

# Use a specific device serial
d = Driver("FMR0223C13000649")