mhxy_fz Automation Script

repository·master·Indexed 43 days ago

A computer vision-based automation script for the mobile version of 'Fantasy Westward Journey' (梦幻西游) on Windows. It utilizes pywin32 for window interaction and mouse simulation, PIL for image capture, and Hamming distance for UI element detection. The project includes a tkinter GUI for task management and a Base64 encoding system to bundle image resources for PyInstaller executables.

Tokens: 2.2K
Snippets: 6
Records: 8
Agent score: 6%

What's inside mhxy_fz

Implement image similarity detection using PIL and Hamming distance
master
The core automation logic relies on comparing real-time screenshots against baseline images.
1. Capture: Use ImageGrab.grab() to take a screenshot of a specific region relative to the game window.
2. Hash: Convert the captured image into a hash using a get_hash() function.
3. Compare: Use the hamming() function to calculate the Hamming distance between the current hash and a pre-saved baseline hash.
A lower Hamming distance indicates higher similarity. If the distance is below a certain threshold (e.g., 20 or 30), the script triggers a corresponding move_click() action.
```
# Example logic for detecting a button and clicking it
# 1. Load baseline
shiyong = Image.open("shiyong_jpg")
shiyong_hash = get_hash(shiyong)

# 2. In a loop, grab current screen area and compare
img_shiyong = ImageGrab.grab((topx + x1, topy + y1, topx + x2, topy + y2))
if hamming(get_hash(img_shiyong), shiyong_hash, 20):
    move_click(target_x, target_y)
    print("Clicked button")
```
https://github.com/haungwanjun/mhxy_fz/blob/master/README.md
Run the script with Administrator privileges
master
To ensure the script can perform mouse clicks and interact with the game window, you must run the script or the compiled executable with Administrator privileges. While screenshotting does not require elevated permissions, mouse control does.
https://github.com/haungwanjun/mhxy_fz/blob/master/README.md

Use the tkinter GUI to control automation tasks

master

The project provides a tkinter graphical interface to launch different automation threads. Each button in the UI is linked to a specific task (e.g., '师门' for Shimen tasks, '帮派' for Guild tasks) via a MyThread wrapper that accepts the task function and the window_size.

import tkinter as tk

# UI Setup Example
root = tk.Tk()
root.title("梦幻西游手游辅助")

# Button to start Shimen tasks in a separate thread
button_shimen = tk.Button(root, text=u"师门", command=lambda: MyThread(shimen, window_size), width=15, height=2)
button_shimen.pack()

# Button to stop automation
button_tingzhi = tk.Button(root, text=u"停止", command=lambda: MyThread(stop), width=15, height=2)
button_tingzhi.pack()

root.mainloop()

Convert images to Python files for PyInstaller bundling

master

When using PyInstaller to bundle your project, external resource files like images are not automatically included. To resolve this, you can convert your images into Base64-encoded strings and store them within a .py file. This allows the images to be imported as code modules, ensuring they are included in the executable.

Use the pic2py function to iterate through a list of image filenames and generate a new Python file where each image is assigned to a variable named after the file (with dots replaced by underscores).

import base64

def pic2py(picture_names, py_name):
    """
    Converts image files to a python file containing base64 strings.
    :param picture_names: List of image file paths
    :param py_name: The name of the output .py file (without extension)
    """
    write_data = []
    for picture_name in picture_names:
        filename = picture_name.replace('.', '_')
        open_pic = open("%s" % picture_name, 'rb')
        b64str = base64.b64encode(open_pic.read())
        open_pic.close()
        write_data.append('%s = "%s"\n' % (filename, b64str.decode()))

    f = open('%s.py' % py_name, 'w+')
    for data in write_data:
        f.write(data)
    f.close()

if __name__ == '__main__':
    pics = ["bangpai.jpg", "shiyong.jpg"] # Example list
    pic2py(pics, 'memory_pic')

Restore images from Base64 encoded Python modules
master
After converting images to a Python module (e.g., memory_pic.py), you must decode and write them back to disk as image files before they can be used by image processing libraries like PIL.
Import your generated module and use the get_pic function to decode the Base64 strings and save them as physical files.
```
import base64
import memory_pic  # The file generated by pic2py

# Function to restore image from base64 code
def get_pic(pic_code, pic_name):
    image = open(pic_name, 'wb')
    image.write(base64.b64decode(pic_code))
    image.close()

# Usage: pass the variable from the imported module
get_pic(memory_pic.bangpai_jpg, 'bangpai_jpg')
get_pic(memory_pic.shiyong_jpg, 'shiyong_jpg')
```
https://github.com/haungwanjun/mhxy_fz/blob/master/README.md
Troubleshoot image similarity detection issues
master
The project relies on calculating the similarity between a live screen capture and a stored reference image to detect game events.
If buttons or UI elements are not being detected (common on older monitors with different brightness/contrast levels):
1. Adjust parameters: Fine-tune the similarity threshold used in your comparison logic.
2. Re-capture reference images: Take new screenshots directly from the current machine to use as the new baseline for similarity calculation.
https://github.com/haungwanjun/mhxy_fz/blob/master/README.md

Simulate mouse clicks with pywin32

master

To automate interactions, use win32api.SetCursorPos to move the mouse and win32api.mouse_event with MOUSEEVENTF_LEFTDOWN and MOUSEEVENTF_LEFTUP to perform a click. You can optionally pass a delay t to wait after the click.

import win32api
import win32con
import time
import random

# Move mouse and click left button
def move_click(x, y, t=0):
    win32api.SetCursorPos((x, y))
    win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN | win32con.MOUSEEVENTF_LEFTUP, x, y, 0, 0)
    if t == 0:
        time.sleep(random.random()*2+1)
    else:
        time.sleep(t)
    return 0

Capture Windows information with pywin32

master

The project uses pywin32 to interact with Windows OS. You can use win32gui.FindWindow to locate the game window by its title and win32gui.GetWindowRect to retrieve its position and size as a rectangle. This is essential for calculating relative coordinates for clicks and screenshots.

import win32gui
import win32api

# Get screen resolution
def resolution():
    return win32api.GetSystemMetrics(0), win32api.GetSystemMetrics(1)

# Get the game window information (returns a rectangle)
def get_window_info():
    wdname = u'《梦幻西游》手游'
    handle = win32gui.FindWindow(0, wdname)
    if handle == 0:
        return None
    else:
        return win32gui.GetWindowRect(handle)