DOCSEARCHbyAlgolia
Submit Docs

PyAutoGUI

repository·master·Indexed 11 days ago

https://github.com/asweigart/pyautogui

A cross-platform Python module for GUI automation that allows programmatic control of the mouse and keyboard. Features include mouse movement, clicking, keyboard typing, hotkey simulation, screenshot capture, and image recognition using Pillow. Includes built-in fail-safes like `pyautogui.FAILSAFE` and `pyautogui.PAUSE` to prevent scripts from running out of control.

Tokens
9K
Snippets
33
Records
38
Agent score
94%

What's inside PyAutoGUI

  1. Install PyAutoGUI on Windows

    master

    On Windows, it is recommended to use the py.exe launcher to ensure you are installing to the correct Python version. You can specify a specific version using a command line argument.

    # Install to the latest version
py -m pip install pyautogui

# Install to a specific version (e.g., 3.8)
py -3.8 -m pip install pyautogui

  2. Install PyAutoGUI on macOS

    master

    On macOS, use python3 to install the package. If you encounter issues installing pyobjc on macOS El Capitan, set the MACOSX_DEPLOYMENT_TARGET environment variable to 10.11 during the installation of pyobjc.

    python3 -m pip install pyautogui

# Troubleshooting El Capitan pyobjc issues
MACOSX_DEPLOYMENT_TARGET=10.11 pip install pyobjc

  4. Install PyAutoGUI on Linux

    master

    On Linux, you must install scrot, Tkinter, and python3-dev in addition to the PyAutoGUI package to ensure full functionality (such as screenshotting and message boxes).

    python3 -m pip install pyautogui
sudo apt-get install scrot
sudo apt-get install python3-tk
sudo apt-get install python3-dev

  5. Install dependencies for screenshot functionality

    master

    PyAutoGUI uses the PyScreeze module for screenshots. To use these features, you must ensure the following dependencies are met:

    1. Pillow: Required for all screenshot operations.
    2. OS-specific tools:
      • macOS: Uses the built-in screencapture command.
      • Linux: Requires scrot. Install it via:
        sudo apt-get install scrot
    sudo apt-get install scrot

  6. Install PyAutoGUI

    master

    Install PyAutoGUI using pip:

    pip install pyautogui

    Platform-Specific Dependencies

    Depending on your operating system, you may need to install additional modules:

    • Windows: No additional dependencies required.
    • macOS: Requires pyobjc-core and pyobjc (install in that order).
    • Linux: Requires python3-xlib (or python-xlib for Python 2).

    Note on Image Support: PyAutoGUI uses Pillow for image-related features. On Linux, you may need to install additional system libraries to ensure Pillow's PNG/JPEG support works correctly.

  7. Run PyAutoGUI unit tests

    master

    If you need to verify the core functionality of your installation, PyAutoGUI includes a test suite located in basicTests.py. Note that these tests are not exhaustive and primarily cover basic mouse and keyboard operations.

    The current test coverage includes:

    • onScreen()
    • size()
    • position()
    • moveTo()
    • moveRel()
    • typewrite()
    • PAUSE

  8. Configure PyAutoGUI fail-safes and pauses

    master

    To prevent automation scripts from running out of control, you can configure two main fail-safe mechanisms:

    1. pyautogui.PAUSE: Sets a delay (in seconds) after every PyAutoGUI function call.
    2. pyautogui.FAILSAFE: When set to True, moving the mouse manually to the upper-left corner of the screen will raise a pyautogui.FailSafeException, allowing you to abort the program.

    Note: All keyword arguments in PyAutoGUI functions are optional.

    import pyautogui

# Set a 2.5 second pause after each call
pyautogui.PAUSE = 2.5

# Enable fail-safe mode
pyautogui.FAILSAFE = True

  9. Handle Chinese input via Copy-Paste

    master

    Because direct character input for Chinese may have issues in some environments, the recommended workaround is to use pyperclip to copy the text to the clipboard and then use pyautogui.hotkey('ctrl', 'v') to paste it.

    import pyperclip
import pyautogui

def paste(foo):
    pyperclip.copy(foo)
    pyautogui.hotkey('ctrl', 'v')

foo = u'学而时习之'
# Move to text box and paste
pyautogui.click(130, 30)
paste(foo)

  10. Install PyAutoGUI on different platforms

    master

    PyAutoGUI supports Python 2.x and 3.x. Installation steps vary by operating system:

    • Windows: No dependencies required.
      pip3 install pyautogui
    • OS X: Requires pyobjc-core and pyobjc.
      sudo pip3 install pyobjc-core
sudo pip3 install pyobjc
sudo pip3 install pyautogui
    • Linux: Requires python3-xlib, scrot, python-tk, and python3-dev.
      sudo pip3 install python3-xlib
sudo apt-get scrot
sudo apt-get install python-tk
sudo apt-get install python3-dev
sudo pip3 install pyautogui
    pip3 install pyautogui

  11. Perform mouse clicks and double-clicks

    master

    Simulate mouse button presses using the following functions:

    • pyautogui.click(x=None, y=None, button='left', clicks=1, interval=0.0): Performs a click. If x and y are provided, it moves to that location before clicking. Use clicks for multiple clicks and interval for the pause between them.
    • pyautogui.doubleClick(x=None, y=None, button='left', interval=0.0): A shortcut for a double-click.
    • pyautogui.tripleClick(x=None, y=None, button='left', interval=0.0): A shortcut for a triple-click.
    • pyautogui.rightClick(x=None, y=None): A shortcut for a right-button click.
    >>> pyautogui.click()                      # click at current position
>>> pyautogui.click(x=100, y=200)           # move to 100, 200, then click left button
>>> pyautogui.click(button='right')        # right-click the mouse
>>> pyautogui.click(clicks=2)               # double-click the left mouse button
>>> pyautogui.click(clicks=2, interval=0.25) # double-click with 0.25s pause
>>> pyautogui.click(button='right', clicks=3, interval=0.25) # triple-click right button
>>> pyautogui.doubleClick()                 # perform a left-button double click
>>> pyautogui.rightClick(x=100, y=200)       # right-click at 100, 200

  12. Use Screenshot and Image Recognition functions

    master

    PyAutoGUI uses Pillow/PIL for image data. On Linux, you must install scrot via sudo apt-get install scrot to use these features.

    Screenshots

    • pyautogui.screenshot(): Returns a Pillow/PIL Image object of the current screen.
    • pyautogui.screenshot('filename.png'): Captures the screen and saves it to the specified file.

    Image Recognition (Locating on Screen)

    These functions return None if the image is not found. Note that these operations can be slow (1-2 seconds).

    • pyautogui.locateOnScreen('image.png'): Returns the (left, top, width, height) of the first instance of the image found.
    • pyautogui.locateAllOnScreen('image.png'): Returns a generator yielding the (left, top, width, height) for all instances found.
    • pyautogui.locateCenterOnScreen('image.png'): Returns the (x, y) coordinates of the center of the first instance found.
    import pyautogui

# Take a screenshot
pyautogui.screenshot('my_screenshot.png')

# Find an image on screen
location = pyautogui.locateOnScreen('button.png')
if location:
    print(f'Found at: {location}')

# Find center of an image to click it
center = pyautogui.locateCenterOnScreen('button.png')
if center:
    pyautogui.click(center)

# Find all instances
for loc in pyautogui.locateAllOnScreen('icon.png'):
    print(loc)