DOCSEARCHbyAlgolia
Submit Docs

TagUI Documentation

repository·master·Indexed 27 days ago

https://github.com/aisingapore/tagui

An open-source Robotic Process Automation (RPA) tool for automating web, desktop, and command-line tasks using natural-language scripts. It features visual automation via images and OCR, data extraction to CSV, and integrations with Python, C# .NET, and Node-RED. Supports advanced workflows including subflows, object repositories, and Live Mode for development.

Tokens
27.1K
Snippets
92
Records
171
Agent score
91%

What's inside TagUI

  1. Overview of TagUI RPA capabilities

    master

    TagUI is a free, open-source, cross-platform Robotic Process Automation (RPA) tool designed to automate desktop, web, mouse, and keyboard actions. It supports multiple ways to identify and interact with UI elements:

    1. Web Identifiers: Using URLs and text-based commands (e.g., click Log in).
    2. Image Recognition: Using images of UI elements to perform actions (e.g., click login_button.png).
    3. Coordinate-based Interaction: Using (x, y) screen coordinates (e.g., click (720,500)).
    // Example: Web identifier
https://login.xero.com/identity/user/login
type email as user@gmail.com
type password as 12345678
click Log in

// Example: Image recognition
type email_box.png as user@gmail.com
type password_box.png as 12345678
click login_button.png

// Example: Coordinate-based
type (720,400) as user@gmail.com
type (720,440) as 12345678
click (720,500)

  2. Overview of TagUI RPA capabilities

    master
    TagUI is a CLI tool for digital process automation (RPA) maintained by AI Singapore. It allows you to automate Chrome (visibly or headlessly), perform visual automation of websites and desktops, and write automation scripts in over 20 human languages or JavaScript. It features native integration with R and Python for big data and AI/ML tasks, and includes a Chrome extension for recording web actions.

  3. Explore TagUI Flavours and Integrations

    master

    TagUI is available across various environments and languages:

    Human Languages

    • English: Main repository
    • Other Languages: Available in the src/languages directory
    • Office: MS Word & Excel integration
    • IDE: VS Code extension
    • Low-Code: Node-RED integration
    • Cloud/Container: Google Colab and Docker support

    Programming Languages

    • Python: RPA-Python library
    • C# .NET: Available via NuGet
    • Go: Community work-in-progress

  4. Explore TagUI third-party helper tools

    master

    Several third-party tools are available to assist in writing and managing TagUI flows:

    • TagUI Writer: A Windows application that provides a popup menu of TagUI steps when pressing Ctrl + Left-click, allowing for easy pasting into text editors.
    • TagUI Screenshoter: An app designed to capture screenshots specifically for TagUI visual automation.
    • TagUI Editor: An editor that allows you to edit and run TagUI scripts via AutoHotKey.

    These tools are maintained by Arnaud Degardin and can be downloaded from the tagui_scripts repository.

  5. Use Repositories for Reusable Objects

    master

    Repositories allow you to define reusable objects or steps to improve readability and maintainability.

    1. Save a file with the same name as your flow filename but with a .csv extension.
    2. The CSV must have at least two columns (e.g., OBJECT and DEFINITION).
    3. In your flow, use the keyword object to replace a placeholder with its definition.

    Example Repository (myflow.csv):

    OBJECTDEFINITION
    emailuser-email-textbox
    create accountbtn btn--green btn-xl signup-btn

    Usage in Flow: type email as user@gmail.com (This replaces email with user-email-textbox and types the address).

  6. Perform test assertions with the check step

    master

    When running TagUI with the test option, the check step performs automatic assertions. This allows you to generate XUnit XML files for CI/CD integration (e.g., Jenkins).

    Common check patterns:

    • Presence: check present('selector') | 'success message' | 'failure message'
    • Text equality: check variable equals to 'expected text' | 'success' | 'failure'
    • Content containment: check text() contains 'substring' | 'success' | 'failure'
    • Count comparisons: check count('selector') more than or equals to 5 | 'success' | 'failure'

    Note: If using the chrome or headless options, CasperJS built-in assertions may not work, but TagUI's check step and smart selectors (tx('selector')) remain available.

    // check whether the element search-buttons exists
check present('search-buttons') | 'search button exists' | 'search button does not exist'

// check whether the search button text is correct
read search-buttons to button
check button equals to 'Search' | 'search button text is correct - ' button | 'search button text is wrong - ' button

  7. Trigger automation flows via API URL

    master

    TagUI can be configured as a web service to trigger automation flows via HTTP requests. This is useful for integrating automation into web applications or triggering flows from emails.

    Syntax: your_website_url/tagui_service.php?SETTINGS="flow_filename option(s)"

    Setup via Crontab: To process the queue of incoming requests, add a crontab entry on your server. This prevents concurrent runs by checking the queue at a set frequency and running pending flows only if no automation is currently in progress.

    Example crontab entry to check every 15 minutes:

    0,15,30,45 * * * * /full_path_on_your_server/tagui_crontab

  8. Install TagUI via Packaged Installation

    master

    TagUI can be installed by downloading and unzipping the pre-packaged version for your platform. For visual automation (desktop or browser), ensure you have Java JDK v8 (64-bit) or later installed.

    Recommended installation locations:

    • Windows: c:\
    • macOS: desktop
    • Linux: /home/your_id

    Important: Avoid using spaces in the folder path, as some components of TagUI may not function correctly with spaces in folder or file names.

  9. Write TagUI automation flows

    master

    TagUI allows you to write automation flows using natural-language-like syntax, JavaScript, or visual identifiers. It converts these intentions into working JavaScript code using Chrome DevTools Protocol, SikuliX, CasperJS, PhantomJS, or SlimerJS.

    Automation Methods

    • Natural Language: Use human-readable commands like click, type, and download.
    • Visual Automation: Use images of elements (e.g., button.png) instead of web identifiers.
    • Coordinate-based: Use (x,y) coordinates for UI elements.
    • JavaScript: Use JavaScript directly in the flow for advanced scripting.
    // Sample flow: login and download
https://www.typeform.com
click login
type username as user@gmail.com
type password as 12345678
click btnlogin
download https://admin.typeform.com/xxx to report.csv

// Using images for visual automation
click login_button.png
type username_box.png as user@gmail.com

// Using coordinates
click (1200,200)
type (800,400) as user@gmail.com

  10. Perform Mouse and Keyboard actions

    master

    TagUI provides several steps for interacting with the UI via mouse and keyboard. Most mouse actions support DOM, XPath, Point, or Image identifiers.

    • click [identifier]: Left clicks on the target.
    • rclick [identifier]: Right clicks on the target.
    • dclick [identifier]: Double left clicks on the target.
    • hover [identifier]: Moves the mouse cursor to the target.
    • type [identifier] as [text]: Types text into a web input. Use [clear] to clear the field before typing and [enter] to hit the Enter key.
    • keyboard [keys]: Enters keystrokes directly. Supports special keys like [shift], [ctrl], [alt], [enter], [space], [tab], [esc], [backspace], [up], [down], etc.
    • mouse down / mouse up: Sends explicit mouse events at the current position.
    click //nav/div/div[2]/ul/li[4]/ul/li[1]/a
click (500,200)
click button.png
type search-term as [clear]John Wick[enter]
keyboard [ctrl]c

  11. Handle exceptions and errors in TagUI

    master

    TagUI provides three levels of error handling:

    1. Local error handling: Use if conditions within the flow to check for specific scenarios (e.g., checking if a UI element exists) and execute alternative steps.
    2. Workflow error handling: Chain flows using shell operators to execute specific flows on success or failure:
      • || (OR): Runs the second flow only if the first one fails.
      • && (AND): Runs the second flow only if the first one succeeds.
    3. Global error handling: Configure TagUI to perform special handling (like syncing reports to a central API or storage) after every run across all flows.
    # Windows (Command Prompt)
call tagui flow.tag || tagui error.tag
call tagui flow.tag && tagui success.tag

# macOS / Linux (Terminal)
tagui flow.tag || tagui error.tag
tagui flow.tag && tagui success.tag

  12. Save and load files

    master

    Manage local files using these steps:

    • dump [text] to [filename]: Saves text to a new file.
    • write [text] to [filename]: Appends a new line to an existing file.
    • load [filename] to [variable]: Loads file content into a variable.
    • snap [DOM/XPath/Region/Image/page] to [filename]: Takes a screenshot of an element, region, or the whole page.
    dump First Name,Last Name to names.csv
write `fullreport` to report.txt
snap page to webpage.png