Alumnium Documentation

Browser Support

Alumnium builds an accessibility tree of the webpage. Because there is no standard browser API for this, it currently only supports Chromium-based browsers (e.g., Google Chrome, Microsoft Edge, Opera).

• Playwright driver: Supports both headful and headless modes.
• Selenium driver: Supports headful mode only.

Mobile Support

Alumnium supports mobile automation via Appium:

• iOS: XCUITest driver.
• Android: UiAutomator2 driver.

Alumnium currently supports the following languages for writing and executing tests:

• Python
• TypeScript
• Java

Alumnium is an experimental test automation abstraction layer designed to simplify mobile and web application testing using AI. Instead of replacing your entire testing stack, it wraps existing automation engines like Appium, Playwright, or Selenium.

How it works:

1. Accessibility Tree Extraction: Alumnium operates on the application's accessibility tree.
2. Compaction & AI Communication: It compacts the tree and sends it to an AI model along with your instructions (actions or assertions).
3. Execution: Once the AI determines the necessary course of action, Alumnium instructs the underlying browser or mobile driver (Playwright, Selenium, or Appium) to execute the command.

This design allows Alumnium to co-exist with your existing end-to-end tests and CI pipelines; you only need to provide access to an AI model.

Alumnium automatically attempts to cast retrieved data into appropriate types. The currently supported types are:

• integer
• float
• boolean
• string
• lists of the above types

Note: In Java and TypeScript, you may need to perform manual typecasting if you intend to store the returned value in a typed variable. If the requested data is not found on the page, Alumnium returns a string explaining why it could not retrieve the data.

Alumnium uses two layers of caching that work together without configuration:

1. Response Cache: Stores full LLM responses keyed by the exact request.
2. Elements Cache: Stores AI decisions alongside the UI elements they reference. It resolves elements to their current IDs in the accessibility tree during lookup. This allows the cache to remain valid even if element IDs change due to page re-renders, as long as the elements are still present. It also uses fuzzy matching on instruction text.

Alumnium is initialized by passing an existing Appium driver instance to the Alumni constructor.

Important Lifecycle Note: Calling al.quit() (or alumni.quit()) shuts down both the AI session and the wrapped Appium driver. When using Alumni, you do not need to call driver.quit() separately.

// Java example
Alumni al = new Alumni(driver);
// ...
al.quit(); // Shuts down both AI and driver

# Python example
@fixture
def al(driver: WebDriver):
    al = Alumni(driver)
    yield al
    al.quit()

// TypeScript example
import { Alumni } from "alumnium";

let al: Alumni;
before(async () => {
  al = new Alumni(browser);
});

after(async () => {
  await al.quit();
});

To reduce flakiness, Alumnium automatically waits for the following conditions before attempting an action:

1. The HTML document is loaded and ready.
2. Resources on the page are loaded.
3. Document mutations are finished.
4. XHR/fetch requests are finished.

Additionally, Alumnium automatically retries actions when errors occur during execution to handle common issues like content changes or LLM inconsistencies.

The Java PlaywrightDriver lacks several features present in the Python implementation:

1. Page Tracking: It does not maintain a list of tracked pages or handle popup/close events automatically. switchToNextTab() and switchToPreviousTab() rely on the underlying Playwright context directly.
2. New Tab Timeout: There is no configurable ALUMNIUM_PLAYWRIGHT_NEW_TAB_TIMEOUT (which defaults to 200ms in Python).

Alumnium allows you to write test cases in plain language using a simple command set. Instead of writing complex browser interaction logic, you describe the intent, and Alumnium's AI translates these instructions into executable commands using the application's accessibility tree and screenshots.

Key commands include:

• do: Describe the steps to perform.
• check: Verify the results.
• get: Extract data from the page.

This approach keeps the test logic in your control while the AI handles the underlying browser interactions.

al.do("create task 'buy milk'")
al.check("task 'buy milk' is pending")
assert al.get("number of pending tasks") == 1

To reduce flakiness, Alumnium automatically waits for the following conditions to be met before attempting any check() verification:

1. The HTML document is loaded and ready.
2. Resources on the page are loaded.
3. Document mutations are finished.
4. XHR/fetch requests are finished.