BrowserGym Documentation

To create a custom task in BrowserGym, inherit from the AbstractBrowserTask class. You must implement several key methods to define the task's lifecycle:

1. get_task_id() (classmethod): Returns a unique string identifier for the task.
2. setup(page): Initializes the environment (e.g., navigating to a URL) and returns a goal string and an info dictionary.
3. validate(page, chat_messages): Determines if the task was successful. Returns a tuple containing reward (float), success (bool), a message (str), and info (dict).
4. cheat(page, chat_messages) (optional): Provides an 'oracle' or hard-coded solution to solve the task in a single step.
5. teardown(): Cleans up resources before the environment closes.

from typing import Tuple
import playwright.sync_api
from browsergym.core.task import AbstractBrowserTask

class SampleTask(AbstractBrowserTask):
    def __init__(self, seed: int) -> None:
        super().__init__(seed)

    @classmethod
    def get_task_id(cls):
        return "sample_task"

    def setup(self, page: playwright.sync_api.Page) -> Tuple[str, dict]:
        page.goto("https://www.google.com", timeout=10000)
        goal = "Search for 'Eiffel Tower' Wikipedia page."
        info = {}
        return goal, info

    def validate(
        self, page: playwright.sync_api.Page, chat_messages: list[str]
    ) -> Tuple[float, bool, str, dict]:
        if page.url == "https://en.wikipedia.org/wiki/Eiffel_Tower":
            return 1.0, True, "Task completed", {}
        else:
            return 0.0, False, "", {}

    def cheat(self, page: playwright.sync_api.Page, chat_messages: list[str]) -> None:
        page.get_by_text("Search").fill("Eiffel Tower")
        page.get_by_text("Google Search").click()
        page.get_by_text("Eiffel Tower - Wikipedia").click()

    def teardown(self) -> None:
        pass

To use WebArena with BrowserGym, you must set several environment variables. These variables use the WA_ prefix and must point to the specific URLs/ports used in your WebArena instance.

Important:

• Ensure the ports match your specific WebArena setup.
• If your instance supports the FULL_RESET feature, set WA_FULL_RESET to the appropriate endpoint.
• If it does not support it, either do not set WA_FULL_RESET or set it to an empty string ("").

BASE_URL=<YOUR_SERVER_URL_HERE>  # example: "http://myazuremachine.eastus.cloudapp.azure.com"

# webarena environment variables (change ports as needed)
export WA_SHOPPING="$BASE_URL:8082/"
export WA_SHOPPING_ADMIN="$BASE_URL:8083/admin"
export WA_REDDIT="$BASE_URL:8080"
export WA_GITLAB="$BASE_URL:9001"
export WA_WIKIPEDIA="$BASE_URL:8081/wikipedia_en_all_maxi_2022-05/A/User:The_other_Kiwix_guy/Landing"
export WA_MAP="$BASE_URL:443"
export WA_HOMEPAGE="$BASE_URL:80"

# if your webarena instance offers the FULL_RESET feature (optional)
export WA_FULL_RESET="$BASE_URL:7565"

# otherwise, be sure to NOT set WA_FULL_RESET, or set it to an empty string
export WA_FULL_RESET=""

VisualWebArena requires several environment variables to map the various web domains to your running server instance. All variables must use the VWA_ prefix.

Ensure the ports used in the environment variables match the ports configured in your WebArena instance.

Note on VWA_FULL_RESET:

• If your WebArena instance supports the FULL_RESET feature, set VWA_FULL_RESET to the appropriate URL (e.g., http://<BASE_URL>:7565).
• If it does not support this feature, do not set the variable, or set it to an empty string (export VWA_FULL_RESET="").

BASE_URL=<YOUR_SERVER_URL_HERE>  # example: "http://myazuremachine.eastus.cloudapp.azure.com"

# visualwebarena environment variables (change ports as needed)
export VWA_CLASSIFIEDS="$BASE_URL:8083"
export VWA_CLASSIFIEDS_RESET_TOKEN="4b61655535e7ed388f0d40a93600254c"
export VWA_SHOPPING="$BASE_URL:8082"
export VWA_REDDIT="$BASE_URL:8080"
export VWA_WIKIPEDIA="$BASE_URL:8081"
export VWA_HOMEPAGE="$BASE_URL:80"

# if your webarena instance offers the FULL_RESET feature (optional)
export VWA_FULL_RESET="$BASE_URL:7565"

# otherwise, be sure to NOT set VWA_FULL_RESET, or set it to an empty string
export VWA_FULL_RESET=""

You can install BrowserGym using pip. Depending on your needs, you can install the full package or specific subsets:

• pip install browsergym: Recommended. Installs everything.
• pip install browsergym-experiments: Installs experiment utilities (agent, loop, benchmarks) and everything else.
• pip install browsergym-core: Installs only core functionalities (no benchmarks, just open-ended tasks).
• pip install browsergym-miniwob: Core + MiniWoB.
• pip install browsergym-webarena: Core + WebArena.
• pip install browsergym-webarena-verified: Core + WebArenaVerified.
• pip install browsergym-visualwebarena: Core + VisualWebArena.
• pip install browsergym-workarena: Core + WorkArena.
• pip install browsergym-assistantbench: Core + AssistantBench.
• pip install weblinx-browsergym: Core + WebLINX.
• pip install browsergym-timewarp: Core + TimeWarp.

After installing, you must install the Playwright Chromium browser:

playwright install chromium