Computer Use Preview

The main.py script is the primary interface for executing browser tasks. Use the --query flag to provide natural language instructions.

Basic usage:

python main.py --query "Go to Google and type 'Hello World' into the search bar"

Available Environments (--env):

• playwright: Runs the browser locally using Playwright (default behavior).
• browserbase: Connects to a Browserbase instance (requires BROWSERBASE_API_KEY and BROWSERBASE_PROJECT_ID).

Example with Playwright and initial URL:

python main.py --query="Go to Google and type 'Hello World' into the search bar" --env="playwright" --initial_url="https://www.google.com/search?q=latest+AI+news"

Follow these steps to set up the repository and its dependencies:

1. Clone the repository:

   git clone https://github.com/google-gemini/computer-use-preview.git
   cd computer-use-preview

2. Set up a Python virtual environment and install requirements:

   python3 -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt

3. Install Playwright and Chrome dependencies:

   playwright install-deps chrome
   playwright install chrome

To use Vertex AI, you must enable it via environment variables and provide your project details:

Current session:

export USE_VERTEXAI=true
export VERTEXAI_PROJECT="YOUR_PROJECT_ID"
export VERTEXAI_LOCATION="YOUR_LOCATION"

Persist in virtual environment:

echo 'export USE_VERTEXAI=true' >> .venv/bin/activate
echo 'export VERTEXAI_PROJECT="your-project-id"' >> .venv/bin/activate
echo 'export VERTEXAI_LOCATION="your-location"' >> .venv/bin/activate
deactivate
source .venv/bin/activate

To use the Gemini Developer API, set the GEMINI_API_KEY environment variable. You can do this in your current session or persist it in your virtual environment.

Current session:

export GEMINI_API_KEY="YOUR_GEMINI_API_KEY"

Persist in virtual environment:

echo 'export GEMINI_API_KEY="YOUR_GEMINI_API_KEY"' >> .venv/bin/activate
deactivate
source .venv/bin/activate

For local development, it is recommended to use a .env file. The project automatically loads this file on startup.

1. Copy the example file:

   cp .env.example .env

2. Open .env and replace the placeholder values with your actual API keys.

On some operating systems, Playwright cannot capture <select> elements because they are rendered by the OS, preventing the agent from seeing them in screenshots.

Mitigation Strategies:

1. Use Browserbase: This is the recommended solution as it avoids local OS rendering issues.
2. Inject a script: Use a library like proxy-select to render custom <select> elements. You must inject proxy-select.css and proxy-select.js into the page.

Example injection logic for Playwright.__enter__:

self._page.add_init_script(PROXY_SELECT_JS)
def inject_style(page):
    try:
        page.add_style_tag(content=PROXY_SELECT_CSS)
    except Exception as e:
        print(f"Error injecting style: {e}")

self._page.on('domcontentloaded', inject_style)

Specify the model using the --model <model name> flag.

Available Models:

• gemini-3.6-flash (Default)
• gemini-3.5-flash-lite (Low-latency, cost-efficient)
• gemini-3.5-flash (Earlier Flash model)
• gemini-2.5-computer-use-preview-10-2025 (Earlier preview model)
• gemini-3-flash-preview (Preview version of Gemini 3 Flash)

The main.py script provides a command-line interface to execute a browser agent that performs tasks based on a natural language query. You can choose between different computer environments (playwright or browserbase), specify the target model, and set an initial URL.

CLI Arguments