DOCSEARCHbyAlgolia
Submit Docs

Browserstation

repository·main·Indexed 41 days ago

https://github.com/reinforcenow/browserstation

An open-source, infra-agnostic hosting framework for browser agents and scrapers. It provides a scalable, Kubernetes-based alternative to managed services, allowing developers to run sandboxed Chrome instances via a unified API and CDP WebSocket proxy. It features a sidecar architecture managed by RayKube and integrates with frameworks like Browser-use, Langchain, and CrewAI. Supports deployment on AWS EKS and Google Kubernetes Engine (GKE) via Terraform.

Tokens
4.8K
Snippets
19
Records
30
Agent score
51%

What's inside Browserstation

  1. How Browserstation's Sidecar Architecture works

    main

    Browserstation uses a Sidecar Pattern managed by RayKube on a Kubernetes cluster. This architecture ensures isolation and efficient resource management for browser agents.

    The Sidecar Pattern

    Each worker pod contains two distinct containers:

    1. Ray Worker (Main Container): Hosts the BrowserActor. It manages the browser lifecycle, handles API requests, allocates resources, and communicates with Chrome via localhost:9222.
    2. Chrome (Sidecar Container): Runs a headless Chrome instance with remote debugging enabled. It exposes the Chrome DevTools Protocol (CDP) on port 9222.

    Each pod provides a single, isolated browser instance.

    Unified CDP WebSocket Proxy

    To allow clients to control browsers across the cluster, Browserstation uses a proxy on the head node.

    • Clients connect to /ws/browsers/{id}/devtools/browser on the head node.
    • The FastAPI-based proxy validates the browser ID.
    • The proxy then establishes a bidirectional WebSocket stream to the specific Chrome container in the corresponding worker pod.

    This allows automation tools to interact with the browser via CDP as if they were connected directly to the instance.

  2. Components deployed by the GKE Terraform configuration

    main

    The deployment manages the following Kubernetes resources:

    • Ray System Namespace: A dedicated namespace for all Ray-related components.
    • BrowserStation API Key Secret: (Optional) A Kubernetes secret containing the browserstation_api_key for authentication.
    • KubeRay Operator: The operator responsible for managing Ray clusters on the Kubernetes cluster.
    • RayService: The core service that deploys BrowserStation with auto-scaling workers.
    • LoadBalancer Service: A service that exposes the BrowserStation API to the public internet.

  3. Quick Start with Browserstation

    main

    Browserstation is an open-source hosting framework for browser agents. Follow these steps to set up a local environment for testing and development.

    1. Prerequisites

    Ensure you have the following installed:

    2. Deploy

    You must set an API key for authentication. Use the quickstart.sh script based on your hardware architecture.

    macOS (Apple Silicon):

    chmod +x scripts/quickstart.sh
./scripts/quickstart.sh --arm --api-key="your-secret-key"

    macOS (Intel), Windows (WSL), or Linux:

    chmod +x scripts/quickstart.sh
./scripts/quickstart.sh --x86 --api-key="your-secret-key"

    3. Test

    Set your API key in your environment and run the provided WebSocket test script:

    export BROWSERSTATION_API_KEY="your-secret-key"
uv run tests/test_websocket.py

    To monitor the cluster, use:

    kubectl get pods -n ray-system
curl http://localhost:8050

    4. Cleanup

    To remove the deployment, run:

    ./scripts/teardown.sh
    chmod +x scripts/quickstart.sh
./scripts/quickstart.sh --arm --api-key="your-secret-key"

  5. Configure Browserstation backend size

    main

    To customize the infrastructure and define the size of your backend based on your workload, modify the following files:

    1. Edit aws/01-infra/variables.tf to adjust infrastructure variables.
    2. Edit aws/02-k8s/templates/rayservice.yaml.tpl to adjust Kubernetes service templates.

    After making changes, re-run the deployment script to apply the new configuration:

    ./deploy.sh --api-key your-secret-key

  8. Disable GKE deletion protection

    main

    By default, the GKE cluster is created with deletion protection enabled. To destroy the cluster, you must first disable this protection in your terraform.tfvars and then target the specific resource for an update.

    1. Set deletion_protection = false in terraform.tfvars.
    2. Run the following command to apply the change specifically to the cluster:
      terraform apply -target=google_container_cluster.primary
    3. Once protection is disabled, you can proceed with terraform destroy.
    # terraform.tfvars
deletion_protection = false
    terraform apply -target=google_container_cluster.primary

  11. Deploy Browserstation on AWS EKS

    main

    Deploy a production-ready Browserstation environment on Amazon Elastic Kubernetes Service (EKS) using Terraform. The deployment provisions the EKS cluster and necessary networking infrastructure.

    Prerequisites

    Ensure the following tools are installed and configured:

    Deployment Steps

    1. Navigate to the AWS Terraform directory:
      cd terraform/aws
    2. Run the deployment script with your desired API key:
      ./deploy.sh --api-key your-secret-key

    Verifying Deployment

    Once Terraform completes (typically 10-15 minutes), look for the browserstation_endpoint output in your terminal. Access the service via: your-load-balancer-dns.elb.amazonaws.com:8050

    Note: If no --api-key was provided during deployment, authentication is disabled.

    cd terraform/aws
./deploy.sh --api-key your-secret-key

  12. Integrate Browserstation with Agent Frameworks

    main

    Browserstation is designed to be compatible with popular browser automation and agent frameworks. You can connect your agents to the Browserstation infrastructure using the following integrations:

    • Browser-use
    • Langchain
    • CrewAI

    Detailed implementation examples can be found in the repository's examples/ directory.