Steel Browser

repository·main·Indexed 27 days ago

https://github.com/steel-dev/steel-browser

An open-source browser API for AI agents and web automation tools. Steel provides high-level management of browser sessions, proxies, extensions, and anti-detection mechanisms, supporting integration with Puppeteer, Playwright, and Selenium. It includes features for session management, fingerprinting, and quick actions like scraping, screenshots, and PDF generation. Available via Node.js and Python SDKs, or as a Docker image.

Tokens
16.7K
Snippets
49
Records
100
Agent score
92%

What's inside steel-browser

  1. Overview of Steel Browser features

    main

    Steel is an open-source browser API designed for AI agents and web automation. It manages the complexity of browser infrastructure, providing:

    • Full Browser Control: Supports Puppeteer, Playwright, and Selenium via Puppeteer and CDP.
    • Session Management: Maintains state, cookies, and local storage across requests.
    • Proxy Support: Built-in proxy chain management for IP rotation.
    • Extension Support: Ability to load custom Chrome extensions.
    • Debugging Tools: Request logging and a UI for viewing/debugging sessions.
    • Anti-Detection: Includes stealth plugins and fingerprint management.
    • Resource Management: Automatic cleanup and lifecycle management.
    • Browser Tools: APIs to convert pages to markdown, readability mode, screenshots, or PDFs.
  2. Interact with Steel Browser via WebSocket and CDP

    main
    The Steel REPL package allows you to interact with a browser instance by connecting to a WebSocket endpoint exposed by the API. This connection uses the Chrome DevTools Protocol (CDP), enabling you to use Puppeteer for browser automation and interaction.
  3. Run Steel Browser locally with Docker Compose

    main

    You can run the API and UI separately using Docker Compose.

    For Mac Silicon users, you must pass the DOCKER_DEFAULT_PLATFORM environment variable to ensure the images run on the correct platform:

    docker compose up
    
    # For Mac Silicon:
    DOCKER_DEFAULT_PLATFORM=linux/arm64 docker compose up
  4. Monitor Steel Browser using structured logging

    main

    Steel Browser uses structured logging via Pino. When implementing custom logic or monitoring, you can use the fastify.log object to emit structured logs. This allows for easier querying and observability by including relevant metadata like sessionId, action, and url alongside the log message.

    fastify.log.info({ 
      sessionId, 
      action: 'page_created',
      url: page.url() 
    }, 'New page created');
  5. Enable debug logging

    main

    To get detailed execution information, enable verbose logging by setting the following environment variables:

    • NODE_ENV=development
    • ENABLE_VERBOSE_LOGGING=true
    • ENABLE_CDP_LOGGING=true
    • LOG_CUSTOM_EMIT_EVENTS=true
    # Enable all debug logging
    export NODE_ENV=development
    export ENABLE_VERBOSE_LOGGING=true
    export ENABLE_CDP_LOGGING=true
    export LOG_CUSTOM_EMIT_EVENTS=true
    
    # Start with debug logging
    npm run dev -w api
  6. Run Steel Browser locally with Node.js

    main

    If you have Node.js and Chrome installed, you can run the server and UI directly. This starts the Steel server on port 3000 and the UI on port 5173.

    1. Install dependencies:
    npm install
    1. Run the development script:
    npm run dev

    Chrome Requirements: Ensure Chrome is installed in one of the following standard paths:

    • Linux: /usr/bin/google-chrome
    • MacOS: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
    • Windows: C:\Program Files\Google\Chrome\Application\chrome.exe or C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

    Custom Chrome Path: If your Chrome executable is in a non-standard location, set the CHROME_EXECUTABLE_PATH environment variable:

    export CHROME_EXECUTABLE_PATH=/path/to/your/chrome
    npm run dev
  7. Install prerequisites for Steel Browser development

    main

    Before setting up the development environment, ensure your system meets these requirements:

    • OS: Linux, macOS, or Windows (WSL2 recommended).
    • RAM: 8GB minimum (16GB+ recommended).
    • Storage: 10GB+ free space.

    Required Software

    1. Node.js (Version 22+)

    • Linux/macOS: Use nvm (recommended) or package managers (apt for Ubuntu/Debian, brew for macOS).
    • Windows: Use choco or download from nodejs.org.

    2. Git

    • Install via apt/yum (Linux), brew (macOS), or choco (Windows).

    3. Chrome/Chromium Browser

    • Install google-chrome-stable or chromium-browser on Linux.
    • Install google-chrome via Homebrew Cask on macOS.
    • Install googlechrome via Chocolatey on Windows.
    • Install via apt/yum (Linux), Homebrew Cask (macOS), or Chocolatey (Windows).
  8. Verify environment requirements

    main

    Ensure your environment meets the following requirements:

    • Node.js: version 22 or higher
    • npm: installed and working
    • Chrome/Chromium: installed and accessible
    • Docker: installed (if using containers)
    # Check Node.js version (should be 22+)
    node --version
    
    # Check npm version
    npm --version
    
    # Check Chrome/Chromium
    google-chrome --version
    # or
    chromium --version
    
    # Check Docker (if using containers)
    docker --version
    docker-compose --version