Camoufox Documentation

repository·main·Indexed 27 days ago

https://github.com/daijro/camoufox

An open-source anti-detect browser optimized for web scraping and AI agents to mitigate detection by anti-bot systems. Includes a Python package for browser management, a Qt-based UI Manager, and a build tester for validating Firefox binaries against anti-bot checks. Also features JSONvv, a schema validation library for configuration dictionaries.

Tokens
26.9K
Snippets
60
Records
173
Agent score
94%

What's inside Camoufox

  1. Overview of Camoufox

    main

    Camoufox is an open-source anti-detect browser specifically designed for web scraping and AI agents. It is built to help users bypass anti-bot mechanisms by providing a browser environment that is difficult to detect as automated.

    Note: This project is currently under active development and may not be suitable for stable production environments. For the most up-to-date documentation, visit camoufox.com.

  2. Understand Per-Context Storage Architecture

    main

    Camoufox uses a thread-safe C++ key-value store called RoverfoxStorageManager to manage fingerprinting values. Values are keyed by a unique userContextId derived from Firefox's container identity system.

    Storage Mechanism:

    • When window.setXxx() is called, the value is stored in a local nsTHashMap cache.
    • The value is also written to Firefox Preferences with a roverfox.s. prefix (all types are serialized as CString internally).
    • In content processes, values are synchronized to the parent process via synchronous IPC (SendRoverfoxStoragePut).

    Fallback Behavior: Certain patches provide a global fallback (using ucid=0) so that workers unable to resolve a specific userContextId can still access the values. Patches with this fallback include: audio, canvas, navigator (all 4 functions), timezone, and webgl. Patches without this fallback include: font-spacing, screen, font-list, speech-voices, and webrtc-ip.

  3. Understand the Self-Destruct Mechanism

    main

    To prevent fingerprinting scripts from detecting the presence of Camoufox's customization functions, every window.setXxx() function implements a dual self-destruct mechanism:

    1. JS_DeleteProperty: Immediately removes the function from the current window object after it is called.
    2. DisableFunction: Marks the function as disabled in storage so it does not reappear on future pages within the same context.

    This ensures that once a fingerprint is set, the API used to set it becomes invisible to the page.

  4. How Cross-Process Storage Works

    main

    To support Firefox's Fission (site isolation), Camoufox uses the cross-process-storage.patch to ensure fingerprinting values are available in worker processes even when the local in-process HashMap is empty.

    Key Components:

    1. IPDL Sync Messages: Uses sync RoverfoxStoragePut to write to the parent process and sync RoverfoxStorageGet to read from it. These are synchronous to ensure availability before workers start.
    2. Parent Process Handlers: ContentParent.cpp handles these messages, allowing only preference names starting with "roverfox.s." for security.
    3. Preference Whitelist: The prefix "roverfox.s." is whitelisted in Preferences.cpp to prevent Firefox from stripping these dynamically-created strings during cross-process sync.

    Thread Safety Note: Synchronous IPC calls are only permitted on the main thread. If a non-main thread (like HarfBuzz font rendering or the compositor) attempts to read a value, the system falls back to the local cache and Firefox Preferences only.

  5. Understand Playwright Patches in Camoufox

    main

    Camoufox utilizes specific patch files to manage the Firefox browser environment used by Playwright. These patches serve two primary purposes:

    1. Upstream Patches (0-playwright.patch): These contain the standard upstream patches from Microsoft Playwright. This file must be kept synchronized with the official Playwright bootstrap diff. If the upstream version diverges significantly, Camoufox will branch off.
    2. Leak Fixes (0-leak-fixes.patch): This patch is used to undo specific parts of the 0-playwright.patch to address leak issues or other compatibility requirements.
  6. Understand Camoufox core features and capabilities

    main

    Camoufox is a headless, undetectable Firefox fork optimized for web scraping and AI agents. It is designed to blend into normal traffic by providing a fresh identity for every run, drawn from real-world device distributions.

    Key Capabilities:

    • Undetectable Automation: Page automation is hidden from JavaScript inspection.
    • Fingerprint Injection & Rotation: Spoofs navigator properties (device, OS, hardware), screen/viewport properties, geolocation, timezone, locale, WebRTC IP (at protocol level), and audio/voice properties without using JS injection.
    • Anti-Graphical Fingerprinting: Spoofs WebGL parameters, extensions, context attributes, shader precision, and fonts.
    • Automation Optimizations: Includes human-like mouse movement, ad blocking/circumvention, and removal of CSS animations for speed.
    • Efficiency: Debloated for memory efficiency and fast launches.
    • Compatibility: Drop-in Playwright compatibility via the Python interface.
  7. Quickstart: Validate JSON with JsonValidator

    main

    To validate a configuration dictionary against a schema, use the JsonValidator class from jsonvv. You define your schema as a dictionary where keys are the expected JSON keys and values are strings representing the types and constraints. Use a try-except block to catch JvvRuntimeException if validation fails.

    from jsonvv import JsonValidator, JvvRuntimeException
    
    config = {
        "username": "johndoe",
        "age": 30
    }
    
    validator = {
        "username": "str",
        "age": "int[>=18]"
    }
    
    val = JsonValidator(validator)
    try:
       val.validate(config)
    except JvvRuntimeException as exc:
       print("Failed:", exc)
    else:
       print('Config is valid!')
  8. Update Juggler files from upstream

    main

    The additions/juggler/ directory contains Playwright's Juggler protocol implementation. To update these files, clone the Playwright repository, compare the directories, and copy the updated files.

    CRITICAL: You must preserve Firefox 146 ESM compatibility. Do not overwrite the following files with upstream versions if they break the ESM wrapper:

    • additions/juggler/components/Juggler.sys.mjs (The ESM wrapper)
    • additions/juggler/components/components.conf (The component registration file using the esModule field)
    # Clone Playwright repository
    git clone https://github.com/microsoft/playwright.git /tmp/playwright
    
    # Compare directories
    diff -r additions/juggler/ /tmp/playwright/browser_patches/firefox/juggler/
    
    # Copy updated files (example)
    cp -r /tmp/playwright/browser_patches/firefox/juggler/* additions/juggler/
  9. Quick Start Camoufox Service Tests

    main

    To run end-to-end antibot-detection tests that verify both the Firefox binary and the Python package using real proxies, follow these steps:

    1. Ensure you have Python 3.9+ and Node.js installed.
    2. Create a proxies.txt file in the directory (see Proxy Format).
    3. Execute the test script:
    ./run_tests.sh

    run_tests.sh automates the installation of npm dependencies, virtualenv creation, building the Python wheel from local source, and running two phases of testing: against a locally compiled binary and against an official fetched binary.

  10. Install the Camoufox Python package

    main

    Install the camoufox package using pip. It is highly recommended to include the [geoip] extra if you are using proxies, as this allows the library to determine geolocation (longitude, latitude, timezone, country, and locale) to avoid proxy protection.

    pip install -U camoufox[geoip]
  11. Fix macOS Sandbox crash in Camoufox

    main

    This patch addresses a segmentation fault on macOS that occurs during content process launch. The crash is triggered when GetRepoDir() or GetObjDir() fail in non-packaged builds (such as cross-compiled builds from Ubuntu to macOS) because the required Info.plist keys are missing.

    The fix replaces the fatal MOZ_CRASH with an NS_WARNING, allowing the browser to continue running without the optional sandbox whitelisting paths (testingReadPath3 and testingReadPath4).

  12. Use Per-Context Fingerprint Patches with Playwright

    main

    Camoufox provides per-context isolation via specific JavaScript functions that allow each Playwright context to have a unique, deterministic fingerprint. This prevents cross-context correlation when running multiple concurrent sessions.

    To apply these fingerprints, use context.addInitScript(). This ensures the functions are called on every new page, tab, or navigation before any website scripts execute.

    Important Notes:

    • Self-Destruction: All fingerprinting functions self-destruct after the first call. They cannot be detected by website code via typeof window.setXxx after execution.
    • Guards: Always use typeof checks (e.g., if (typeof window.setTimezone === 'function')) within your init script to prevent ReferenceError if the patches are not present or if the function has already self-destructed.
    const { firefox } = require('playwright');
    
    const browser = await firefox.launch({
      executablePath: '/path/to/camoufox',
    });
    
    const context = await browser.newContext({
      viewport: { width: 1280, height: 720 },
    });
    
    // Apply fingerprints via addInitScript — fires on every new page automatically
    await context.addInitScript((values) => {
      const w = window;
    
      if (typeof w.setFontSpacingSeed === 'function') {
        w.setFontSpacingSeed(values.fontSpacingSeed);
      }
      if (typeof w.setAudioFingerprintSeed === 'function') {
        w.setAudioFingerprintSeed(values.audioFingerprintSeed);
      }
      if (typeof w.setTimezone === 'function') {
        w.setTimezone(values.timezone);
      }
      if (typeof w.setScreenDimensions === 'function') {
        w.setScreenDimensions(values.screenWidth, values.screenHeight);
      }
      if (typeof w.setScreenColorDepth === 'function') {
        w.setScreenColorDepth(values.screenColorDepth);
      }
      if (typeof w.setWebRTCIPv4 === 'function') {
        w.setWebRTCIPv4(values.webrtcIPv4);
      }
      if (typeof w.setNavigatorPlatform === 'function') {
        w.setNavigatorPlatform(values.navigatorPlatform);
      }
      if (typeof w.setNavigatorOscpu === 'function') {
        w.setNavigatorOscpu(values.navigatorOscpu);
      }
      if (typeof w.setNavigatorHardwareConcurrency === 'function') {
        w.setNavigatorHardwareConcurrency(values.hardwareConcurrency);
      }
      if (typeof w.setNavigatorUserAgent === 'function') {
        w.setNavigatorUserAgent(values.userAgent);
      }
      if (typeof w.setWebGLVendor === 'function') {
        w.setWebGLVendor(values.webglVendor);
      }
      if (typeof w.setWebGLRenderer === 'function') {
        w.setWebGLRenderer(values.webglRenderer);
      }
      if (typeof w.setCanvasSeed === 'function') {
        w.setCanvasSeed(values.canvasSeed);
      }
      if (values.fontList && values.fontList.length > 0 && typeof w.setFontList === 'function') {
        w.setFontList(values.fontList.join(','));
      }
      if (values.speechVoices && typeof w.setSpeechVoices === 'function') {
        w.setSpeechVoices(values.speechVoices);
      }
    }, {
      fontSpacingSeed: 12345678,
      audioFingerprintSeed: 87654321,
      timezone: 'America/New_York',
      screenWidth: 1920,
      screenHeight: 1080,
      screenColorDepth: 24,
      webrtcIPv4: '203.0.113.1',  // your proxy IP
      navigatorPlatform: 'MacIntel',
      navigatorOscpu: 'Intel Mac OS X 10.15',
      hardwareConcurrency: 8,
      userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:146.0) Gecko/20100101 Firefox/146.0',
      webglVendor: 'Intel Inc.',
      webglRenderer: 'Intel Iris OpenGL Engine',
      canvasSeed: 55555555,
      fontList: ['Arial', 'Helvetica', 'Georgia', 'Courier New', 'Verdana', 'Times New Roman'],
      speechVoices: 'Microsoft David,Microsoft Zira,Google US English',
    });
    
    const page = await context.newPage();
    await page.goto('https://example.com');