repository·main·Indexed 27 days ago
https://github.com/autoscrape-labs/pydollA stealth-first Python library for automating Chromium-based browsers (Chrome and Edge) without a WebDriver. Pydoll communicates directly with the Chrome DevTools Protocol (CDP) via WebSockets to simulate realistic human interactions, such as Bezier curve mouse movements and humanized typing, to bypass bot protections like Cloudflare Turnstile. It features structured data extraction with Pydantic, Shadow DOM support, HAR network recording, and hybrid UI/API automation.
Pydoll provides three distinct ways to locate elements depending on the information you have available:
find(): Use when you know specific HTML attributes (e.g., id, class_name, name).query(): Use when you have a specific CSS selector or XPath expression.WebElement (e.g., using get_children_elements()).The browser commands module provides low-level control over browser instances and their configuration. It is primarily used internally by Chrome and Edge classes, but can be used directly for advanced scenarios.
Available capabilities include:
Pydoll provides high-level interfaces for interacting with Chrome DevTools Protocol (CDP) domains through specialized command modules. Each module corresponds to a specific CDP domain and provides methods for various browser operations:
browser_commands.py): Browser-level operations and window management.dom_commands.py): DOM tree manipulation and element operations.input_commands.py): Input event simulation (keyboard, mouse, touch).network_commands.py): Network monitoring and request interception.page_commands.py): Page lifecycle management and navigation.runtime_commands.py): JavaScript execution and runtime management.storage_commands.py): Browser storage access (cookies, local storage, etc.).target_commands.py): Target management and tab operations.fetch_commands.py): Network request interception and modification.pydoll.browser.managers module provides specialized classes designed to handle specific aspects of the browser lifecycle and configuration. While these managers are primarily used internally by core browser classes like Chrome and Edge, they can be used directly for advanced automation scenarios or when extending the library.Proxy detection is a probabilistic process where websites use multiple signals to determine if a connection is proxied. Understanding these signals helps in choosing the right proxy type and evasion strategy for Pydoll automation.
Intl.DateTimeFormat(), Accept-Language header, or physical impossibility of travel speed) trigger alarms.Via, X-Forwarded-For, or X-Real-IP reveals proxies. Even the absence of standard headers or incorrect header ordering can be a signal.User-Agent and the actual TCP stack characteristics (TTL, window size).User-Agent, it is flagged.A Tab instance in Pydoll represents a single browser tab or window and serves as the primary interface for page automation. Each tab operates with its own independent context, including:
target_id.pydoll.commands.storage_commands module provides comprehensive management of browser storage mechanisms, including cookies, localStorage, sessionStorage, IndexedDB, and Cache Storage. You can use these commands to persist, retrieve, or clear data across different storage types.Pydoll is built on several foundational architectural principles:
Pydoll is organized into several functional modules:
Chrome, Edge), configuration (Options), and tabs (Tab).WebElement).DOM, Network, Page, Runtime).Pydoll utilizes a dual-connection model to balance browser-wide control with tab-specific automation:
ws://localhost:port/devtools/browser/... used by the Browser instance for global operations like managing contexts, downloads, and browser-wide events.Tab instance creates its own dedicated ConnectionHandler with a unique WebSocket connection to ws://localhost:port/devtools/page/{targetId}.Benefits of this approach:
Pydoll provides several core benefits for automation developers: