SeleniumBase

repository·master·Indexed 11 days ago

https://github.com/seleniumbase/seleniumbase

An all-in-one browser automation framework for Python designed for web crawling, testing, scraping, and stealthy automation. It features CDP Mode and UC Mode to bypass bot-detection, integration with Behave BDD for Gherkin-style testing, and a specialized Stealthy Playwright Mode that allows playwright-python to attach to stealthy browser sessions via Chrome DevTools Protocol.

Tokens
66.1K
Snippets
209
Records
248
Agent score
96%

What's inside SeleniumBase

  1. Explore SeleniumBase Documentation and Resources

    master

    The SeleniumBase help documentation is organized into several key areas to assist with different stages of automation development:

    Core Setup and Installation

    • Installation Guides: Instructions for installing Python, Pip, Git, SeleniumBase, and Webdrivers.
    • Environment Management: Tutorials on using Python Virtual Environments.
    • Verification: Steps to verify that your Webdriver installation is working correctly.

    Testing Features and Modes

    • Undetected Mode (UC Mode): Specialized mode for bypassing bot detection.
    • CDP Mode & Stealth API: Access to Chrome DevTools Protocol for advanced automation and stealth.
    • Mobile Testing: Support for mobile device emulation and testing.
    • Recorder Mode: Tools to record browser actions to generate test scripts.
    • Visual Testing: Capabilities for image-based verification.

    Advanced Automation and Tooling

    • Selenium Grid: Instructions for distributed testing using Selenium Grid.
    • Command Line & GUI: Tutorials for using Console Scripts and the pytest Commander (GUI).
    • Behave BDD: Support for Behavior-Driven Development using Behave.
    • Handling iframes & Shadow DOM: Specialized guidance for interacting with complex web elements.

    Utilities and Reporting

    • The Dashboard: Visual reporting and log analysis.
    • Chart Maker & Presentation Maker: Tools for generating charts and presentations from test data.
    • Language Support: Tools for translations and managing locale codes.

    Learning and Examples

    • Usage Examples: A collection of practical code snippets and patterns.
    • Demo Pages: Various web pages (e.g., Coffee Cart, TinyMCE, MFA Login) designed specifically to test different automation scenarios like drag-and-drop, file uploads, and iframes.
  2. Understand SeleniumBase resource loading and bonus features

    master

    SeleniumBase utilizes various JavaScript libraries to power bonus features such as the Website Tour Maker, Presentation Maker, Chart Maker, Demo Mode, and the HTML Inspector.

    Most of these resources are retrieved via CDN links to keep the core package size small. However, some specific assets like favicon.ico are hosted within the repository.

    Many of the specialized feature assets (like those for website tours or notifications) have been moved to a dedicated repository: github.com/seleniumbase/resource-files.

  3. Create interactive product tours with SeleniumBase

    master

    SeleniumBase allows you to create interactive walkthroughs on any website by wrapping five popular JavaScript libraries: IntroJS, Bootstrap Tour, DriverJS, Shepherd, and Hopscotch.

    To use these features, you typically extend BaseCase and use the tour-related methods provided by the framework. You can create a tour, add steps that tether to specific CSS selectors, and play the tour automatically or manually.

    from seleniumbase import BaseCase
    
    class MyTourClass(BaseCase):
        def test_tour(self):
            self.goto("https://example.com")
            self.create_tour(theme="shepherd")
            self.add_tour_step("Welcome!", title="Hello")
            self.add_tour_step("Click here.", ".my-button")
            self.play_tour()
    
    BaseCase.main(__name__, __file__)
  4. Understand the SeleniumBase framework structure

    master

    SeleniumBase is organized into several functional directories that define its capabilities. Understanding these helps in locating specific features or extending the framework:

    • fixtures: Contains base_case.py, where the core SeleniumBase test methods are defined.
    • core: Contains browser_launcher.py, responsible for spinning up browsers for tests.
    • plugins: Contains pytest_plugin.py, which adds command-line options to pytest.
    • console_scripts: Contains run.py, the entry point for calling SeleniumBase console scripts.
    • drivers: The directory where web drivers are automatically downloaded.
    • config: Contains settings.py, providing default configuration options for tests.
    • js_code: Contains JavaScript code used by various SeleniumBase components.
    • undetected: Contains logic for preventing websites from detecting Selenium automation.
    • extensions: Contains Chromium extensions available for use in tests.
    • common: Contains Python decorators available for use in tests.
    • utilities: Contains code for setting up a custom Selenium Grid.
    • resources: Contains JavaScript resources used by tests.
    • translate: Contains code for translating tests into different languages.
    • behave: Contains integration code for the Behave BDD test runner.
    • masterqa: Integration with MasterQA for combining automation with manual verification.
  5. Generate HTML charts with SeleniumBase ChartMaker

    master

    SeleniumBase ChartMaker allows you to generate interactive HTML charts (using HighCharts) directly from Python code. You can create various chart types including pie, bar, column, line, and area charts, and organize them into presentations with multiple slides.

    To use ChartMaker, inherit from BaseCase and use the provided chart creation and data management methods within your test methods.

    from seleniumbase import BaseCase
    BaseCase.main(__name__, __file__)
    
    class MyChartMakerClass(BaseCase):
        def test_chart_maker(self):
            self.create_presentation()
            self.create_pie_chart(title="Automated Tests")
            self.add_data_point("Passed", 7, color="#95d96f")
            self.add_data_point("Untested", 2, color="#eaeaea")
            self.add_data_point("Failed", 1, color="#f1888f")
            self.add_slide("<p>Pie Chart</p>" + self.extract_chart())
            self.begin_presentation(filename="my_chart.html")
  6. What is Stealthy Playwright Mode?

    master

    Stealthy Playwright Mode is a specialized configuration of SeleniumBase CDP Mode. It allows playwright-python to attach to a stealthy browser session managed by SeleniumBase via a remote-debugging URL.

    This mode uses Playwright's connect_over_cdp() method under the hood, enabling you to bypass bot-detection while leveraging the APIs of both SeleniumBase and Playwright in tandem.

  7. What is UC Mode (Undetected-Chromedriver Mode)?

    master

    UC Mode is a feature in SeleniumBase designed to make bots appear human, allowing them to evade detection from anti-bot services and bypass CAPTCHAs. It is based on undetected-chromedriver but includes specialized uc_*() methods for bypassing specific challenges like Cloudflare Turnstile and Google reCAPTCHA.

    Key mechanisms of UC Mode:

    1. Modifies Chromedriver: Renames Chrome DevTools Console variables to prevent detection.
    2. Stealthy Launch: Launches Chrome browsers before attaching chromedriver to them.
    3. Disconnection: Disconnects chromedriver from Chrome during sensitive actions (like page loads or clicks) to prevent detection.

    Note: For the successor to plain UC Mode, see CDP Mode.

  8. What is CDP Mode and how does it differ from UC Mode?

    master

    CDP Mode Overview

    CDP Mode is a stealth mode that uses the Chrome DevTools Protocol (via MyCDP) to control the web browser. It is designed to bypass advanced anti-bot detection by interacting with the browser directly via CDP rather than through the standard WebDriver interface.

    UC Mode vs. CDP Mode

    • UC Mode (Undetectable Crawler Mode): Uses a modified chromedriver (uc_driver) that avoids detection by disconnecting and reconnecting the WebDriver from the browser at strategic times.
    • CDP Mode: Provides a higher level of stealth by using CDP directly. It can be used as a subset of UC Mode or as Pure CDP Mode (where WebDriver is not available at all).

    Key Advantages of CDP Mode

    • Stealthier: Using CDP directly is harder for anti-bot systems to detect than WebDriver.
    • Hybrid Usage: You can use WebDriver and CDP calls together.
    • Full Access: Provides full access to advanced CDP library methods.
    • Playwright Integration: Can be used to make the Playwright library stealthy.
  9. What is MasterQA and how does it work?

    master

    MasterQA is a hybrid testing approach that combines automated browser actions with manual verification steps.

    How it works:

    1. Automation Checkpoints: You write standard SeleniumBase automation code (e.g., self.goto(), self.highlight()).
    2. Manual Verification: You insert self.verify() calls at specific points. After each self.verify() call, a pop-up window appears during the test run, asking the user to manually verify the state of the application.
    3. Results Reporting: Once all manual questions are answered, a results page is displayed. This page summarizes the test run, including successes and failures.

    Handling Failures:

    • Automated Failures: If an automated step fails (e.g., an element is not found), the Incomplete Test Runs row on the results page will be non-zero.
    • Manual Failures: If a user answers a verification question negatively, it is recorded as a failure.
    • Reporting: A results_table.csv file is generated containing a spreadsheet of details for both manual and automated failures. Failed verifications also provide links to screenshots and log files.
    from seleniumbase import MasterQA
    
    class MasterQATests(MasterQA):
        def test_masterqa(self):
            self.goto("https://xkcd.com/1700/")
            self.verify("Do you see a webcomic?")
            self.verify("Can you find the moon?") # Custom question
  10. Automated Visual Regression Testing with check_window()

    master

    SeleniumBase provides automated visual regression testing via the self.check_window() method. Instead of pixel-by-pixel comparison, it detects layout changes by comparing HTML tags and attributes against a stored baseline. This is useful for detecting unintended layout shifts, redesigns, or structural changes.

    How it works

    1. Baseline Creation: The first time self.check_window(name="unique_name") is called with a new name, SeleniumBase creates a baseline folder containing:
      • page_url.txt: The URL of the window.
      • baseline.png: The baseline screenshot.
      • tags_level1.txt: HTML tags.
      • tags_level2.txt: HTML tags + attribute names.
      • tags_level3.txt: HTML tags + attribute names + values.
    2. Comparison: Subsequent calls with the same name compare the current window's HTML structure to the baseline files. A latest.png screenshot is also saved to help you inspect the current state.
    3. Failures: The test fails if the layout doesn't match the baseline, unless using level=0. It also fails with Page Domain Mismatch Failure if the current URL domain differs from the baseline URL.
    self.check_window(name="first_test", level=3)
  11. Create multi-step highlights using `<mk-n>` tags

    master

    You can simulate progressive highlighting within a single slide by using special tags: <mk-0>, <mk-1>, <mk-2>, etc. SeleniumBase Presenter will automatically convert these into multiple generated slides where only the specified element is highlighted using the HTML <mark> tag at each step.

    Example:

    self.add_slide(
        code=(
            '<p><mk-0>Highlight this on the 1st generated slide</mk-0></p>'
            '<p><mk-1>Highlight this on the 2nd generated slide</mk-1></p>'
        )
    )
    self.add_slide(
        code=(
            '<p><mk-0>Highlight this on the 1st generated slide</mk-0></p>'
            '<p><mk-1>Highlight this on the 2nd generated slide</mk-1></p>'
            '<p><mk-2>Highlight this on the 3rd generated slide</mk-2></p>'
            '<p><mk-3>Highlight this on the 4th generated slide</mk-3></p>'
        )
    )
  12. Implement the Page Object Pattern with `objectify`

    master

    SeleniumBase provides a workflow to automatically refactor tests into the Page Object Pattern:

    1. extract-objects [SB_FILE.py]: Scans a test file for selectors and saves them as objects in a page_objects.py file in the same directory.
    2. inject-objects [SB_FILE.py]: Replaces matching selectors in the test file with the objects defined in page_objects.py.
    3. objectify [SB_FILE.py]: A shortcut that performs both extraction and injection in one step.
    4. revert-objects [SB_FILE.py]: Reverts changes made by objectify or inject-objects by replacing object names back with their original selectors.

    Options for all commands:

    • -c / --comments: Add/keep object selectors in the code comments.
    sbase objectify my_test.py