PuppeteerExtraSharp

repository·master·Indexed 42 days ago

https://github.com/overmiind/puppeteer-sharp-extra

A .NET port of the puppeteer-extra library providing an extensible framework for browser automation in C#. It includes plugins for stealth (evading bot detection), CAPTCHA solving via providers like CapSolver and TwoCaptcha, and resource blocking to optimize crawls and reduce bandwidth.

Tokens
6.7K
Snippets
20
Records
23
Agent score
51%

What's inside PuppeteerExtraSharp

  1. Quickstart with the Stealth Plugin

    master

    To use the Stealth plugin to evade bot detection, initialize a PuppeteerExtra instance, create a StealthPlugin instance, and register it using the .Use() method before launching the browser. This allows you to navigate to sites like https://bot.sannysoft.com/ with enhanced stealth capabilities.

    var extra = new PuppeteerExtra();
    // initialize stealth plugin
    var stealth = new StealthPlugin();
    
    var browser = await extra.Use(stealth).LaunchAsync(new LaunchOptions());
    
    var page = await browser.NewPageAsync();
    
    await page.GoToAsync("https://bot.sannysoft.com/");
  2. Install and use the CaptchaSolver Plugin

    master

    The CaptchaSolverPlugin is included in PuppeteerExtraSharp. To use it, instantiate a provider (like CapSolver or TwoCaptcha), create the plugin, and add it to your PuppeteerExtra instance using .Use().

    Supported Vendors:

    • Google reCAPTCHA: v2 Checkbox, v2 Invisible, v3, Enterprise
    • GeeTest: v3, v4
    • Cloudflare Turnstile: Managed, Non-interactive, Invisible
    • DataDome: Slider, Interstitial, Device Check
    • hCaptcha: Detection only (solving is disabled)

    Supported Providers:

    • CapSolver
    • TwoCaptcha
    // Create plugin with CapSolver provider
    var captchaSolver = new CaptchaSolverPlugin(
        new CapSolver("YOUR_CAPSOLVER_API_KEY")
    );
    
    // Add plugin to PuppeteerExtra
    var extra = new PuppeteerExtra();
    extra.Use(captchaSolver);
    
    // Launch browser
    var browser = await extra.LaunchAsync(new LaunchOptions
    {
        Headless = false
    });
    
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://example.com/captcha-page");
    
    // Solve CAPTCHAs on the page
    var result = await captchaSolver.SolveCaptchaAsync(page);
  3. Configure Stealth Plugin options

    master

    The StealthPlugin can be customized by passing specific option objects to its constructor. This allows you to spoof hardware concurrency, vendor information, languages, and WebGL settings to better mimic a real user environment.

    var extra = new PuppeteerExtra();
    // Initialize hardware concurrency plugin options
    var stealthHardwareConcurrencyOptions = new StealthHardwareConcurrencyOptions(12);
    // Put it on stealth plugin constructor
    var stealth = new StealthPlugin(stealthHardwareConcurrencyOptions);
    var browser = await extra.Use(stealth).LaunchAsync(new LaunchOptions());
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://bot.sannysoft.com/");
  4. Quick Start with PuppeteerExtraSharp and Plugins

    master

    To use PuppeteerExtraSharp with plugins, initialize a PuppeteerExtra instance, register your desired plugins using the .Use() method, and then call .LaunchAsync() to start the browser with those plugins applied.

    // Initialize plugin builder
    var extra = new PuppeteerExtra();
    
    // Enable the Stealth plugin
    extra.Use(new StealthPlugin());
    
    // Launch the browser with plugins applied
    var browser = await extra.LaunchAsync();
    
    // Create a new page
    var page = await browser.NewPageAsync();
    
    await page.GoToAsync("https://google.com");
    await Task.Delay(2000);
    
    // Take a screenshot
    await page.ScreenshotAsync("extra.png");
  5. Use the reCAPTCHA Plugin to solve challenges

    master

    The reCAPTCHA plugin allows you to solve reCAPTCHA v2 (checkbox), v2 Invisible (button/callback), and v3 (score-based) challenges programmatically. It automates the process of detecting the widget, requesting a token from a provider API, and injecting that token into the page in a single call.

    Supported providers include TwoCaptcha and CapSolver.

    // Initialize the 2Captcha provider with your API key
    var twoCaptchaProvider = new TwoCaptcha("<YOUR_2CAPTCHA_API_KEY>");
    var recaptchaPlugin = new RecaptchaPlugin(twoCaptchaProvider);
    
    var puppeteerExtra = new PuppeteerExtra();
    
    // Launch browser with the reCAPTCHA plugin enabled
    var browser = await puppeteerExtra.Use(recaptchaPlugin).LaunchAsync();
    
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://www.google.com/recaptcha/api2/demo");
    
    // Single call to detect the widget, request a solution via the API, and inject the token
    await recaptchaPlugin.SolveCaptchaAsync(page);
    
    // Submit the form on the page
    var submitButton = await page.QuerySelectorAsync("#recaptcha-demo-submit");
    await submitButton.ClickAsync();
  6. Install PuppeteerExtraSharp via NuGet

    master

    PuppeteerExtraSharp is a .NET port of the Node.js puppeteer-extra library. You can install it using NuGet to gain access to extended browser automation features and plugins.

    # Use the NuGet package manager to install PuppeteerExtraSharp
    dotnet add package PuppeteerExtraSharp
  7. Use the Block Resources Plugin

    master

    The BlockResourcesPlugin allows you to block unwanted network requests (scripts, images, documents, etc.) using composable rules. This is useful for speeding up crawls, reducing bandwidth, and blocking ads or trackers.

    To use it, instantiate BlockResourcesPlugin, register it with your PuppeteerExtra instance using .Use(), and define rules using the AddRule method. For maximum effectiveness, add your rules before performing any navigation (e.g., before page.GoToAsync).

    // Create the plugin and register it
    var blockResourcesPlugin = new BlockResourcesPlugin();
    var puppeteerExtra = new PuppeteerExtra();
    
    var browser = await puppeteerExtra.Use(blockResourcesPlugin).LaunchAsync();
    var page = await browser.NewPageAsync();
    
    // Add rules BEFORE navigation
    blockResourcesPlugin.AddRule(s => 
        s.Page(page)
         .Url("ads.js")
         .Resources(ResourceType.Script));
    
    await page.GoToAsync("https://example.com");
  8. Solve DataDome CAPTCHAs with Proxy configuration

    master

    DataDome requires that the browser and the solving service use the same IP address. To achieve this, you must use a proxy with a sticky IP (via a ProxySessionId) and pass the proxy credentials to the SolveCaptchaAsync method.

    Steps:

    1. Generate a unique ProxySessionId (e.g., a GUID).
    2. Configure the browser to use the proxy via LaunchOptions and authenticate using the session ID in the username (e.g., user;sessid.{sessionId}).
    3. Pass the same ProxySessionId and proxy credentials to SolveCaptchaAsync via CaptchaSolverOptions.
    var provider = new CapSolver("YOUR_API_KEY");
    var plugin = new CaptchaSolverPlugin(provider);
    
    // Generate unique session ID for sticky IP
    var sessionId = Guid.NewGuid().ToString("N");
    
    // Configure browser to use proxy
    var launchOptions = new LaunchOptions
    {
        Headless = false,
        Args = new[] { "--proxy-server=http://proxy.example.com:8080" }
    };
    
    var extra = new PuppeteerExtra().Use(plugin);
    var browser = await extra.LaunchAsync(launchOptions);
    var page = await browser.NewPageAsync();
    
    // Authenticate with proxy using session ID
    // Format: "user;sessid.{sessionId}"
    await page.AuthenticateAsync(new Credentials
    {
        Username = $"myuser;sessid.{sessionId}",
        Password = "mypassword"
    });
    
    await page.GoToAsync("https://site-with-datadome.com");
    
    // Solve with matching proxy configuration
    var result = await plugin.SolveCaptchaAsync(page, new CaptchaSolverOptions
    {
        EnabledVendors = new Dictionary<CaptchaVendor, ICaptchaSolveOptions?> 
        { 
            { CaptchaVendor.DataDome, null } 
        },
        ProxyAddress = "proxy.example.com",
        ProxyPort = 8080,
        ProxyLogin = "myuser",
        ProxyPassword = "mypassword",
        ProxySessionId = sessionId  // Crucial for sticky IP
    });
  9. Configure Provider-specific options

    master

    Use CaptchaProviderOptions to tune the behavior of the underlying solving service (e.g., CapSolver or 2Captcha).

    • Timeout: Total timeout for the solving operation (default: 5 minutes).
    • StartTimeout: Initial delay before checking for a solution (default: 5 seconds).
    • PollingInterval: Frequency of checking for the solution (default: 2 seconds).
    var providerOptions = new CaptchaProviderOptions
    {
        Timeout = TimeSpan.FromMinutes(3),
        StartTimeout = TimeSpan.FromSeconds(3),
        PollingInterval = TimeSpan.FromSeconds(2)
    };
    
    var provider = new CapSolver("YOUR_API_KEY", providerOptions);
    var captchaSolver = new CaptchaSolverPlugin(provider);
  10. Configure CaptchaSolverOptions

    master

    Use CaptchaSolverOptions to control how the plugin detects, filters, and solves CAPTCHAs. You can set these globally when creating the plugin or override them for a specific request.

    Key Options:

    • CaptchaWaitTimeout: Max time to wait for CAPTCHA to appear (default: 5s).
    • ThrowOnError: If true, throws exceptions instead of returning error messages.
    • MinScore: Minimum acceptable score for score-based CAPTCHAs (0.0 - 1.0, default: 0.5).
    • SolveInViewportOnly: If true, only solves CAPTCHAs visible in the viewport.
    • SolveScoreBased: Whether to solve score-based CAPTCHAs like reCAPTCHA v3.
    • SolveInvisibleChallenges: Whether to solve invisible CAPTCHA challenges.
    • EnabledVendors: A dictionary to enable/disable specific vendors and provide vendor-specific options (e.g., GoogleOptions).
    var options = new CaptchaSolverOptions
    {
        CaptchaWaitTimeout = TimeSpan.FromSeconds(10),
        ThrowOnError = false,
        MinScore = 0.7,
        SolveInViewportOnly = false,
        SolveScoreBased = true,
        SolveInvisibleChallenges = true,
        Debug = false,
        EnabledVendors = new Dictionary<CaptchaVendor, ICaptchaSolveOptions?>
        {
            { CaptchaVendor.Google, new GoogleOptions() },
            { CaptchaVendor.GeeTest, null },
            { CaptchaVendor.Cloudflare, null },
            { CaptchaVendor.HCaptcha, null },
            { CaptchaVendor.DataDome, null }
        }
    };
    
    var captchaSolver = new CaptchaSolverPlugin(
        new CapSolver("YOUR_API_KEY"),
        options
    );
  11. Configure RecaptchaSolveOptions

    master

    Use RecaptchaSolveOptions to fine-tune how the RecaptchaPlugin detects and interacts with challenges on the page. Key options include:

    • ThrowOnError: If true, throws exceptions on failure; otherwise returns a soft failure.
    • SolveInViewportOnly: If true, only attempts to solve widgets visible in the viewport.
    • SolveScoreBased: Enables handling of reCAPTCHA v3.
    • SolveInactiveChallenges: Attempts to solve lazy or inactive widgets.
    • CaptchaWaitTimeout: Time to wait for a widget/challenge to appear.
    • Debug: Enables verbose debug logging.
    • MinV3RecaptchaScore: The minimum acceptable score for reCAPTCHA v3.
    • SolveInvisibleChallenges: Handles invisible or triggered reCAPTCHA.
    var pluginOptions = new RecaptchaSolveOptions
    {
        ThrowOnError = true,
        SolveInViewportOnly = true,
        SolveScoreBased = true,
        SolveInactiveChallenges = true,
        CaptchaWaitTimeout = TimeSpan.FromSeconds(10),
        Debug = false,
        MinV3RecaptchaScore = 0.3,
        SolveInvisibleChallenges = true,
    };
    
    var recaptchaPlugin = new RecaptchaPlugin(twoCaptchaProvider, pluginOptions);