Puppeteer Sharp Documentation

repository·master·Indexed 26 days ago

https://github.com/hardkoded/puppeteer-sharp

A .NET port of the official Node.js Puppeteer API for controlling Chrome, Chromium, and other browsers. It provides high-level APIs for web automation, scraping, and PDF generation. Available as a NetStandard 2.0 library (for .NET Framework 4.6.1 and .NET Core 2.0+) and a dedicated .NET 8 version. Key features include PDF and screenshot generation, JavaScript evaluation, HTML injection, and remote browser connection via WebSocket.

Tokens
12.6K
Snippets
51
Records
59
Agent score
87%

What's inside Puppeteer Sharp

  1. Quick Start with Puppeteer Sharp

    master

    You can launch a browser, open a new page, navigate to a URL, and take a screenshot using the following pattern. This example uses LaunchOptions to run in headless mode.

    using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true });
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://example.com");
    await page.ScreenshotAsync("screenshot.png");
  2. Take screenshots with Puppeteer Sharp

    master

    To capture a screenshot of a web page, launch a browser, navigate to a URL using GoToAsync, and call ScreenshotAsync. You can use SetViewportAsync to define the dimensions of the viewport before taking the screenshot.

    using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true });
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://www.google.com");
    await page.ScreenshotAsync("screenshot.png");
    
    // Optional: Set viewport size before screenshot
    await page.SetViewportAsync(new ViewPortOptions
    {
        Width = 500,
        Height = 500
    });
  3. Use Locators to interact with elements

    master

    Locators provide a high-level API to find and interact with elements on a page. Use page.Locator(selector) followed by action methods like ClickAsync() to interact with elements using CSS selectors.

    using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true });
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://example.com");
    await page.Locator("button.submit").ClickAsync();
  4. Download specific browser versions using BrowserFetcher

    master

    If you need to use a specific version of a browser (e.g., for testing Chrome Extensions), you can use the BrowserFetcher class to download specific versions from the 'Chrome for Testing' repository.

    1. Instantiate BrowserFetcher with the desired SupportedBrowser (e.g., SupportedBrowser.Chrome).
    2. Call DownloadAsync(version) with the specific version string.
    3. Use the returned object's GetExecutablePath() method to provide the ExecutablePath when launching the browser via Puppeteer.LaunchAsync.
    Console.WriteLine("Downloading browsers");
    
    var browserFetcher = new BrowserFetcher(SupportedBrowser.Chrome);
    var chrome118 = await browserFetcher.DownloadAsync("118.0.5993.70");
    var chrome119 = await browserFetcher.DownloadAsync("119.0.5997.0");
    
    Console.WriteLine("Navigating");
    await using (var browser = await Puppeteer.LaunchAsync(new()
    {
        ExecutablePath = chrome118.GetExecutablePath(),
    }))
    {
        await using var page = await browser.NewPageAsync();
        await page.GoToAsync("https://www.whatismybrowser.com/");
    
        Console.WriteLine("Generating PDF");
        await page.PdfAsync(Path.Combine(Directory.GetCurrentDirectory(), "118.pdf"));
    
        Console.WriteLine("Export completed");
    }
  5. Map JavaScript objects to .NET objects using EvaluateFunctionAsync<T>

    master

    When you need to retrieve complex data from the browser context and map it directly to a .NET type, use Page.EvaluateFunctionAsync<T>. This method evaluates a JavaScript function within the browser and automatically deserializes the returned JavaScript object into a .NET object of type T.

    public class Data
    {
        public string Title { get; set; }
        public string Url { get; set; }
    }
    
    using (var browser = await Puppeteer.LaunchAsync(options))
    using (var page = await browser.NewPageAsync())
    {
        await page.GoToAsync("https://news.ycombinator.com/");
        
        var jsCode = @"() => {
            const selectors = Array.from(document.querySelectorAll('a[class=""storylink""'])");
            return selectors.map( t=> {return { title: t.innerHTML, url: t.href}});
        }";
    
        // The generic type <Data[]> tells Puppeteer Sharp how to deserialize the JS array
        var results = await page.EvaluateFunctionAsync<Data[]>(jsCode);
    
        foreach (var result in results)
        {
            Console.WriteLine(result.ToString());
        }
    }
  6. Connect to a remote browser

    master

    Instead of launching a local browser instance, you can connect to an existing browser via a WebSocket endpoint using Puppeteer.ConnectAsync and ConnectOptions.BrowserWSEndpoint.

    var options = new ConnectOptions()
    {
        BrowserWSEndpoint = $"wss://www.externalbrowser.io?token={apikey}"
    };
    
    using var browser = await Puppeteer.ConnectAsync(options);
    using var page = await browser.NewPageAsync();
    await page.GoToAsync("https://www.google.com/");
    await page.PdfAsync("output.pdf");
  7. Test a Chrome Extension using Puppeteer Sharp

    master

    To test a Chrome extension, use Puppeteer.LaunchAsync with specific command-line arguments passed via LaunchOptions. You must set Headless = false because extensions generally do not work in headless mode.

    Use the following arguments in the Args array:

    • --disable-extensions-except="{pathToExtension}"
    • --load-extension="{pathToExtension}"
    using var browserFetcher = new BrowserFetcher();
    await browserFetcher.DownloadAsync(BrowserFetcher.DefaultRevision);
    
    var pathToExtension = "path/to/extension";
    var launchOptions = new LaunchOptions()
    {
        Headless = false,
        Args = new []
        {
            $@"--disable-extensions-except=""{pathToExtension}""",
            $@"--load-extension=""{pathToExtension}"""
        }
    };
    
    using (var browser = await Puppeteer.LaunchAsync(launchOptions))
    using (var page = await browser.NewPageAsync())
    {
        // test your extension here
    }
  8. Extract all links from a page using EvaluateExpressionAsync

    master

    To retrieve all hyperlinks (href attributes) from a web page, use Page.EvaluateExpressionAsync<T> to execute JavaScript within the browser context. This allows you to run DOM queries like document.querySelectorAll('a') and map the results directly to a C# array or list.

    using (var browser = await Puppeteer.LaunchAsync(options))
    using (var page = await browser.NewPageAsync())
    {
        await page.GoToAsync("http://www.google.com");
        var jsSelectAllAnchors = @"Array.from(document.querySelectorAll('a')).map(a => a.href);";
        var urls = await page.EvaluateExpressionAsync<string[]>(jsSelectAllAnchors);
        foreach (string url in urls)
        {
            Console.WriteLine($"Url: {url}");
        }
        Console.WriteLine("Press any key to continue...");
        Console.ReadLine();
    }
  9. Use Puppeteer Sharp in AOT compilation environments

    master

    Puppeteer Sharp is prepared for Ahead-of-Time (AOT) compilation. However, if you use custom classes when passing data to or receiving data from an Evaluate function, you must provide a JsonSerializerContext to handle serialization via source generation.

    To resolve this, follow these steps:

    1. Define a JsonSerializerContext using the [JsonSerializable] attribute for your custom classes.
    2. Assign the context to Puppeteer.ExtraJsonSerializerContext before launching the browser.

    Note: ExtraJsonSerializerContext is used the first time Puppeteer Sharp performs serialization or deserialization. It must be set before launching the browser and cannot be changed once set.

    // 1. Define your custom class
    public class TestClass
    {
        public string Name { get; set; }
    }
    
    // 2. Create a serialization context for AOT
    [JsonSerializable(typeof(TestClass))]
    public partial class DemoJsonSerializationContext : JsonSerializerContext
    {}
    
    // 3. Set the context BEFORE launching the browser
    Puppeteer.ExtraJsonSerializerContext = DemoJsonSerializationContext.Default;
    
    // 4. Proceed with browser launch and evaluation
    var options = new LaunchOptions { Headless = true };
    await using var browser = await Puppeteer.LaunchAsync(options);
    await using var page = await browser.NewPageAsync();
    await page.GoToAsync("https://www.google.com");
    
    // Now you can safely use custom classes in Evaluate functions
    var result = await page.EvaluateFunctionAsync<TestClass>("test => test", new TestClass { Name = "Dario" });
  10. Install and Setup Puppeteer Sharp

    master

    Puppeteer Sharp is a .NET port of the Node.js Puppeteer API that controls headless Chrome or Chromium via the DevTools Protocol.

    Prerequisites:

    • Target Frameworks: Available as a NetStandard 2.0 library (for .NET Framework 4.6.1 and .NET Core 2.0+) and a dedicated .NET 8 version.
    • Linux Users: An X-server is required. If you encounter issues running Chrome on Linux, refer to the official Puppeteer troubleshooting guide.
  11. Download and reuse Chrome from a custom location

    master

    To avoid downloading Chrome to the default location every time, you can use BrowserFetcherOptions to specify a custom directory for the browser download. This allows you to reuse a previously downloaded Chrome instance from a specific path.

    var downloadPath = "/Users/dario/chrome";
    var browserFetcherOptions = new BrowserFetcherOptions { Path = downloadPath };
    var browserFetcher = new BrowserFetcher(browserFetcherOptions);
    var installedBrowser = await browserFetcher.DownloadAsync();
  12. Generate PDF files

    master

    You can save a web page as a PDF using PdfAsync. For advanced layouts, use PdfOptions to specify Format (e.g., PaperFormat.A4), DisplayHeaderFooter, MarginOptions, and custom FooterTemplate or HeaderTemplate HTML strings.

    using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true });
    var page = await browser.NewPageAsync();
    await page.GoToAsync("https://www.google.com");
    await page.PdfAsync("output.pdf", new PdfOptions
    {
        Format = PaperFormat.A4,
        DisplayHeaderFooter = true,
        MarginOptions = new MarginOptions
        {
            Top = "20px",
            Right = "20px",
            Bottom = "40px",
            Left = "20px"
        },
        FooterTemplate = "<div id=\"footer-template\" style=\"font-size:10px !important; color:#808080; padding-left:10px\">Footer Text</div>"
    });