rod

Rod is a high-level web automation and scraping driver for Go, built directly on the Chrome DevTools Protocol. It provides both high-level helpers for common tasks and low-level access for deep customization.

Key features include:

• Chained context design: Intuitive way to handle timeouts or cancellations for long-running tasks.
• Auto-waiting: Automatically waits for elements to be ready before interaction.
• Robustness: Handles nested iframes and Shadow DOMs correctly, and uses leakless to ensure no zombie browser processes remain after a crash.
• Debugging: Supports auto input tracing and remote monitoring of headless browsers.
• Concurrency: All operations are thread-safe.
• Browser Management: Automatically finds or downloads the required browser via the launcher package.

To learn how to use Rod, you can explore several resources:

1. examples_test.go: The primary starting point for quick code snippets.
2. lib/examples folder: A collection of more structured examples.
3. Unit Tests: For specific method usage (e.g., HandleAuth), search the repository's *_test.go files to see how the methods are implemented and called in real scenarios.
4. Comparison: If you are coming from chromedp, check lib/examples/compare-chromedp to see the differences in implementation style.

You can intercept network requests at either the Browser level (intercepting all requests from all pages) or the Page level (intercepting requests only for a specific page) using HijackRequests().

When hijacking is enabled, page caching is disabled, though 304 Not Modified responses will still function.

To use the router, you must call .Add() to define patterns, and then call .Run() to start the interception process. After calling .Run(), you should not add new handlers to the router.

The rod package provides a suite of methods prefixed with Must (e.g., MustClick, MustNavigate). These are wrappers around the standard API methods that automatically trigger a panic if an error is returned. This is useful for simplifying code when you expect operations to succeed and want to avoid repetitive error handling.

By default, these methods panic. However, you can customize this behavior using WithPanic to define a custom failure function (e.g., for logging or controlled exits).

Use ExposeHelpers to inject Go functions into the browser's JavaScript environment. This allows you to call Go code directly from the DevTools console or from within page.Evaluate calls. The functions are attached to a global window.rod object in the browser.

Pass one or more *js.Function pointers to the method.

Extract visual representations of the page:

• Screenshot(fullPage bool, req *proto.PageCaptureScreenshot): Captures a screenshot. If fullPage is true, it adjusts the viewport to capture the entire scrollable area.
• ScrollScreenshot(opt *ScrollScreenshotOptions): Performs a scroll-and-stitch screenshot. This is useful for long pages but may repeat fixed-position elements (like headers). Use FixedTop and FixedBottom to optimize.
• PDF(req *proto.PagePrintToPDF): Prints the page as a PDF. Returns a *StreamReader.
• CaptureDOMSnapshot(): Returns a flattened array of the full DOM tree, including iframes and shadow DOM, along with layout and style information.

The Overlay method draws a message directly over a specific *Element. This is useful for verifying which element is being targeted by an action.

It returns a removeOverlay function that clears the overlay from that specific element.

Customize how the page makes network requests:

• SetExtraHeaders(dict []string): Sets extra HTTP headers to be sent with every request. The input dict should be a slice of strings in [key, value, key, value, ...] format. Returns a func() to clear these headers and an error.
• SetUserAgent(req *proto.NetworkSetUserAgentOverride): Sets the User Agent. If req is nil, a default laptop Chrome User Agent is used.
• SetBlockedURLs(urls []string): Blocks specific requests using patterns (wildcards * allowed). An empty pattern "" blocks all requests.