pup

pup reads HTML from stdin and prints the processed result to stdout. You can use CSS selectors to filter parts of the page and optional display functions to change the output format.

Basic Syntax:

cat index.html | pup [flags] '[selectors] [display function]'

You can install pup using Go, Homebrew (on macOS), or by downloading binaries from the releases page.

Using Go:

go get github.com/ericchiang/pup

Using Homebrew (macOS):

brew install https://raw.githubusercontent.com/EricChiang/pup/master/pup.rb

pup supports a wide range of CSS selectors to filter HTML elements.

Common selector patterns:

• Tag: pup 'title'
• ID: pup 'span#See_also'
• Attribute: pup 'th[scope="row"]'
• Multiple selectors: Use a comma , to specify multiple groups, e.g., pup 'title, h1 span[dir="auto"]'
• Chaining: Combine selectors to drill down into nodes, e.g., pup 'h1#firstHeading span'

By default, pup will fill in missing tags and properly indent the page. Use the --color flag to enable syntax highlighting for the output.

cat robots.html | pup --color

pup implements a majority of relevant CSS selectors, including:

pup '.class'
pup '#id'
pup 'element'
pup 'selector + selector'
pup 'selector > selector'
pup '[attribute]'
pup '[attribute="value"]'
pup '[attribute*="value"]'
pup '[attribute~="value"]'
pup '[attribute^="value"]'
pup '[attribute$="value"]'
pup ':empty'
pup ':first-child'
pup ':first-of-type'
pup ':last-child'
pup ':last-of-type'
pup ':only-child'
pup ':only-of-type'
pup ':contains("text")'
pup ':nth-child(n)'
pup ':nth-of-type(n)'
pup ':nth-last-child(n)'
pup ':nth-last-of-type(n)'
pup ':not(selector)'
pup ':parent-of(selector)'

The ParseDisplayer function maps a command string to a specific Displayer implementation. This is used to determine how the results of a selection should be rendered to the output.

Supported command formats:

• text{}: Uses TextDisplayer to print the text content of the selected nodes.
• json{}: Uses JSONDisplayer to print the selected nodes as a JSON list.
• attr{<attribute_name>}: Uses AttrDisplayer to print the value of a specific attribute (e.g., attr{href}).

The ParseCommands function splits a single command string into individual selector/command components. It is aware of spaces, commas, and quoted text (both single ' and double " quotes), allowing for complex selectors that contain spaces or commas within quotes.

func ParseCommands(cmdString string) ([]string, error)

The Select function takes a Selector and returns a SelectorFunc. When called with a slice of *html.Node, it performs a recursive search through the tree to find all nodes that match the provided selector.

SelectorFunc signature: func(nodes []*html.Node) []*html.Node

Use ParseSelector to convert a CSS selector string (e.g., div#my-button.btn[href^="http"]) into a CSSSelector object. This object implements the Selector interface and can be used to match nodes in an HTML tree.

Supported selector components include:

• Tags: div
• IDs: #my-id
• Classes: .my-class
• Attributes: [attr="val"], [attr^="val"] (starts with), [attr$="val"] (ends with), [attr*="val"] (contains), [attr~="val"] (whitespace separated)
• Pseudo-classes: :empty, :first-child, :last-child, :only-child, :first-of-type, :last-of-type, :only-of-type, :contains("text"), :not(selector), :parent-of(selector), and nth-* variants.

The ParseHTML function parses an HTML document from an io.Reader. It handles character encoding by either attempting to guess the charset if cs is empty, or using the specific charset provided in the cs argument. If a specific charset is provided, it uses charset.Lookup to find the appropriate decoder.

func ParseHTML(r io.Reader, cs string) (*html.Node, error)