Soup Sieve

Soup Sieve is a CSS selector library for Beautiful Soup 4 that enables selecting, matching, and filtering using modern CSS selectors (from CSS level 1 through the latest CSS level 4 drafts).

It can be used automatically by Beautiful Soup's .select() method (as of version 4.7.0) or imported directly for specialized parsing.

Supported selector types include:

• Classes: .classes
• IDs: #ids
• Attributes: [attributes=value]
• Combinators: parent child, parent > child, sibling ~ sibling, sibling + sibling
• Pseudo-classes: :not(), :is(), :has()

Note: Selectors that do not have meaningful functionality in a non-browser environment will simply not match anything.

Soup Sieve supports modern CSS selectors from Level 1 through Level 4 drafts. Selectors can be multi-line and include CSS-style comments for documentation.

Supported Selector Types

• Classes: .class
• IDs: #id
• Attributes: [attr=value]
• Combinators: parent child, parent > child, sibling ~ sibling, sibling + sibling
• Pseudo-classes: :not(), :is(), :has()

Multi-line and Commented Selectors

You can define complex selectors as multi-line strings and include comments within them:

selector = """
/* This is a comment */
.a,
/* This is another comment */
.b
"""
sv.select(selector, soup)

The child combinator uses the > symbol. It matches an element only if it is a direct child of the first selector (i.e., the first selector is its immediate parent).

>>> from bs4 import BeautifulSoup as bs
>>> html = """
... <html>
... <head></head>
... <body>
... <div><p>Paragraph 1</p></div>
... <div><ul><li><p>Paragraph 2</p></li></ul></div>
... </body>
... </html>
... """
>>> soup = bs(html, 'html5lib')
>>> print(soup.select('div > p'))
[<p>Paragraph 1</p>]

The ID selector (#id) matches an element based on its id attribute. The ID must be an exact match. While primarily an HTML concept, this is also supported for XML.

>>> from bs4 import BeautifulSoup as bs
>>> html = """
... <html>
... <head></head>
... <body>
...    <div id="some-id">Here is some text.</div>
...    <div>Here is some more text.</div>
... </body>
... </html>
... """
>>> soup = bs(html, 'html5lib')
>>> print(soup.select('#some-id'))
[<div id="some-id">Here is some text.</div>]

You can use a comma (,) to create a selector list. When using a selector list, any element that matches any of the individual selectors in the list will be returned.

>>> from bs4 import BeautifulSoup as bs
>>> html = """
... <html>
... <head></head>
... <body>
... <h1>Title</h1>
... <p>Paragraph</p>
... </body>
... </html>
... """
>>> soup = bs(html, 'html5lib')
>>> print(soup.select('h1, p'))
[<h1>Title</h1>, <p>Paragraph</p>]

Soup Sieve uses standard CSS terminology to categorize selectors:

• Selector: A general term covering simple, compound, or complex selectors.
• Simple Selector: Represents a single condition on an element. Examples include:
  • Type selector
  • Universal selector
  • ID selector
  • Class selector
  • Attribute selector
  • Pseudo-class selector
• Compound Selector: A sequence of simple selectors without combinators. If a universal or type selector is used, it must appear first, and you cannot use both a universal and a type selector in the same compound selector.
• Complex Selector: Multiple simple or compound selectors joined together using combinators.
• Selector List: A list of selectors separated by commas (,). A match is valid if any selector in the list matches the element.

Soup Sieve recognizes several CSS pseudo-classes that are considered "non-applicable" in a Beautiful Soup environment. While the parser will correctly recognize and parse these selectors without error, they will never match any elements.

This is because these pseudo-classes rely on a live, interactive browser environment (such as user interaction, media playback state, or focus state) which does not exist when parsing static HTML/XML documents. If these pseudo-classes are removed from the CSS specification in the future, Soup Sieve will simply remove them from its supported set.

Soup Sieve adjusts its behavior based on the document type detected from the Beautiful Soup object:

• HTML (HTML, HTML5): Tag names and attribute names are treated case-insensitively.
• XML (including XHTML): Tag names and attribute names are treated case-sensitively.
• Namespaces: Evaluated based on the parser's support.

Namespace Support Tips

• For HTML5 namespace support, use the html5lib parser. The lxml HTML parser does not provide proper namespaces.
• For XML/XHTML, use the lxml-xml parser (often referred to as xml) to ensure proper namespace handling.

Attribute Case Sensitivity

While attribute values are generally case-sensitive, the type attribute in HTML/HTML5 is always treated as case-insensitive. To force case-sensitivity for the type attribute, use the s flag in your selector: #!css [type="submit" s].

The class selector (.class) matches elements based on the values in their class attribute. The class attribute is treated as a whitespace-separated list of individual classes. This is also supported for XML.

>>> from bs4 import BeautifulSoup as bs
>>> html = """
... <html>
... <head></head>
... <body>
...    <div class="some-class">Here is some text.</div>
...    <div>Here is some more text.</div>
... </body>
... </html>
... """
>>> soup = bs(html, 'html5lib')
>>> print(soup.select('.some-class'))
[<div class="some-class">Here is some text.</div>]

The Universal selector (*) matches every element in the document regardless of its type.

>>> from bs4 import BeautifulSoup as bs
>>> html = """
... <html>
... <head></head>
... <body>
...    <p>Here is some text.</p>
...    <div>Here is some more text.</div>
... </body>
... </html>
... """
>>> soup = bs(html, 'html5lib')
>>> print(soup.select('*'))
[<html><head></head>... <head></head>... <body...>... <div...>...]