cssparser

repository·main·Indexed 21 days ago

https://github.com/servo/rust-cssparser

A Rust implementation of the CSS Syntax Module Level 3. It provides foundational parsing logic, including tokenization and component value construction, to build a CSS engine. The library handles UTF-8 input and provides specialized utilities for parsing CSS colors (hex, named, and predefined color spaces), as well as an efficient string type called CowRcStr. It includes a system for resolving stylesheet encoding and a comprehensive test suite for validating AST node representations.

Tokens
10.5K
Snippets
34
Records
44
Agent score
75%

What's inside cssparser

  1. Understand the JSON test file format

    main

    Each .json file in the test suite corresponds to a specific CSS parsing function. The files are UTF-8 encoded and follow a specific structure:

    • The file contains a single JSON array.
    • The array contains an even number of items.
    • Items are organized in pairs: [input, expected_result].

    Common Test Files

    FilePurpose
    stylesheet.jsonTests parsing a full stylesheet.
    rule_list.jsonTests parsing a list of rules.
    declaration_list.jsonTests parsing a list of declarations.
    one_rule.jsonTests parsing a single rule.
    one_declaration.jsonTests parsing a single declaration.
    component_value_list.jsonTests parsing a list of component values.
    color3.jsonTests <color> syntax (outputs RGBA arrays).
    An+B.jsonTests An+B syntax (outputs [A, B] integers).
    urange.jsonTests urange syntax (outputs [start, end] integers).

    Special Input Formats

    For stylesheet_bytes.json, the input is a JSON object containing:

    • css_bytes: The input byte string (U+0000 to U+00FF represent raw bytes).
    • protocol_encoding (optional): A string label or null.
    • environment_encoding (optional): A string label or null.
    • comment (optional): Ignored.
  2. Understand the rust-cssparser parsing model

    main

    The rust-cssparser library implements the CSS Syntax Module Level 3 through a multi-step process, though it combines some steps for efficiency:

    1. Encoding/Decoding: The library currently assumes UTF-8 input. It does not handle character encoding detection (e.g., via @charset or Content-Type headers) automatically.
    2. Tokenization & Component Value Construction: Instead of producing a flat stream of raw tokens, the library performs tokenization and the construction of a tree of component values simultaneously. This means raw tokens are not materialized; you interact directly with component values which can be:
      • Preserved tokens
      • Blocks/Functions (e.g., { ... }, [ ... ], ( ... ), or foo( ... )) containing nested component values.
    3. Rule/Declaration Parsing: Component values can be organized into generic rules or declarations. At this stage, the header, body, and declaration values are still represented as lists of component values (see the Token enum in src/tokenizer.rs).

    Note on Selectors and Properties: rust-cssparser does not perform the final step of parsing component values into specific CSS Selectors or specific CSS properties. This is by design to allow consumers to define their own logic for property and selector support. However, it provides helpers for specific types like CSS colors and An+B (used in :nth-child() selectors).

  3. Import CSS parsing tests via git-subtree

    main

    The recommended way to integrate these implementation-independent CSS parsing tests into your project is using git subtree. This allows you to maintain the tests as a sub-directory within your own repository.

    To import the tests for the first time into a ./css-parsing-tests directory, run:

    git subtree add -P css-parsing-tests https://github.com/SimonSapin/css-parsing-tests.git master

    To pull and merge subsequent updates from the upstream repository, run:

    git subtree pull -P css-parsing-tests https://github.com/SimonSapin/css-parsing-tests.git master
    git subtree add -P css-parsing-tests https://github.com/SimonSapin/css-parsing-tests.git master
  4. Use CowRcStr for efficient string handling

    main

    CowRcStr<'a> is a specialized string type designed for memory efficiency. It can represent a string in two states:

    1. Borrowed: A reference to an existing &'a str.
    2. Shared: A heap-allocated String managed via reference counting (Rc<String>).

    This type is more compact than a standard enum { Borrowed(&'a str), Shared(Rc<String>) }. It implements Deref<Target = str>, AsRef<str>, and Borrow<str>, allowing it to be used almost anywhere a &str is expected.

    use cssparser::CowRcStr;
    
    // From a borrowed string
    let borrowed = CowRcStr::from("hello");
    
    // From an owned String (becomes shared/Rc)
    let owned = CowRcStr::from(String::from("world"));
    
    // Use it as a &str via Deref
    println!("{}", borrowed); 
    assert_eq!(borrowed.as_ref(), "hello");
  5. Manage Parser state with ParserState

    main

    The ParserState struct captures the internal state of a Parser, including its position within the input, current line number, and current line start position. You can use Parser::state() to obtain a ParserState and Parser::reset() to restore it later. This is useful for implementing backtracking or trial parsing.

    Note: A ParserState should only be used with the Parser instance it was originally obtained from.

    // Capture state
    let state = parser.state();
    
    // ... perform some parsing that might fail ...
    
    // Restore state if needed
    parser.reset(&state);
  6. Follow parsing function conventions

    main

    When implementing custom parsing functions for use with cssparser, follow these rules:

    1. Signature: Functions should take at least a &mut cssparser::Parser parameter and return Result<_, ()>.
    2. Success (Ok): On success, the function must have consumed exactly the amount of input representing the parsed value.
    3. Failure (Err): On failure, any amount of input may have been consumed.

    Error Handling and Parser::try

    Because an error might leave the Parser in an indeterminate position, you have two choices when calling a parsing function:

    • Propagate Errors: Use the ? operator to propagate Err(()) immediately. This is standard for tail calls or when the error is fatal to the current context.
    • Use Parser::try: If you want to attempt a parse without permanently advancing the parser position on failure, wrap the call in input.try_parse(|input| { ... }). The try method saves the current position and restores it if the provided closure returns Err.
    // Example: Using try_parse to attempt an optional parse without advancing on error
    fn parse_border_spacing(_context: &ParserContext, input: &mut Parser) 
        -> Result<(LengthOrPercentage, LengthOrPercentage), ()> 
    {
        let first = LengthOrPercentage::parse?; 
        // If LengthOrPercentage::parse fails inside try_parse, 'input' position is restored
        let second = input.try_parse(LengthOrPercentage::parse).unwrap_or(first);
        (first, second)
    }
  7. Handle parsing errors with ParseError

    main

    The ParseError<'i, E> type is the extensible error type used by the parser. It consists of a ParseErrorKind<'i, E> and a SourceLocation.

    ParseErrorKind can be one of two types:

    1. Basic(BasicParseErrorKind<'i>): Fundamental errors triggered by built-in routines (e.g., UnexpectedToken, EndOfInput, AtRuleInvalid).
    2. Custom(E): Errors reported by downstream consumer code, where E is a user-defined error type.

    You can use SourceLocation methods like new_basic_error or new_custom_error to create errors at specific locations.

  8. Handle non-UTF-8 input with stylesheet_encoding

    main

    The Parser objects in cssparser operate on &str (UTF-8) input. If your source data is in bytes (e.g., from a file or network) and uses a character encoding other than UTF-8, use the stylesheet_encoding function. This function can be used in conjunction with rust-encoding or encoding-rs to convert bytes into a UTF-8 string suitable for parsing.

    use cssparser::from_bytes::stylesheet_encoding;
    // Use stylesheet_encoding to convert bytes to a UTF-8 string before parsing.
  9. Reference: AST Node Result Representations

    main

    The test suite represents parsed AST nodes using compact JSON arrays. Use these schemas to validate your parser's output against the expected results.

    Rules and Declarations

    TypeJSON Structure
    At-rule["at-rule", name, prelude_array, block_array_or_null]
    Qualified rule["qualified rule", prelude_array, block_array]
    Declaration["declaration", name, value_array, important_boolean]

    Component Values

    TypeJSON Structure
    <ident>["ident", value_string]
    <at-keyword>["at-keyword", value_string]
    <hash>["hash", value_string, "id" | "unrestricted"]
    <string>["string", value_string]
    <bad-string>["bad-string"]
    <url>["url", value_string]
    <bad-url>["bad-url"]
    <delim>"char" (single character string)
    <number>["number", representation_string, numeric_value, "integer" | "number"]
    <percentage>["percentage", representation_string, numeric_value, "integer" | "number"]
    <dimension>["dimension", representation_string, numeric_value, "integer" | "number", unit_string]
    Function["function", name_string, ...arguments_component_values]
    {} block["{}", ...content_component_values]
    [] block["[]", ...content_component_values]
    () block["()", ...content_component_values]
    <include-match>"~="
    <dash-match>"|="
    <prefix-match>"^="
    <suffix-match>"$="
    <substring-match>"*="
    <whitespace>" " (single space)
    <CDO>"<!--"
    <CDC>"-->"
    <colon>":"
    <semicolon>";"
    <comma>","
    Error States["error", "bad-string"], ["error", "bad-url"], ["error", "}"], ["error", "]"], or ["error", ")"]
  10. Use try_parse for trial parsing and backtracking

    main

    The try_parse method executes a closure that performs parsing. If the closure returns an Err, the parser's internal state (including its position) is automatically restored to what it was before the call. This is a convenient way to attempt a parsing pattern without manually managing ParserState.

    parser.try_parse(|p| {
        // Attempt to parse a specific pattern
        p.expect_ident()?;
        p.expect_colon()?;
        // ...
        Ok(())
    })?
  11. Serialize CSS identifiers and names

    main

    Use these functions to write CSS identifiers or names while ensuring proper character escaping according to CSS syntax rules.

    • serialize_identifier<W>(mut value: &str, dest: &mut W) -> fmt::Result: Writes a CSS identifier. It handles special cases like custom properties (starting with --) and ensures that identifiers starting with digits or hyphens are correctly escaped.
    • serialize_name<W>(value: &str, dest: &mut W) -> fmt::Result: Writes a CSS name (e.g., a custom property name).

    Note: Use serialize_identifier whenever in doubt; serialize_name should only be used when you are certain of the specific requirements of the name being written.

    // Example usage of serialize_identifier
    let mut buffer = String::new();
    serialize_identifier("--my-custom-prop", &mut buffer).unwrap();
  12. Implement a DeclarationParser to parse CSS property values

    main

    To parse CSS declarations (e.g., color: red;), implement the DeclarationParser trait. This trait allows you to define how specific property names are handled and how their values are parsed.

    When implementing parse_value, the input is a delimited parser that ends at the next semicolon or the end of the current block. If the property supports !important, you should call input.try_parse(parse_important).is_ok() at the end of your implementation.

    Note: Declaration name matching should be case-insensitive in the ASCII range.

    impl<'i> DeclarationParser<'i> for MyParser {
        type Declaration = MyValue;
        type Error = MyError;
    
        fn parse_value<'t>(
            &mut self,
            name: CowRcStr<'i>,
            input: &mut Parser<'i, 't>,
            _declaration_start: &ParserState,
        ) -> Result<Self::Declaration, ParseError<'i, Self::Error>> {
            // Implementation logic here
            // Example: check for !important
            // input.try_parse(parse_important).is_ok();
            todo!()
        }
    }