swift-parsing

repository·main·Indexed 21 days ago

https://github.com/pointfreeco/swift-parsing

A Swift library for transforming unstructured data into structured data. It emphasizes composition, performance, generality, and invertibility, allowing parsers to be converted into printers for serialization. The library features a Parser protocol, a @Parse result builder for declarative syntax, and specialized tools like OneOf for backtracking and Many for collection parsing.

Tokens
10.1K
Snippets
36
Records
47
Agent score
76%

What's inside swift-parsing

  1. What is swift-parsing?

    main

    swift-parsing is a library designed to transform unstructured data into structured data. It focuses on four core pillars:

    • Composition: Breaking complex parsing problems into smaller, simpler parsers that can be combined.
    • Performance: Composed parsers are designed to perform as well as highly-tuned, hand-written parsers.
    • Generality: The ability to parse any kind of input (e.g., String, Substring, UTF8View, Array) into any kind of output.
    • Invertibility: Parsers can be converted into 'printers', allowing you to transform structured data back into unstructured data (serialization).
  2. What is backtracking in swift-parsing?

    main

    Backtracking is the process of restoring the input to its original state when a parser fails. This allows subsequent parsers to attempt to consume the same input.

    While useful, excessive backtracking can lead to performance degradation because it often causes the same input to be parsed multiple times. Most parsers in this library do not require explicit backtracking management unless you are implementing custom Parser conformances.

  3. What is round-tripping in parser-printers?

    main

    Round-tripping is a critical property for parser-printer pairs to ensure that complex parsers built from simpler components behave predictably. A parser-printer p satisfies round-tripping if it meets two conditions:

    1. Parse-then-Print: For every input where p.parse(input) succeeds, the result of printing that parsed value (p.print(p.parse(input))) must be identical to the original input.
    2. Print-then-Parse: For every output where p.print(output) succeeds, the result of parsing that printed string (p.parse(p.print(output))) must be identical to the original output.

    Ensuring round-tripping helps guarantee that no information is lost or transformed unexpectedly during the conversion between raw input and structured data.

  4. Overview of the swift-parsing approach

    main

    Parsing in swift-parsing is achieved by composing many small parsers that incrementally consume bits from the beginning of an input. You build complex parsers by listing small parsers (like Int.parser()) and literal separators (like `

    // Example of composing a parser for a User struct
    let user = Parse(input: Substring.self, User.init) {
      Int.parser()
      ,"
      Prefix { $0 != "" }.map(String.init)
      ,"
      Bool.parser()
    }
    
    // Example of using Many to parse a collection
    let users = Many {
      user
    } separator: {
      "\n"
    } terminator: {
      End()
    }
    
    try users.parse(input)
  5. Implement backtracking in custom Parser conformances

    main

    If you are creating your own conformances to the Parser protocol, you must manage backtracking manually.

    Rule of thumb: If your parser recovers from failures within the parse method (i.e., it catches the error and returns a successful result instead of throwing), it must backtrack the input to its state before the error occurred. This ensures consistency with built-in parsers like OneOf, Backtracking, Optionally, Not, and replaceError(with:).

  6. How to use the Parse type to create a parser

    main

    The Parse type is the primary entry point for describing a sequence of parsers that consume input. You can specify the input type (e.g., Substring.self) and provide a transformation closure to map the parsed components into a structured type.

    There are three common ways to define the transformation:

    1. Using .map: Define the parser and then map the resulting tuple to your type.
    2. Using the with: trailing argument: Pass the transformation closure as the first argument and the parser definition in the with: block.
    3. Point-free style: Pass the type's initializer directly to Parse if the parser outputs match the initializer's arguments.
    // 1. Using .map
    let user = Parse(input: Substring.self) {
      Int.parser()
      ,
      Prefix { $0 != "," }
      ,
      Bool.parser()
    }.map { User(id: $0, name: String($1), isAdmin: $2) }
    
    // 2. Using 'with:'
    let user = Parse(input: Substring.self) { 
      User(id: $0, name: String($1), isAdmin: $2) 
    } with: {
      Int.parser()
      ,
      Prefix { $0 != "," }
      ,
      Bool.parser()
    }
    
    // 3. Point-free style
    let user = Parse(input: Substring.self, User.init(id:name:isAdmin:)) {
      Int.parser()
      ,
      Prefix { $0 != "," }.map(String.init)
      ,
      Bool.parser()
    }
  7. Use common parsers to build complex logic

    main

    The library provides a collection of built-in parsers that serve as building blocks. You can combine these using operators to construct complex parsing logic.

    Commonly used primitive parsers include:

    • Int, Float, Bool, UUID for specific types.
    • Digits, String, CharacterSet for character-based parsing.
    • Whitespace, Newline for structural parsing.

    Combinators and control parsers include:

    • OneOf: Matches one of several possible parsers.
    • Many: Matches zero or more occurrences.
    • Optionally: Matches an optional component.
    • Prefix, PrefixThrough, PrefixUpTo: For handling segments of input.
    • Always, Fail, End: For controlling parser flow and validation.
    • Skip, Peek, Not: For lookahead and skipping logic.
  8. Create a parser with Parse

    main

    The Parse type is the entry point for describing a sequence of parsers that consume input one after another. You can specify the input type (e.g., Substring.self) to ensure the parser processes the correct abstraction.

    Common patterns include:

    • Trailing closure: Use .map on the Parse instance to transform the resulting tuple into a custom type.
    • with: syntax: Pass the transformation closure as the first argument to Parse to make the target data more prominent.
    • Point-free style: Pass an initializer directly to Parse if the parser outputs match the initializer's arguments.
    // Using .map
    let user = Parse(input: Substring.self) {
      Int.parser()
      ","
      Prefix { $0 != "," }
      ","
      Bool.parser()
    }
    .map { User(id: $0, name: String($1), isAdmin: $2) }
    
    // Using 'with:'
    let user = Parse(input: Substring.self) {
      User(id: $0, name: String($1), isAdmin: $2)
    } with: {
      Int.parser()
      ","
      Prefix { $0 != "," }
      ","
      Bool.parser()
    }
    
    // Point-free style
    let user = Parse(input: Substring.self, User.init(id:name:isAdmin:)) {
      Int.parser()
      ","
      Prefix { $0 != "," }.map(String.init)
      ","
      Bool.parser()
    }
  9. Turn a parser into a printer using ParserPrinter

    main

    A parser-printer is a parser that can also perform the inverse process of printing: turning well-structured data back into raw data (like a String) for saving to disk or sending over a network.

    Most Parser conformances in this library also conform to the ParserPrinter protocol. To ensure your parser is a printer, you must use operations that are bidirectional. If you use one-directional transformations (like the standard map(_:) that takes a simple function), the resulting object will be a Parser but not a ParserPrinter.

    To explicitly define a parser-printer, you can use the ParsePrint entry point.

    // A simple parser that is also a printer because all its components are printers
    let quotedField = Parse {
      "\"
      Prefix { $0 != "\"" }
      "\"
    }
    
    // Using the print(_:) method from ParserPrinter
    quotedField.print("Blob, Esq.") // ✅ "\"Blob, Esq.\""
  10. Understand string abstraction levels in swift-parsing

    main

    The library operates on different "views" into a string rather than String directly. Choosing an abstraction level allows you to trade performance for correctness.

    Abstraction Levels

    • Substring: A collection of Characters (extended grapheme clusters).

      • Pros: Easy to use, handles UTF-8 complexities (like normalization) automatically.
      • Cons: Less efficient; scanning is an $O(n)$ operation because elements are variable width.
    • UnicodeScalarView: A collection of Unicode.Scalars.

      • Pros: More efficient than Substring because scalars are fixed-width (21-bit).
      • Cons: Higher complexity. A single visual Character (like a flag emoji) may consist of multiple scalars. Different scalar sequences can represent the same visual character (e.g., a precomposed 'é' vs. a base 'e' with a combining accent), meaning equality checks on the view may fail even if the characters look identical.
    • UTF8View: A collection of Unicode.UTF8.CodeUnits (typealiased to UInt8).

      • Pros: Very efficient scanning.
      • Cons: High complexity; requires manual handling of multi-byte UTF-8 sequences.
    • ArraySlice<UInt8>: A raw collection of bytes.

      • Pros: Most efficient; does not require valid UTF-8 representation.
      • Cons: No guarantee of lossless conversion back to a String.
  11. Understand and debug parsing errors

    main

    When a parser fails, it throws an error containing information about the failure. The specific error type used by the library is internal and should be treated as opaque. To obtain a human-readable debug description, simply stringify the error (e.g., by printing it).

    OneOf Error Prioritization

    When using the OneOf parser, if multiple parsers fail, the library prioritizes the error message from the parser that progressed the furthest into the input. This helps identify the most likely intended structure even when the input is malformed.

    do {
      var input = "1234 Hello"[...].utf8
      let number = try UInt8.parser().parse(&input)
    } catch {
      print(error)
      // error: failed to process "UInt8"
      //  --> input:1:1-4
      // 1 | 1234 Hello
      //   | ^^^^ overflowed 255
    }
  12. How the Parser protocol and combinators work

    main

    The library is designed around a Parser protocol. A parser is a type that can consume input and return an output.

    Parser transformations, also known as "combinators", are methods (like .map) that take an existing parser and return a new concrete type that conforms to the Parser protocol. This allows you to build complex parsing logic by composing simpler parsers.

    Because the type of a parser encodes every operation performed (e.g., Parsers.Map<Prefix<Substring>, Substring>), the Swift compiler can often inline and optimize these nested types for high performance.

    // Using a combinator like Prefix
    let parser = Prefix { $0 != "," }
    
    var input = "Hello,World"[...]
    try parser.parse(&input) // "Hello"