PetitParser for Dart

repository·main·Indexed 19 days ago

https://github.com/petitparser/dart-petitparser

A parser combinator library for Dart that enables the creation of dynamic, composable, and reusable grammars. It combines PEG, packrat parsing, and scannerless parsing techniques. The library includes tools for handling recursive grammars via GrammarDefinition, operator precedence via ExpressionBuilder, and debugging utilities such as trace, profile, and progress.

Tokens
11.2K
Snippets
36
Records
47
Agent score
67%

What's inside petitparser

  1. Overview of PetitParser for Dart

    main
    PetitParser for Dart is a parser combinator library that allows you to model grammars and parsers as dynamic objects. It combines concepts from scannerless parsing, parser combinators, parsing expression grammars (PEG), and packrat parsing to overcome the limitations of static grammar specifications, such as difficulty in composition and ambiguity.
  2. Using ExpressionBuilder for arithmetic expressions

    main

    The ExpressionBuilder is a specialized tool for defining grammars with operator precedence, associativity (left, right, or non-associative), and prefix/postfix operators.

    Workflow:

    1. Instantiate ExpressionBuilder<T>().
    2. Define a primitive (the base values, e.g., numbers).
    3. Define group().wrapper(...) for parentheses.
    4. Define operators using .prefix(), .right(), or .left() within groups to set precedence levels.
    final builder = ExpressionBuilder<num>();
    
    // 1. Primitive
    builder.primitive(digit().plus().flatten().map(num.parse));
    
    // 2. Parentheses (Wrapper)
    builder.group().wrapper(char('(').trim(), char(')').trim(), (left, value, right) => value);
    
    // 3. Operators (Precedence: highest to lowest)
    builder.group().prefix(char('-').trim(), (op, val) => -val);
    builder.group().right(char('^').trim(), (l, op, r) => math.pow(l, r));
    builder.group()
      ..left(char('*').trim(), (l, op, r) => l * r)
      ..left(char('+').trim(), (l, op, r) => l + r);
    
    final parser = builder.build().end();
  3. How GrammarDefinition works for complex grammars

    main

    For large or recursive grammars, use the GrammarDefinition class. This allows you to define productions as methods and resolve circular dependencies using ref(Function).

    1. Create a class extending GrammarDefinition.
    2. Define each production as a method returning a Parser.
    3. Use ref(methodName) to refer to other productions within the same class.
    4. Call .build() on the definition instance to create the final, resolved parser.

    To test specific parts of a large grammar, use buildFrom(productionMethod) to create a parser starting at that specific production.

    class ExpressionDefinition extends GrammarDefinition {
      Parser start() => ref(term).end();
    
      Parser term() => ref(add) | ref(prod);
      Parser add() => ref(prod) & char('+').trim() & ref(term);
    
      Parser prod() => ref(mul) | ref(prim);
      Parser mul() => ref(prim) & char('*').trim() & ref(prod);
    
      Parser prim() => ref(parens) | ref(number);
      Parser parens() => char('(').trim() & ref(term) & char(')').trim();
    
      Parser number() => digit().plus().flatten().trim();
    }
    
    final definition = ExpressionDefinition();
    final parser = definition.build();
  4. Debugging Parsers with trace, profile, and progress

    main

    When a parser behaves unexpectedly, use these three debugging tools:

    1. trace(Parser): Transforms the parser to print every activation and result. It uses indentation to show the tree structure of the parsing process.
    2. profile(Parser): Generates a table showing how many times each parser was activated and its runtime performance.
    3. progress(Parser): Visualizes how the parser moves through the input, including backtracking steps.
    final parser = letter() & word().star();
    
    // Use trace to see the execution tree
    trace(parser).parse('f1');
    
    // Use profile for performance stats
    profile(parser).parse('f1');
    
    // Use progress to see movement/backtracking
    progress(parser).parse('f1');
  5. Testing and Linting Grammars

    main

    Testing individual productions

    If using GrammarDefinition, use buildFrom to isolate and test specific parts of your grammar:

    final parser = definition.buildFrom(definition.number);

    Linting

    Use the linter function from package:petitparser/reflection.dart to detect common bugs like infinite loops, unreachable parsers, or unresolved references.

    import 'package:petitparser/reflection.dart';
    
    test('detect common problems', () {
      final definition = EvaluatorDefinition();
      final parser = definition.build();
      expect(linter(parser), isEmpty);
    });

    You can exclude specific rules using excludedRules: {'Rule Name'}.

  6. Install PetitParser for Dart

    main

    Follow the standard installation instructions on pub.dev.

    To use the library, import the main package:

    import 'package:petitparser/petitparser.dart';

    Alternatively, you can perform selective imports to reduce footprint:

    • package:petitparser/core.dart for core infrastructure.
    • package:petitparser/parser.dart for basic parsers.

    Important: This library relies heavily on static extension methods. If you use a library prefix (e.g., import ... as p;) or only selectively show classes, you may lose access to many of the parser's functionalities.

    import 'package:petitparser/petitparser.dart';
  7. Understand the SeparatedList return type

    main

    When using any of the starSeparated, plusSeparated, timesSeparated, or repeatSeparated methods, the resulting parser returns a SeparatedList<R, S>.

    This object holds two distinct collections:

    1. The elements of type R (the items being repeated).
    2. The separators of type S (the delimiters found between elements).

    This allows you to reconstruct the original structure or perform logic based on which specific separators were used.

  8. Implement alternative parsing paths with ChoiceParser

    main

    A ChoiceParser allows you to define multiple alternative parsing paths. It attempts to parse using each child parser in order, returning the result of the first one that succeeds. This is an exclusive ordered choice: if multiple parsers could match, the first one in the list that succeeds will be the one used.

    Key Behaviors

    • Ordering Matters: If you have overlapping parsers, such as letter().or(char('a')), the first parser (letter()) will always consume the input if it matches, potentially making the second parser unreachable.
    • Error Handling: If all parsers in the choice fail, the ChoiceParser returns a single Failure. You can control which failure is reported using a FailureJoiner.

    Creating a Choice Parser

    You can create a choice parser in three ways:

    1. Using the .or() extension: parserA.or(parserB)
    2. Using the | operator: parserA | parserB
    3. Using .toChoiceParser() on an Iterable: [parserA, parserB, parserC].toChoiceParser() (This is the recommended way to handle type safety when working with lists of parsers).
    // Using .or()
    final parser = letter().or(digit());
    
    // Using the | operator
    final parser = letter() | digit();
    
    // Using an Iterable (recommended for type safety)
    final parsers = [letter(), digit(), char('!')];
    final choiceParser = parsers.toChoiceParser();
  9. Use ExpressionBuilder to construct complex grammars

    main

    The ExpressionBuilder<T> provides a fluent API for defining expression grammars involving prefix, postfix, and infix operators (both left- and right-associative).

    To use it, follow these steps:

    1. Initialize: Create an instance with the desired return type: final builder = ExpressionBuilder<T>();.
    2. Define Primitives: Use .primitive(parser) to define the base values (e.g., numbers, booleans) that the expression will consist of. This parser is responsible for converting raw input into type T.
    3. Define Operator Groups: Use .group() to create groups of operators. Groups should be defined in descending order of precedence (highest precedence first).
      • Wrappers: Use .wrapper() for grouping symbols like parentheses. It takes the opening parser, closing parser, and a mapping function (left, value, right) => value.
      • Prefix Operators: Use .prefix(operatorParser, (operator, value) => result).
      • Postfix Operators: Use .postfix(operatorParser, (value, operator) => result).
      • Infix Operators:
        • Use .left(operatorParser, (left, operator, right) => result) for left-associative operators (e.g., +, -, *, /).
        • Use .right(operatorParser, (left, operator, right) => result) for right-associative operators (e.g., ^).
    4. Build: Call .build() to generate the final Parser<T>.
    final builder = ExpressionBuilder<num>();
    
    // 1. Define primitives
    builder.primitive(digit().plus().seq(char('.').seq(digit().plus()).optional()).flatten().trim().map(num.parse));
    
    // 2. Define high-precedence groups (parentheses)
    builder.group().wrapper(char('(').trim(), char(')').trim(), (left, value, right) => value);
    
    // 3. Define prefix operators (negation)
    builder.group().prefix(char('-').trim(), (operator, value) => -value);
    
    // 4. Define right-associative operators (power)
    builder.group().right(char('^').trim(), (left, operator, right) => math.pow(left, right));
    
    // 5. Define left-associative operators (multiplication/division)
    builder.group()
      ..left(char('*').trim(), (left, operator, right) => left * right)
      ..left(char('/').trim(), (left, operator, right) => left / right);
    
    // 6. Define left-associative operators (addition/subtraction)
    builder.group()
      ..left(char('+').trim(), (left, operator, right) => left + right)
      ..left(char('-').trim(), (left, operator, right) => left - right);
    
    final parser = builder.build();
  10. Explore PetitParser example grammars

    main

    The dart-petitparser-examples repository contains a wide variety of ready-to-use example grammars and language experiments. You can use these to understand how to implement complex parsers for different language types.

    Available examples include: