PetitParser for Dart
repository·main·Indexed 19 days ago
https://github.com/petitparser/dart-petitparserA parser combinator library for Dart that enables the creation of dynamic, composable, and reusable grammars. It combines PEG, packrat parsing, and scannerless parsing techniques. The library includes tools for handling recursive grammars via GrammarDefinition, operator precedence via ExpressionBuilder, and debugging utilities such as trace, profile, and progress.
What's inside petitparser
- PetitParser for Dart is a parser combinator library that allows you to model grammars and parsers as dynamic objects. It combines concepts from scannerless parsing, parser combinators, parsing expression grammars (PEG), and packrat parsing to overcome the limitations of static grammar specifications, such as difficulty in composition and ambiguity.
Using ExpressionBuilder for arithmetic expressions
mainThe
ExpressionBuilderis a specialized tool for defining grammars with operator precedence, associativity (left, right, or non-associative), and prefix/postfix operators.Workflow:
- Instantiate
ExpressionBuilder<T>(). - Define a
primitive(the base values, e.g., numbers). - Define
group().wrapper(...)for parentheses. - Define operators using
.prefix(),.right(), or.left()within groups to set precedence levels.
final builder = ExpressionBuilder<num>(); // 1. Primitive builder.primitive(digit().plus().flatten().map(num.parse)); // 2. Parentheses (Wrapper) builder.group().wrapper(char('(').trim(), char(')').trim(), (left, value, right) => value); // 3. Operators (Precedence: highest to lowest) builder.group().prefix(char('-').trim(), (op, val) => -val); builder.group().right(char('^').trim(), (l, op, r) => math.pow(l, r)); builder.group() ..left(char('*').trim(), (l, op, r) => l * r) ..left(char('+').trim(), (l, op, r) => l + r); final parser = builder.build().end();- Instantiate
How GrammarDefinition works for complex grammars
mainFor large or recursive grammars, use the
GrammarDefinitionclass. This allows you to define productions as methods and resolve circular dependencies usingref(Function).- Create a class extending
GrammarDefinition. - Define each production as a method returning a
Parser. - Use
ref(methodName)to refer to other productions within the same class. - Call
.build()on the definition instance to create the final, resolved parser.
To test specific parts of a large grammar, use
buildFrom(productionMethod)to create a parser starting at that specific production.class ExpressionDefinition extends GrammarDefinition { Parser start() => ref(term).end(); Parser term() => ref(add) | ref(prod); Parser add() => ref(prod) & char('+').trim() & ref(term); Parser prod() => ref(mul) | ref(prim); Parser mul() => ref(prim) & char('*').trim() & ref(prod); Parser prim() => ref(parens) | ref(number); Parser parens() => char('(').trim() & ref(term) & char(')').trim(); Parser number() => digit().plus().flatten().trim(); } final definition = ExpressionDefinition(); final parser = definition.build();- Create a class extending
Explore PetitParser examples
mainYou can find more elaborate examples of using PetitParser in Dart in the official example repository or via the hosted demo page.
- Example Repository: https://github.com/petitparser/dart-petitparser-examples
- Demo Page: https://petitparser.github.io/
Debugging Parsers with trace, profile, and progress
mainWhen a parser behaves unexpectedly, use these three debugging tools:
trace(Parser): Transforms the parser to print every activation and result. It uses indentation to show the tree structure of the parsing process.profile(Parser): Generates a table showing how many times each parser was activated and its runtime performance.progress(Parser): Visualizes how the parser moves through the input, including backtracking steps.
final parser = letter() & word().star(); // Use trace to see the execution tree trace(parser).parse('f1'); // Use profile for performance stats profile(parser).parse('f1'); // Use progress to see movement/backtracking progress(parser).parse('f1');Testing and Linting Grammars
mainTesting individual productions
If using
GrammarDefinition, usebuildFromto isolate and test specific parts of your grammar:final parser = definition.buildFrom(definition.number);Linting
Use the
linterfunction frompackage:petitparser/reflection.dartto detect common bugs like infinite loops, unreachable parsers, or unresolved references.import 'package:petitparser/reflection.dart'; test('detect common problems', () { final definition = EvaluatorDefinition(); final parser = definition.build(); expect(linter(parser), isEmpty); });You can exclude specific rules using
excludedRules: {'Rule Name'}.Install PetitParser for Dart
mainFollow the standard installation instructions on pub.dev.
To use the library, import the main package:
import 'package:petitparser/petitparser.dart';Alternatively, you can perform selective imports to reduce footprint:
package:petitparser/core.dartfor core infrastructure.package:petitparser/parser.dartfor basic parsers.
Important: This library relies heavily on static extension methods. If you use a library prefix (e.g.,
import ... as p;) or only selectively show classes, you may lose access to many of the parser's functionalities.import 'package:petitparser/petitparser.dart';Understand the SeparatedList return type
mainWhen using any of the
starSeparated,plusSeparated,timesSeparated, orrepeatSeparatedmethods, the resulting parser returns aSeparatedList<R, S>.This object holds two distinct collections:
- The elements of type
R(the items being repeated). - The separators of type
S(the delimiters found between elements).
This allows you to reconstruct the original structure or perform logic based on which specific separators were used.
- The elements of type
Implement alternative parsing paths with ChoiceParser
mainA
ChoiceParserallows you to define multiple alternative parsing paths. It attempts to parse using each child parser in order, returning the result of the first one that succeeds. This is an exclusive ordered choice: if multiple parsers could match, the first one in the list that succeeds will be the one used.Key Behaviors
- Ordering Matters: If you have overlapping parsers, such as
letter().or(char('a')), the first parser (letter()) will always consume the input if it matches, potentially making the second parser unreachable. - Error Handling: If all parsers in the choice fail, the
ChoiceParserreturns a singleFailure. You can control which failure is reported using aFailureJoiner.
Creating a Choice Parser
You can create a choice parser in three ways:
- Using the
.or()extension:parserA.or(parserB) - Using the
|operator:parserA | parserB - Using
.toChoiceParser()on an Iterable:[parserA, parserB, parserC].toChoiceParser()(This is the recommended way to handle type safety when working with lists of parsers).
// Using .or() final parser = letter().or(digit()); // Using the | operator final parser = letter() | digit(); // Using an Iterable (recommended for type safety) final parsers = [letter(), digit(), char('!')]; final choiceParser = parsers.toChoiceParser();- Ordering Matters: If you have overlapping parsers, such as
Use ExpressionBuilder to construct complex grammars
mainThe
ExpressionBuilder<T>provides a fluent API for defining expression grammars involving prefix, postfix, and infix operators (both left- and right-associative).To use it, follow these steps:
- Initialize: Create an instance with the desired return type:
final builder = ExpressionBuilder<T>();. - Define Primitives: Use
.primitive(parser)to define the base values (e.g., numbers, booleans) that the expression will consist of. This parser is responsible for converting raw input into typeT. - Define Operator Groups: Use
.group()to create groups of operators. Groups should be defined in descending order of precedence (highest precedence first).- Wrappers: Use
.wrapper()for grouping symbols like parentheses. It takes the opening parser, closing parser, and a mapping function(left, value, right) => value. - Prefix Operators: Use
.prefix(operatorParser, (operator, value) => result). - Postfix Operators: Use
.postfix(operatorParser, (value, operator) => result). - Infix Operators:
- Use
.left(operatorParser, (left, operator, right) => result)for left-associative operators (e.g.,+,-,*,/). - Use
.right(operatorParser, (left, operator, right) => result)for right-associative operators (e.g.,^).
- Use
- Wrappers: Use
- Build: Call
.build()to generate the finalParser<T>.
final builder = ExpressionBuilder<num>(); // 1. Define primitives builder.primitive(digit().plus().seq(char('.').seq(digit().plus()).optional()).flatten().trim().map(num.parse)); // 2. Define high-precedence groups (parentheses) builder.group().wrapper(char('(').trim(), char(')').trim(), (left, value, right) => value); // 3. Define prefix operators (negation) builder.group().prefix(char('-').trim(), (operator, value) => -value); // 4. Define right-associative operators (power) builder.group().right(char('^').trim(), (left, operator, right) => math.pow(left, right)); // 5. Define left-associative operators (multiplication/division) builder.group() ..left(char('*').trim(), (left, operator, right) => left * right) ..left(char('/').trim(), (left, operator, right) => left / right); // 6. Define left-associative operators (addition/subtraction) builder.group() ..left(char('+').trim(), (left, operator, right) => left + right) ..left(char('-').trim(), (left, operator, right) => left - right); final parser = builder.build();- Initialize: Create an instance with the desired return type:
Run the command-line calculator example
mainThe
example/directory contains a command-line calculator implementation used in the introductory tutorial. You can run it using the Dart CLI by passing a mathematical expression as a string argument.dart example/calc.dart "1 + 2 * 3"Explore PetitParser example grammars
mainThe
dart-petitparser-examplesrepository contains a wide variety of ready-to-use example grammars and language experiments. You can use these to understand how to implement complex parsers for different language types.Available examples include: