pest

To run a specific fuzzing target, follow these steps:

1. Navigate to the sub-directory of the crate containing the fuzzing targets (pest_meta or pest_grammars).
2. Switch to the nightly Rust toolchain using rustup.
3. Execute the target using cargo fuzz run [target].

Note for macOS users: If the first compilation fails with an error regarding the missing proc_macro dynamic library, simply run the cargo fuzz run [target] command a second time to resolve it.

To run the fuzzing targets provided in the pest repository, you must first install the cargo-fuzz tool.

cargo install cargo-fuzz

Use the #[derive(Parser)] procedural macro to automatically generate a parser from a .pest grammar file. You must specify the path to the grammar file using the #[grammar = "..."] attribute. The path is relative to the src directory.

#[derive(Parser)]
#[grammar = "path/to/my_grammar.pest"] // relative to src
struct MyParser;

The most common way to use pest is to use the pest_derive crate. You define a dummy struct and use the #[derive(Parser)] attribute along with a #[grammar = "..."] attribute pointing to your .pest file (relative to src).

#[derive(Parser)]
#[grammar = "path/to/my_grammar.pest"] // relative to src
struct MyParser;

Instead of using an external .pest file, you can define your grammar directly within your Rust code using the #[grammar_inline = "..."] attribute on your parser struct.

#[derive(Parser)]
#[grammar_inline = "rule = { "a" }"]
struct MyParser;

When using the pest_derive macro to generate a parser, you must provide the grammar source using one of two attributes on your struct, union, or enum:

1. #[grammar = "PATH"]: Specifies a path to a .pest file.
2. #[grammar_inline = "GRAMMAR CONTENTS"]: Specifies the grammar content directly as a string literal.

If no grammar attribute is provided, the macro will panic. Additionally, you can use the #[non_exhaustive] attribute to indicate that the generated Rule enum should be marked as #[non_exhaustive].

#[grammar = "path/to/grammar.pest"]
#[non_exhaustive]
pub struct MyParser<'a, T>;

The following fuzzing targets are available within the pest repository:

pest_meta crate

Located in the pest_meta/fuzz directory.

• parser: Tests pest_meta::parser::parse by providing random inputs to parse pest grammar files.

pest_grammars crate

Located in the pest_grammars/fuzz directory.

• http: Tests the HTTP request grammar.
• toml: Tests the TOML grammar.
• json: Tests the JSON grammar.

These targets interact with the pest::Parser::parse function provided by the respective Parsers in each module.

If the const_prec_climber feature is enabled, you can use the prec_climber! macro to define a PrecClimber as a static or const item. This is more convenient than manually defining precedence levels. Use L for left associativity and R for right associativity. Operators separated by | share the same precedence level.

Note: This requires the const_prec_climber feature to be active.

# use pest::prec_climber::{Assoc, PrecClimber};
# use pest::prec_climber;
# #[allow(non_camel_case_types)]
# #[allow(dead_code)]
# #[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
# enum Rule { plus, minus, times, divide, power }

static CLIMBER: PrecClimber<Rule> = prec_climber![
    L   plus | minus,
    L   times | divide,
    R   power,
];

Pest errors can be converted into miette::Error types using the .into_miette() method. This allows you to leverage miette's diagnostic capabilities, such as providing rich, colorized, and formatted error messages that include source code snippets and precise location information (e.g., using Position).

let input = "abc\ndef";
let pos = Position::new(input, 4).unwrap();
let error: Error<u32> = Error::new_from_pos(
    ErrorVariant::ParsingError {
        positives: vec![1, 2, 3],
        negatives: vec![4, 5, 6],
    },
    pos,
);

let miette_error = miette::Error::new(error.into_miette());

You can parse Pest grammar files into an Abstract Syntax Tree (AST) by using PestParser::parse with the Rule::grammar_rules rule, then processing the resulting pairs with consume_rules_with_spans. The resulting AST can be mapped to a custom AstRule structure for easier manipulation.

Note that the parser may throw errors for:

• Integer overflows in repetition counts (e.g., u32 overflow).
• Zero-length repetitions (e.g., {0,0}).
• Missing quotes in PUSH_LITERAL calls.
• Invalid syntax like missing assignment operators or incorrect modifiers.

let input = r#"
/// This is line comment
/// This is rule
rule = _{ a{1} ~ "a"{3,} ~ b{, 2} ~ "b"{1, 2} | !(^"c" | PUSH('d'..'e'))?* }
"#;

let pairs = PestParser::parse(Rule::grammar_rules, input).unwrap();
let ast = consume_rules_with_spans(pairs).unwrap();
let ast: Vec<_> = ast.into_iter().map(convert_rule).collect();