syn

repository·master·Indexed 25 days ago

https://github.com/dtolnay/syn

A parsing library for Rust source code that converts a stream of tokens into a structured syntax tree. Primarily used for building procedural macros, syn provides robust data structures for representing Rust source code and detailed span information for error reporting. Version 3.0.3.

Tokens
12.9K
Snippets
28
Records
66
Agent score
85%

What's inside syn

  1. Use proc-macro2 for better compatibility

    master

    Syn operates on the proc-macro2 token representation rather than the compiler's built-in proc_macro crate. This allows Syn code to run in contexts like unit tests or build.rs outside of a procedural macro environment.

    Best Practice: Write all your logic against proc-macro2. The only exception is the signature of your procedural macro entry points, which must use proc_macro::TokenStream as required by the Rust language. The proc-macro2 crate will automatically switch to using the compiler's data structures when running inside a procedural macro.

  2. Implement a derive macro with Syn

    master

    To create a derive macro, define a function with the #[proc_macro_derive(Name)] attribute. The function receives a proc_macro::TokenStream, which you should parse into a syn::DeriveInput using the parse_macro_input! macro. You can then use the quote crate to generate the expanded code and return it as a TokenStream.

    Dependencies

    Ensure your Cargo.toml is configured for a procedural macro library:

    [package]
    ...
    
    [lib]
    proc-macro = true
    
    [dependencies]
    syn = "3"
    quote = "1"
    use proc_macro::TokenStream;
    use quote::quote;
    use syn::{parse_macro_input, DeriveInput};
    
    #[proc_macro_derive(MyMacro)]
    pub fn my_macro(input: TokenStream) -> TokenStream {
        // Parse the input tokens into a syntax tree
        let input = parse_macro_input!(input as DeriveInput);
    
        // Build the output, possibly using quasi-quotation
        let expanded = quote! {
            // ...
        };
    
        // Hand the output tokens back to the compiler
        TokenStream::from(expanded)
    }
  3. Regenerate syn_codegen files

    master

    The syn_codegen crate is an internal tool used to generate the files in the gen/ directory of the syn repository. This ensures that implementations for the Fold, Visit, and VisitMut traits remain synchronized with the actual AST.

    Note: This program is intentionally slow and is not executed during the standard syn build process to optimize compile times. Only run this if you are modifying the AST and need to update the generated trait implementations.

  4. Troubleshoot Syn parsers with syn-dev

    master

    Use the syn-dev project skeleton to troubleshoot syn parsers, particularly when adding support for new Rust syntax.

    To use it:

    1. Place a sample of the syntax you are working on into main.rs.
    2. Run cargo check.

    If the input parses successfully, the tool will reveal the resulting syntax tree. If the input fails to parse, it will display the error message and the position where the failure occurred.

    cargo check
  5. Trigger custom warnings and errors in procedural macros

    master

    The lazy-static example demonstrates how to use syn to inspect macro input and emit diagnostic messages (warnings and errors) tied to specific token spans.

    • Warnings: Can be used to suggest better practices (e.g., warning against uncreative names like FOO).
    • Errors: Can be used to prevent compilation of invalid or nonsensical syntax (e.g., refusing to lazily initialize the unit type ()).
  6. Debug expanded procedural macro code

    master

    To inspect the code generated by your procedural macro, you can use cargo expand or the unstable Rust compiler flags.

    Using cargo-expand

    To see the expanded code for a crate:

    cargo expand

    To see the expanded code for a specific test case:

    cargo expand --test the_test_case

    Using rustc flags

    Alternatively, use the following command:

    cargo rustc -- -Zunstable-options -Zunpretty=expanded
  7. Configure Syn feature flags

    master

    Syn uses aggressive feature gating to optimize compile times. Enable only the features your procedural macro requires.

    Available Features

    • derive (enabled by default): Data structures for derive macro inputs (structs, enums, types).
    • full: Data structures for the full Rust syntax tree (items, expressions).
    • parsing (enabled by default): Ability to parse input tokens into syntax tree nodes.
    • printing (enabled by default): Ability to print syntax tree nodes back to Rust tokens.
    • visit: Trait for traversing a syntax tree.
    • visit-mut: Trait for traversing and mutating a syntax tree in place.
    • fold: Trait for transforming an owned syntax tree.
    • clone-impls (enabled by default): Clone implementations for syntax tree types.
    • extra-traits: Debug, Eq, PartialEq, and Hash implementations for syntax tree types.
    • proc-macro (enabled by default): Runtime dependency on the compiler's libproc_macro library.
  8. Configure Syn dependencies for derive macros

    master

    When building a procedural macro crate, ensure your Cargo.toml is configured with proc-macro = true and includes syn and quote as dependencies.

    # Cargo.toml
    [package]
    ...
    
    [lib]
    proc-macro = true
    
    [dependencies]
    syn = "3"
    quote = "1"
  9. Parse delimiters with `bracketed!`, `braced!`, and `parenthesized!`

    master

    When parsing Rust syntax, use these macros to handle delimited groups:

    • bracketed!: Parses content inside square brackets [...].
    • braced!: Parses content inside curly braces {...}.
    • parenthesized!: Parses content inside parentheses (...).
  10. Parse Syn syntax tree descriptions with `syn-codegen`

    master

    The syn-codegen crate provides canonical Rust data structures for parsing the machine-readable syn.json file provided with every Syn release. This allows you to programmatically inspect the Syn syntax tree (types, tokens, and their structures) in Rust code.

    To use it, include the syn.json file from the Syn repository and deserialize it into the Definitions struct using serde_json.

    use syn_codegen::Definitions;
    
    // Load the syn.json file
    const SYN: &str = include_str!("../../syn.json");
    
    fn main() {
        // Deserialize the JSON into the Definitions struct
        let defs: Definitions = serde_json::from_str(SYN).unwrap();
    
        // Iterate through the defined types
        for node in &defs.types {
            println!("syn::{}", node.ident);
        }
    }
  11. Report errors in procedural macros using `syn::Error`

    master

    When writing procedural macros, you should report errors by emitting a compile_error! invocation in the generated code rather than panicking.

    1. During initial parsing: Use the parse_macro_input! macro. It automatically converts syn::Result errors into compile_error! tokens.
    2. For errors occurring after parsing: Use Error::to_compile_error() or Error::into_compile_error() to manually convert a syn::Error into a proc_macro2::TokenStream containing the error message.
  12. Use the `Token!` macro for Rust tokens

    master

    Instead of remembering the specific type names for Rust punctuation, keywords, and delimiters, use the Token! macro. This macro expands to the correct token type.

    As a Type

    Use Token![...] in struct fields or for type annotations in parse methods.

    As an Expression

    Use Token![...] to:

    • Peek: input.peek(Token![...])
    • Parse: input.parse::<Token![...]>()?
    • Construct: let the_token = Token![...](span); (where span is a proc_macro2::Span)
    • Print: Use with the quote! macro: quote!(... #the_token ...)
    use syn::{Ident, Token};
    use syn::parse::{Parse, ParseStream, Result};
    
    // Example: Using Token! in a struct and parsing
    pub struct UnitStruct {
        struct_token: Token![struct],
        ident: Ident,
        semi_token: Token![;],
    }
    
    impl Parse for UnitStruct {
        fn parse(input: ParseStream) -> Result<Self> {
            let struct_token: Token![struct] = input.parse()?;
            let ident: Ident = input.parse()?;
            let semi_token = input.parse::<Token![;]>()?;
            Ok(UnitStruct {
                struct_token,
                ident,
                semi_token,
            })
        }
    }
    
    // Example: Using Token! for peeking and construction
    fn make_unit_struct(name: Ident) -> UnitStruct {
        let span = name.span();
        UnitStruct {
            struct_token: Token![struct](span),
            ident: name,
            semi_token: Token![;](span),
        }
    }
    
    fn check_struct(input: &ParseStream) -> bool {
        input.peek(Token![struct])
    }