DOCSEARCHbyAlgolia
Submit Docs

nom

repository·main·Indexed 27 days ago

https://github.com/rust-bakery/nom

A byte-oriented, zero-copy parser combinators library for Rust designed to build safe, fast, and memory-efficient parsers for binary, text, and programming language formats. Version 8.0.0 supports byte, bit, and string-oriented parsing, streaming data, and custom input types. It provides a wide array of combinators for sequences, choices, repetitions, and transformations, as well as specialized tools for parsing integers with configurable endianness and length-prefixed binary data.

Tokens
17.4K
Snippets
60
Records
101
Agent score
94%

What's inside nom

  1. Overview of nom technical features

    main

    nom is a high-performance parser combinators library with the following characteristics:

    • Byte-oriented: Works primarily on &[u8] slices.
    • Bit-oriented: Can address byte slices as bit streams.
    • String-oriented: Supports UTF-8 strings.
    • Zero-copy: Returns slices of the input instead of copying data.
    • Streaming: Handles partial data and detects when more input is required.
    • Descriptive/Custom Errors: Supports aggregating error codes and custom error types.
    • Safe and Fast: Leverages Rust's memory safety and is optimized for speed, often outperforming handwritten C parsers.

  2. Explore projects using nom

    main

    nom is a widely used parser combinator library in Rust. It is used to implement parsers for a vast range of formats, including:

    • Text file formats: CSV, INI, ISO 8601 dates, and more.
    • Programming languages: PHP, Lua, Python, SQL, and Wasm.
    • Interface definition formats: Thrift.
    • Audio, video, and image formats: GIF, MIDI, WAVE, and Matroska (MKV).
    • Document formats: TAR and GZ.
    • Cryptographic formats: X.509.
    • Network protocol formats: HTTP, DNS, TLS, DHCP, and many others.
    • Misc formats: Game Boy ROM, Version Numbers, and URI.

    If you are looking for inspiration or want to see how specific formats are parsed, you can explore the various open-source projects listed in the repository documentation.

  3. Improve error usability with nom_locate and nom-supreme

    main

    If the built-in error types are insufficient, consider these external crates:

    • nom_locate: Wraps input data in a Span type that tracks line and column information.
    • nom-supreme: Provides the ErrorTree<I> type. Unlike VerboseError, which only tracks the last branch tried in an alt combinator, ErrorTree accumulates errors from all branches tried, allowing you to explore the entire parsing attempt.

  4. Build parsers using combinators

    main

    Parsers are built bottom-up. Start by writing small functions for the smallest elements of your format, then assemble them into complex parsers using combinators like preceded, tag, take_while1, etc.

    Note: When using tuples of parsers, they are not FnMut and must be wrapped in the .parse() method to be used like other parsers.

    // Example of combining small parsers into a request line parser
let method = take_while1(is_alpha);
let space = take_while1(|c| c == ' ');
let url = take_while1(|c| c!= ' ');
let is_version = |c| c >= b'0' && c <= b'9' || c == b'.';
let http = tag("HTTP/");
let version = take_while1(is_version);
let line_ending = tag("\r\n");

let http_version = preceded(http, version);

fn request_line(i: &[u8]) -> IResult<&[u8], Request> {
  let (input, (method, _, url, _, version, _)) = 
    (method, space, url, space, http_version, line_ending).parse(i)?;

  Ok((input, Request { method, url, version }))
}

  5. Implement custom input types for nom parsers

    main

    While nom primarily uses &[u8] and &str, you can use any custom type as input by implementing a specific set of traits. This allows you to parse complex structures like token lists or types that carry metadata (e.g., line and column information via nom_locate).

    To use a custom type MyInput in a parser with the signature fn parser(i: MyInput) -> IResult<MyInput, Output>, you must implement the relevant traits for both the input container (MyInput) and its constituent elements (MyItem).

  6. Consume whitespace around a parser

    main

    You can create a wrapper combinator to automatically consume leading and trailing whitespace around an inner parser using delimited with multispace0.

    • To consume both leading and trailing whitespace: Use delimited(multispace0, inner, multispace0).
    • To consume only trailing whitespace: Use terminated(inner, multispace0).
    • To consume only leading whitespace: Use preceded(multispace0, inner).
    use nom::{
  IResult,
  Parser,
  error::ParseError,
  sequence::delimited,
  character::complete::multispace0,
};

/// A combinator that takes a parser `inner` and produces a parser that also consumes both leading and 
/// trailing whitespace, returning the output of `inner`.
pub fn ws<'a, O, E: ParseError<&'a str>, F>(
    inner: F,
) -> impl Parser<&'a str, Output = O, Error = E>
where
    F: Parser<&'a str, Output = O, Error = E>,
{
    delimited(multispace0, inner, multispace0)
}

  9. Organize parser code in modules

    main

    To maintainable and testable code, separate your parsing logic into its own module rather than embedding it directly within your main application logic. This allows you to test small parsing functions independently. A common pattern is to define a pub mod parser; in src/lib.rs and implement the logic in src/parser.rs.

    // src/lib.rs
pub mod parser;

// src/parser.rs
use nom::IResult;
use nom::number::complete::be_u16;
use nom::bytes::complete::take;

pub fn length_value(input: &[u8]) -> IResult<&[u8],&[u8]> {
    let (input, length) = be_u16(input)?;
    take(length)(input)
}

  10. Implement FromStr using nom

    main

    To implement the std::str::FromStr trait using nom, you must handle the conversion from nom's reference-based results to owned types (like String), as FromStr requires the returned type to be owned. Use the Finish trait to convert IResult into a standard Result that can be returned from from_str.

    use nom::{
  IResult, Parser, Finish, error::Error,
  bytes::complete::{tag, take_while},
};
use std::str::FromStr;

fn parse_name(input: &str) -> IResult<&str, &str> {
  let (i, _) = tag("Hello, ").parse(input)?;
  let (i, name) = take_while(|c:char| c.is_alphabetic())(i)?;
  let (i, _) = tag("!")(i)?;

  Ok((i, name))
}

#[derive(Debug)]
pub struct Name(pub String);

impl FromStr for Name {
  type Err = Error<String>;

  fn from_str(s: &str) -> Result<Self, Self::Err> {
      match parse_name(s).finish() {
          Ok((_remaining, name)) => Ok(Name(name.to_string())),
          Err(Error { input, code }) => Err(Error {
              input: input.to_string(),
              code,
          })
      }
  }
}

fn main() {
  println!("parsed: {:?}", "Hello, nom!".parse::<Name>());
  println!("parsed: {:?}", "Hello, 123!".parse::<Name>());
}

  11. Debug parsers using `dbg_dmp`

    main

    To observe a parser's input and output during development, use the dbg_dmp function. If a parser fails, dbg_dmp will print a hexdump of the input at the point of failure, helping you identify exactly where the parser went wrong.

    fn f(i: &[u8]) -> IResult<&[u8], &[u8]> {
    dbg_dmp(tag("abcd"), "tag")(i)
}

let a = &b"efghijkl"[..];

// Will print a hexdump of the input if 'tag' fails
f(a);

  12. Configure nom features for no_std environments

    main

    By default, nom enables std and alloc features. If you are working in a no_std environment, you must disable default features and explicitly enable alloc if you need combinators that require memory allocation (such as many0).

    [dependencies.nom]
version = "8"
default-features = false
features = ["alloc"]