binread

repository·master·Indexed 43 days ago

https://github.com/jam1garner/binread

A Rust crate for reading structs from binary data using procedural macros. It allows developers to define parsing rules declaratively via struct attributes, supporting magic numbers, assertions, endianness, and alignment. Version 2.2.0. Note: This crate has been superseded by binrw, which adds binary writing support.

Tokens
4.5K
Snippets
13
Records
20
Agent score
54%

What's inside binread

  1. Parse structs from binary data using BinRead

    master

    BinRead allows you to declaratively define binary parsing logic for Rust structs using the BinRead derive macro. You can use attributes starting with #[br(...)] to specify magic numbers, assertions, endianness, counts for collections, and alignment requirements.

    #[derive(BinRead)]
    #[br(magic = b"DOG", assert(name.len() != 0))]
    struct Dog {
        bone_pile_count: u8,
    
        #[br(big, count = bone_pile_count)]
        bone_piles: Vec<u16>,
    
        #[br(align_before = 0xA)]
        name: NullString
    }
    
    let mut reader = Cursor::new(b"DOG\x02\x00\x01\x00\x12\0\0Rudy\0");
    let dog: Dog = reader.read_ne().unwrap();
    assert_eq!(dog.bone_piles, &[0x1, 0x12]);
    assert_eq!(dog.name.into_string(), "Rudy")
  2. Quick start for adding a new directive to `BinRead`

    master

    To extend BinRead with a new directive, follow these steps to integrate it into the parser and code generation pipeline:

    1. Define the Keyword: Add a keyword for the new directive in parser::keywords.
    2. Define Meta-type: Define the meta-type in parser::attrs. If a new meta-type is required, add it to parser::meta_types and include tests.
    3. Handle Special Types: If the directive requires a special final type (e.g., CondEndian), add it to a new parser::types module and export it from parser::types. New types must implement parser::TrySet (or implement From/TryFrom to leverage trait generic implementations).
    4. Register Attributes: Add the new directive as a field to the relevant structs in parser::top_level_attrs and parser::field_level_attrs.
    5. Implement Validation: If the directive has invalid combinations with other directives that cannot be expressed via an enum (like ReadMode), add validation in:
      • FromInput::push_field (for immediate validation after field construction).
      • FromInput::validate (for validation after the entire struct is parsed).
    6. Emit Code: Use the new fields to emit code in the appropriate locations within codegen::read_options.
    7. Test Integration: Add new integration tests in the binread crate’s tests directory.
    8. Test Errors: If the directive generates new errors, add unit tests in parser::tests (in mod.rs) and add corresponding trybuild tests in the binread crate’s tests/ui directory. (Note: trybuild tests require a nightly compiler).
  3. Configure binread using attributes

    master

    Both the #[derive(BinRead)] and #[binread] macros support custom attributes to control how fields are parsed. You can use the following attribute prefixes:

    • binread
    • br

    These attributes are used to define parsing behavior, such as marking fields as temporary or specifying custom parsing logic. The macro will automatically clean up these attributes so they do not interfere with the rest of your code after expansion.

  4. Use the BinRead derive macro for declarative parsing

    master

    The most powerful feature of binread is the #[derive(BinRead)] macro. It allows you to define how a struct or tuple should be parsed from binary data using declarative attributes. This replaces imperative parsing code with a struct definition.

    Basic Usage

    #[derive(BinRead)]
    struct MyType {
        first: u32,
        second: u32
    }
    
    // Works with tuple types too
    #[derive(BinRead)]
    struct MyType2(u32, u32);

    Attributes and Precedence

    You can use attributes like #[br(big)] or #[br(little)] at the struct level or field level to override byte order.

    Precedence (highest to lowest):

    1. Field-level
    2. Variant-level (for enums)
    3. Top-level (struct/tuple level)
    4. Configured (endianness passed via read_options)
    5. Native endianness

    Generic Parsing

    You can build reusable parsers by using generics. To do this, you must bound the generic type T such that its Args are known (often by bounding <T as BinRead>::Args to ()).

    #[derive(BinRead)]
    struct U32CountVec<T: BinRead<Args=()>> {
        count: u32,
        #[br(count = count)]
        data: Vec<T>,
    }
    #[derive(BinRead)]
    #[br(magic = b"DOG", assert(name.len() != 0))]
    struct Dog {
        bone_pile_count: u8,
    
        #[br(big, count = bone_pile_count)]
        bone_piles: Vec<u16>,
    
        #[br(align_before = 0xA)]
        name: NullString
    }
  5. Configure parsing behavior with ReadOptions

    master

    The ReadOptions struct allows you to configure how a type is read during the parsing process. You can control byte order (endianness), limit the number of items read, or specify a starting offset in the byte stream.

    Available fields:

    • endian: Specifies the Endianness (e.g., Little or Big endian) to use for the read operation.
    • count: An Option<usize> that, if set to Some(n), limits the read to exactly n items.
    • offset: A u64 representing the byte offset from the start of the stream where reading should begin.

    Note: If the debug_template feature is enabled, the following additional fields are available:

    • dont_output_to_template: A boolean flag.
    • variable_name: An Option<&'static str> used for debugging template output.
    // Example of initializing ReadOptions
    let options = ReadOptions {
        endian: Endian::Little,
        count: Some(10),
        offset: 0,
        ..Default::default()
    };
  6. Implement the BinRead trait

    master

    The BinRead trait defines how a type is read from a byte stream. It is implemented for most primitives and simple collections.

    Core Methods

    • read<R: Read + Seek>(reader: &mut R) -> BinResult<Self>: A simple shortcut that assumes no arguments are needed (Args = ()). It panics if args_default() is not implemented.
    • read_args<R: Read + Seek>(reader: &mut R, args: Self::Args) -> BinResult<Self>: Reads the type using the provided arguments.
    • read_options<R: Read + Seek>(reader: &mut R, options: &ReadOptions, args: Self::Args) -> BinResult<Self>: The primary method for reading with specific configuration (like endianness) and arguments.

    Associated Types

    • type Args: Any + Copy: The type of arguments required to parse this type. For types that require no arguments, use the unit type ().
  7. Retrieve custom error data using custom_err

    master

    If a parsing error was triggered by a Custom variant, you can attempt to downcast the contained error to a specific type using the custom_err<T>() method. This is useful when you want to pass specific error context from your own logic through the binread error pipeline.

    Returns Some(&T) if the error is a Custom variant and matches the requested type T, otherwise returns None.

    // Example of retrieving a custom error type
    if let Err(e) = result {
        if let Some(my_err) = e.custom_err::<MySpecificError>() {
            println!("Caught specific error: {:?}", my_err);
        }
    }
  8. Convert `NullString` and `NullWideString` to standard strings

    master

    Both NullString and NullWideString provide methods to convert their internal byte/word buffers into standard Rust String types:

    • into_string(): Returns a String using lossy conversion (String::from_utf8_lossy for NullString and String::from_utf16_lossy for NullWideString). This is useful when you want to ensure the conversion succeeds even if the binary data contains invalid UTF sequences.
    • into_string_lossless(): Returns a Result<String, E>, where E is the specific error type for the encoding (FromUtf8Error or FromUtf16Error). This is preferred when you need to handle invalid encoding as an error.
  9. Read null-terminated UTF-16 strings with `NullWideString`

    master

    NullWideString is a specialized type for reading null-terminated UTF-16 (wide) strings. It reads u16 values until a 0u16 is encountered.

    Note: This type is endian-dependent on a per-character basis. You must use the appropriate endianness method (e.g., read_le or read_be) when reading from the stream. The null terminator is consumed from the reader but not included in the data.

    You can convert the result into a String using into_string() (lossy UTF-16 conversion) or into_string_lossless() (returns a Result).

    use binread::{BinRead, BinReaderExt, NullWideString, io::Cursor};
    
    const WIDE_STRINGS: &[u8] = b"w\0i\0d\0e\0 \0s\0t\0r\0i\0n\0g\0s\0\0\0";
    
    let mut wide_strings = Cursor::new(WIDE_STRINGS);
    
    // Use read_le for little-endian wide strings
    assert_eq!(
        wide_strings.read_le::<NullWideString>().unwrap().into_string(),
        "wide strings"
    );
  10. Use the #[binread] attribute macro

    master

    The #[binread] attribute macro provides an alternative way to apply binread logic to a type. It functions similarly to the BinRead derive macro but is implemented as an attribute macro, which can be useful for more complex transformations or when specific attribute processing is required during the parsing phase.

    #[binread]
    struct MyData {
        field1: u32,
        field2: u8,
    }