binrw Documentation

repository·master·Indexed 21 days ago

https://github.com/jam1garner/binrw

A declarative binary data parsing and serialization library for Rust. It uses procedural macros (#[derive(BinRead)], #[derive(BinWrite)], and #[binrw]) to map binary formats to structs and enums, allowing developers to define formats using attributes for endianness, padding, alignment, and validation instead of manual byte manipulation. Supports no_std environments and works with any source implementing std::io::Read or std::io::Write.

Tokens
23.5K
Snippets
79
Records
99
Agent score
69%

What's inside binrw

  1. Overview of binrw

    master
    binrw is a library for writing maintainable and declarative binary data readers and writers in Rust. It uses procedural macros to generate efficient parsers and serializers for structs and enums, allowing you to define binary formats using attributes rather than manual byte manipulation.
  2. Pass arguments using Raw arguments

    master

    Raw arguments allow you to specify the Args type explicitly and receive all arguments into a single variable. This is particularly useful for argument forwarding, where a parent passes a single argument object down to multiple children.

    • Importing: Use #[br(import_raw(binding: Type))] or #[bw(import_raw(binding: Type))].
    • Passing: Use #[br(args_raw = value)] or #[bw(args_raw = value)].
    # use binrw::prelude::*;
    type Args = (u32, u16);
    
    #[derive(BinRead)]
    #[br(import_raw(args: Args))]
    struct Child {
        // ...
    }
    
    #[derive(BinRead)]
    #[br(import_raw(args: Args))]
    struct Middle {
        #[br(args_raw = args)]
        test: Child,
    }
    
    #[derive(BinRead)]
    struct Parent {
        count: u32,
        #[br(args(1, 2))]
        mid: Middle,
        #[br(args_raw = (1, 2))]
        mid2: Middle,
    }
  3. Use the temp directive to skip fields

    master

    The temp directive causes a field to be treated as a temporary variable instead of an actual field in the struct. The field is read from the stream but is removed from the struct definition generated by the #[binread] macro.

    Requirements:

    • This directive can only be used with the #[binread] macro; it will not work with #[derive(BinRead)].
    • The #[binread] attribute must be placed before other attributes like #[derive(Debug)] to prevent compilation errors.
    #[binread]
    #[br(big)]
    struct Test {
        #[br(temp)]
        len: u32,
    
        #[br(count = len)]
        data: Vec<u8>
    }
  4. Key features of binrw

    master

    binrw provides several capabilities for binary data handling:

    • Declarative Parsing: Uses #[derive(BinRead)] and #[derive(BinWrite)] to generate parsers/serializers.
    • Stream Support: Works with any source implementing std::io::Read or std::io::Write.
    • Attribute Directives: Handles magic numbers, byte ordering (endianness), padding, alignment, and data validation directly in attributes.
    • Reusable Types: Includes built-in types like NullString for null-terminated strings and FilePtr for data indirection via offsets.
    • Extensibility: Supports parsing third-party types via custom free functions or value maps.
    • Efficiency: Uses in-memory representations without requiring #[repr(C)] or #[repr(packed)].
    • Ergonomics: Attribute code is written as actual Rust code (not strings), providing better IDE support.
    • Environment: Supports no_std environments.
  5. Use the try directive to handle parsing failures

    master

    The try directive allows a field's parsing to fail gracefully instead of returning an error for the entire operation.

    When #[br(try)] is applied to a field:

    1. If parsing fails, the position of the reader is restored to where it was before the field was attempted.
    2. The value of the field is set to its type's Default value.

    This is particularly useful when working with Option<T> types where a failed parse should simply result in None.

    #[derive(BinRead)]
    struct MyType {
        #[br(try)]
        maybe_u32: Option<u32>
    }
  6. Pass arguments using Tuple-style (ordered) arguments

    master

    Tuple-style arguments are the most common way to pass data between nested structures. They are passed via args() in the parent and received via import() in the child. This pattern behaves similarly to a standard function call where arguments are matched by position.

    • For Reading (BinRead): Use #[br(import(name: type, ...))] and #[br(args(value, ...))].
    • For Writing (BinWrite): Use #[bw(import(name: type, ...))] and #[bw(args(value, ...))].
    # use binrw::prelude::*;
    #[derive(BinRead)]
    #[br(import(val1: u32, val2: &str))]
    struct Child {
        // ...
    }
    
    #[derive(BinRead)]
    struct Parent {
        val: u32,
        #[br(args(val + 3, "test"))]
        test: Child
    }
  7. Use directives for common binary patterns

    master

    You can use attributes like #[br], #[bw], and #[brw] to apply directives for handling common binary data tasks such as:

    • Endianness: e.g., #[br(little)] or #[br(big)].
    • Magic Numbers: e.g., #[br(magic = b"SHAP")].
    • Padding & Alignment: e.g., #[br(align_before = 0xA)].
    • Counting: e.g., #[br(count = some_field)] to read a specific number of elements into a Vec.
    • Assertions: e.g., #[brw(assert(condition))].
    • Calculated values: e.g., #[bw(try_calc(expression))].

    Directives can reference earlier fields by name. For tuple types, earlier fields are addressable using self_N, where N is the index of the field.

    use binrw::{prelude::*, io::Cursor, NullString};
    
    #[binrw]
    #[brw(big, magic = b"DOG", assert(name.len() != 0))]
    struct Dog {
        #[bw(try_calc(u8::try_from(bone_pile_count.len())))]
        bone_pile_count: u8,
    
        #[br(count = bone_pile_count)]
        bone_piles: Vec<u16>,
    
        #[br(align_before = 0xA)]
        name: NullString
    }
    
    let mut data = Cursor::new(b"DOG\x02\x00\x01\x00\x12\0\0Rudy\0");
    let dog = Dog::read(&mut data).unwrap();
    assert_eq!(dog.bone_piles, &[0x1, 0x12]);
    assert_eq!(dog.name.to_string(), "Rudy")
  8. Handle padding and alignment with directives

    master

    Use pad_before, pad_after, align_before, align_after, and pad_size_to to manage data structure alignment.

    • pad_before/pad_after: Skips a specific number of bytes.
    • align_before/align_after: Aligns the next read/write to a specific byte boundary.
    • pad_size_to: Ensures the reader/writer has advanced at least a certain number of bytes after the field is processed (useful for fixed-size buffers containing variable-length data).

    Note on Writing: When writing, padding directives write zeroes to the padded region, overwriting existing bytes. To leave existing bytes untouched, use seek_before with an appropriate SeekFrom value instead.

    #[derive(BinRead)]
    struct MyType {
        #[br(align_before = 4, pad_after = 1, align_after = 4)]
        str: NullString,
    
        #[br(pad_size_to = 0x10)]
        test: u64,
    
        #[br(seek_before = SeekFrom::End(-4))]
        end: u32,
    }
  9. Use the `magic` directive for validation and enum selection

    master

    The magic directive allows you to specify a value that must be present in the data stream.

    • For Structs: It acts as a validation check. If the data does not match the specified magic value, a BadMagic error is returned and the reader's position is reset to where parsing started.
    • For Enums: It is used to select the correct enum variant based on the value read from the stream.

    You can use byte strings (e.g., b"TEST") or float literals (e.g., 1.2f32) as magic values.

    #[derive(BinRead)]
    #[br(magic = b"TEST")]
    struct Test {
        val: u32
    }
    
    #[derive(BinRead)]
    enum Command {
        #[br(magic = 0u8)] Nop,
        #[br(magic = 1u8)] Jump { loc: u32 },
    }
  10. Use arguments to provide extra data for reading or writing

    master

    Arguments allow you to pass extra data required for parsing or writing an object that isn't present in the raw data stream.

    • The import and args directives define the type of BinRead::Args and BinWrite::Args respectively.
    • These types are used when calling BinRead::read_options or BinWrite::write_options.
    • Any field or import defined earlier in the structure can be referenced within the args directive.
  11. Pass arguments using Named arguments

    master

    Named arguments allow passing data using key-value pairs, similar to a struct literal. This is useful for container objects like Vec or when you want to provide optional or unordered arguments.

    • Syntax: Use curly braces {} for both import and args.
    • Features: Supports field init shorthand (e.g., count instead of count: count) and optional arguments with default values (e.g., other: u16 = 0).
    • Nesting: To nest named arguments, use the binrw::args! macro to construct the required argument object for the child.
    • Manual Implementation: For types that manually implement BinRead or BinWrite, you can implement the binrw::NamedArgs trait to support this syntax.
    # use binrw::prelude::*;
    #[derive(BinRead)]
    #[br(import {
        count: u32,
        other: u16 = 0 // optional argument
    })]
    struct Child {
        // ...
    }
    
    #[derive(BinRead)]
    struct Parent {
        count: u32,
    
        #[br(args {
            count, // field init shorthand
            other: 5
        })]
        test: Child,
    
        #[br(args { count: 3 })]
        test2: Child,
    }
  12. Optimize enum parsing by ordering variants

    master
    When binrw parses an enum, it attempts to parse each variant sequentially starting from the top of the definition. To minimize the number of failed parsing attempts, place the most frequently occurring enum variants at the top of the enum declaration.