binrw Documentation
repository·master·Indexed 21 days ago
https://github.com/jam1garner/binrwA declarative binary data parsing and serialization library for Rust. It uses procedural macros (#[derive(BinRead)], #[derive(BinWrite)], and #[binrw]) to map binary formats to structs and enums, allowing developers to define formats using attributes for endianness, padding, alignment, and validation instead of manual byte manipulation. Supports no_std environments and works with any source implementing std::io::Read or std::io::Write.
What's inside binrw
- binrw is a library for writing maintainable and declarative binary data readers and writers in Rust. It uses procedural macros to generate efficient parsers and serializers for structs and enums, allowing you to define binary formats using attributes rather than manual byte manipulation.
Pass arguments using Raw arguments
masterRaw arguments allow you to specify the
Argstype explicitly and receive all arguments into a single variable. This is particularly useful for argument forwarding, where a parent passes a single argument object down to multiple children.- Importing: Use
#[br(import_raw(binding: Type))]or#[bw(import_raw(binding: Type))]. - Passing: Use
#[br(args_raw = value)]or#[bw(args_raw = value)].
# use binrw::prelude::*; type Args = (u32, u16); #[derive(BinRead)] #[br(import_raw(args: Args))] struct Child { // ... } #[derive(BinRead)] #[br(import_raw(args: Args))] struct Middle { #[br(args_raw = args)] test: Child, } #[derive(BinRead)] struct Parent { count: u32, #[br(args(1, 2))] mid: Middle, #[br(args_raw = (1, 2))] mid2: Middle, }- Importing: Use
Use the temp directive to skip fields
masterThe
tempdirective causes a field to be treated as a temporary variable instead of an actual field in the struct. The field is read from the stream but is removed from the struct definition generated by the#[binread]macro.Requirements:
- This directive can only be used with the
#[binread]macro; it will not work with#[derive(BinRead)]. - The
#[binread]attribute must be placed before other attributes like#[derive(Debug)]to prevent compilation errors.
#[binread] #[br(big)] struct Test { #[br(temp)] len: u32, #[br(count = len)] data: Vec<u8> }- This directive can only be used with the
Key features of binrw
masterbinrw provides several capabilities for binary data handling:
- Declarative Parsing: Uses
#[derive(BinRead)]and#[derive(BinWrite)]to generate parsers/serializers. - Stream Support: Works with any source implementing
std::io::Readorstd::io::Write. - Attribute Directives: Handles magic numbers, byte ordering (endianness), padding, alignment, and data validation directly in attributes.
- Reusable Types: Includes built-in types like
NullStringfor null-terminated strings andFilePtrfor data indirection via offsets. - Extensibility: Supports parsing third-party types via custom free functions or value maps.
- Efficiency: Uses in-memory representations without requiring
#[repr(C)]or#[repr(packed)]. - Ergonomics: Attribute code is written as actual Rust code (not strings), providing better IDE support.
- Environment: Supports
no_stdenvironments.
- Declarative Parsing: Uses
Use the try directive to handle parsing failures
masterThe
trydirective allows a field's parsing to fail gracefully instead of returning an error for the entire operation.When
#[br(try)]is applied to a field:- If parsing fails, the position of the reader is restored to where it was before the field was attempted.
- The value of the field is set to its type's
Defaultvalue.
This is particularly useful when working with
Option<T>types where a failed parse should simply result inNone.#[derive(BinRead)] struct MyType { #[br(try)] maybe_u32: Option<u32> }Pass arguments using Tuple-style (ordered) arguments
masterTuple-style arguments are the most common way to pass data between nested structures. They are passed via
args()in the parent and received viaimport()in the child. This pattern behaves similarly to a standard function call where arguments are matched by position.- For Reading (
BinRead): Use#[br(import(name: type, ...))]and#[br(args(value, ...))]. - For Writing (
BinWrite): Use#[bw(import(name: type, ...))]and#[bw(args(value, ...))].
# use binrw::prelude::*; #[derive(BinRead)] #[br(import(val1: u32, val2: &str))] struct Child { // ... } #[derive(BinRead)] struct Parent { val: u32, #[br(args(val + 3, "test"))] test: Child }- For Reading (
Use directives for common binary patterns
masterYou can use attributes like
#[br],#[bw], and#[brw]to apply directives for handling common binary data tasks such as:- Endianness: e.g.,
#[br(little)]or#[br(big)]. - Magic Numbers: e.g.,
#[br(magic = b"SHAP")]. - Padding & Alignment: e.g.,
#[br(align_before = 0xA)]. - Counting: e.g.,
#[br(count = some_field)]to read a specific number of elements into aVec. - Assertions: e.g.,
#[brw(assert(condition))]. - Calculated values: e.g.,
#[bw(try_calc(expression))].
Directives can reference earlier fields by name. For tuple types, earlier fields are addressable using
self_N, whereNis the index of the field.use binrw::{prelude::*, io::Cursor, NullString}; #[binrw] #[brw(big, magic = b"DOG", assert(name.len() != 0))] struct Dog { #[bw(try_calc(u8::try_from(bone_pile_count.len())))] bone_pile_count: u8, #[br(count = bone_pile_count)] bone_piles: Vec<u16>, #[br(align_before = 0xA)] name: NullString } let mut data = Cursor::new(b"DOG\x02\x00\x01\x00\x12\0\0Rudy\0"); let dog = Dog::read(&mut data).unwrap(); assert_eq!(dog.bone_piles, &[0x1, 0x12]); assert_eq!(dog.name.to_string(), "Rudy")- Endianness: e.g.,
Handle padding and alignment with directives
masterUse
pad_before,pad_after,align_before,align_after, andpad_size_toto manage data structure alignment.pad_before/pad_after: Skips a specific number of bytes.align_before/align_after: Aligns the next read/write to a specific byte boundary.pad_size_to: Ensures the reader/writer has advanced at least a certain number of bytes after the field is processed (useful for fixed-size buffers containing variable-length data).
Note on Writing: When writing, padding directives write zeroes to the padded region, overwriting existing bytes. To leave existing bytes untouched, use
seek_beforewith an appropriateSeekFromvalue instead.#[derive(BinRead)] struct MyType { #[br(align_before = 4, pad_after = 1, align_after = 4)] str: NullString, #[br(pad_size_to = 0x10)] test: u64, #[br(seek_before = SeekFrom::End(-4))] end: u32, }Use the `magic` directive for validation and enum selection
masterThe
magicdirective allows you to specify a value that must be present in the data stream.- For Structs: It acts as a validation check. If the data does not match the specified magic value, a
BadMagicerror is returned and the reader's position is reset to where parsing started. - For Enums: It is used to select the correct enum variant based on the value read from the stream.
You can use byte strings (e.g.,
b"TEST") or float literals (e.g.,1.2f32) as magic values.#[derive(BinRead)] #[br(magic = b"TEST")] struct Test { val: u32 } #[derive(BinRead)] enum Command { #[br(magic = 0u8)] Nop, #[br(magic = 1u8)] Jump { loc: u32 }, }- For Structs: It acts as a validation check. If the data does not match the specified magic value, a
Use arguments to provide extra data for reading or writing
masterArguments allow you to pass extra data required for parsing or writing an object that isn't present in the raw data stream.
- The
importandargsdirectives define the type ofBinRead::ArgsandBinWrite::Argsrespectively. - These types are used when calling
BinRead::read_optionsorBinWrite::write_options. - Any field or
importdefined earlier in the structure can be referenced within theargsdirective.
- The
Pass arguments using Named arguments
masterNamed arguments allow passing data using key-value pairs, similar to a struct literal. This is useful for container objects like
Vecor when you want to provide optional or unordered arguments.- Syntax: Use curly braces
{}for bothimportandargs. - Features: Supports field init shorthand (e.g.,
countinstead ofcount: count) and optional arguments with default values (e.g.,other: u16 = 0). - Nesting: To nest named arguments, use the
binrw::args!macro to construct the required argument object for the child. - Manual Implementation: For types that manually implement
BinReadorBinWrite, you can implement thebinrw::NamedArgstrait to support this syntax.
# use binrw::prelude::*; #[derive(BinRead)] #[br(import { count: u32, other: u16 = 0 // optional argument })] struct Child { // ... } #[derive(BinRead)] struct Parent { count: u32, #[br(args { count, // field init shorthand other: 5 })] test: Child, #[br(args { count: 3 })] test2: Child, }- Syntax: Use curly braces
Optimize enum parsing by ordering variants
masterWhenbinrwparses an enum, it attempts to parse each variant sequentially starting from the top of the definition. To minimize the number of failed parsing attempts, place the most frequently occurring enum variants at the top of the enum declaration.